quickfix-developers

[Quickfix-developers] C++ Quickfix Memory keeps growing

[Michele C. <mch...@gm...>]

Hi I've seen previous threads similar to this but not seen what the fix is.
I am currently using Quickfix 1.12.4, when i stress the system, i.e send
about 20000 trades through QF, my application memory grows very rapidly.
Anybody experienced the same thing?

Re: [Quickfix-developers] C++ Quickfix Memory keeps growing

[Alexey Z. <ale...@in...>]

That's probably the maps keeping messages for resend (store factory).
Take a look into the /PersistMessages /configuration parameter.
As alternative you can write your own store factory.

Regards,
	Alexey Zubko





Michele Chitson wrote:
> Hi I've seen previous threads similar to this but not seen what the 
> fix is. I am currently using Quickfix 1.12.4, when i stress the 
> system, i.e send about 20000 trades through QF, my application memory 
> grows very rapidly. Anybody experienced the same thing? 

Re: [Quickfix-developers] C++ Quickfix Memory keeps growing

[Djalma R. d. S. F. <drs...@gm...>]

Yes, this is a pseudo memory leak.

Keeping all outbound sequence numbers in maps wasn't a smart solution for
the FileStore.

To avoid development effort, you can use the other available store factories
with their pros and cons (memory, MySQL, ODBC etc).

But, what sounds to me as a solution to save the current FileStore from this
problem is the .header file in a binary format (a sequence of fixed blocks /
serialized structs). In this file QF would append and seek on demmand the
seqnum's position data (offset | message length) in corresponding .body file
(additionally a cache for the LRU would be nice, but not all data in
memory).



On Wed, Jan 21, 2009 at 5:42 PM, Alexey Zubko <
ale...@in...> wrote:

> QuickFIX Documentation:
> http://www.quickfixengine.org/quickfix/doc/html/index.html
> QuickFIX Support: http://www.quickfixengine.org/services.html
>
> That's probably the maps keeping messages for resend (store factory).
> Take a look into the /PersistMessages /configuration parameter.
> As alternative you can write your own store factory.
>
> Regards,
>        Alexey Zubko
>
>
>
>
>
> Michele Chitson wrote:
> > Hi I've seen previous threads similar to this but not seen what the
> > fix is. I am currently using Quickfix 1.12.4, when i stress the
> > system, i.e send about 20000 trades through QF, my application memory
> > grows very rapidly. Anybody experienced the same thing?
>
>
> ------------------------------------------------------------------------------
> This SF.net email is sponsored by:
> SourcForge Community
> SourceForge wants to tell your story.
> http://p.sf.net/sfu/sf-spreadtheword
> _______________________________________________
> Quickfix-developers mailing list
> Qui...@li...
> https://lists.sourceforge.net/lists/listinfo/quickfix-developers
>

Re: [Quickfix-developers] C++ Quickfix Memory keeps growing

[Djalma R. d. S. F. <drs...@gm...>]

Yes, this is a pseudo memory leak.

Keeping all outbound sequence numbers in maps wasn't a smart solution for
the FileStore.

To avoid development effort, you can use the other available store factories
with their pros and cons (memory, MySQL, ODBC etc).

But, what sounds to me as a solution to save the current FileStore from this
problem is the .header file in a binary format (a sequence of fixed blocks /
serialized structs). In this file QF would append and seek on demmand the
seqnum's position data (offset | message length) in corresponding .body file
(additionally a cache for the LRU would be nice, but not all data in
memory).

On Wed, Jan 21, 2009 at 5:42 PM, Alexey Zubko <
ale...@in...> wrote:

> QuickFIX Documentation:
> http://www.quickfixengine.org/quickfix/doc/html/index.html
> QuickFIX Support: http://www.quickfixengine.org/services.html
>
> That's probably the maps keeping messages for resend (store factory).
> Take a look into the /PersistMessages /configuration parameter.
> As alternative you can write your own store factory.
>
> Regards,
>        Alexey Zubko
>
>
>
>
>
> Michele Chitson wrote:
> > Hi I've seen previous threads similar to this but not seen what the
> > fix is. I am currently using Quickfix 1.12.4, when i stress the
> > system, i.e send about 20000 trades through QF, my application memory
> > grows very rapidly. Anybody experienced the same thing?
>
>
> ------------------------------------------------------------------------------
> This SF.net email is sponsored by:
> SourcForge Community
> SourceForge wants to tell your story.
> http://p.sf.net/sfu/sf-spreadtheword
> _______________________________________________
> Quickfix-developers mailing list
> Qui...@li...
> https://lists.sourceforge.net/lists/listinfo/quickfix-developers
>

Re: [Quickfix-developers] C++ Quickfix Memory keeps growing

[Alexey Z. <ale...@in...>]

I would make a change to populate the map from the header file only 
during the resend request.
The binary format for header could speed up the resend process, but I 
don't think it's really matter.

Regards,
	Alexey Zubko




Djalma Rosa dos Santos Filho wrote:
> Yes, this is a pseudo memory leak.
>
> Keeping all outbound sequence numbers in maps wasn't a smart solution 
> for the FileStore.
>
> To avoid development effort, you can use the other available store 
> factories with their pros and cons (memory, MySQL, ODBC etc).
>
> But, what sounds to me as a solution to save the current FileStore 
> from this problem is the .header file in a binary format (a sequence 
> of fixed blocks / serialized structs). In this file QF would append 
> and seek on demmand the seqnum's position data (offset | message 
> length) in corresponding .body file (additionally a cache for the LRU 
> would be nice, but not all data in memory).
>
> On Wed, Jan 21, 2009 at 5:42 PM, Alexey Zubko 
> <ale...@in... 
> <mailto:ale...@in...>> wrote:
>
>     QuickFIX Documentation:
>     http://www.quickfixengine.org/quickfix/doc/html/index.html
>     QuickFIX Support: http://www.quickfixengine.org/services.html
>
>     That's probably the maps keeping messages for resend (store factory).
>     Take a look into the /PersistMessages /configuration parameter.
>     As alternative you can write your own store factory.
>
>     Regards,
>            Alexey Zubko
>
>
>
>
>
>     Michele Chitson wrote:
>     > Hi I've seen previous threads similar to this but not seen what the
>     > fix is. I am currently using Quickfix 1.12.4, when i stress the
>     > system, i.e send about 20000 trades through QF, my application
>     memory
>     > grows very rapidly. Anybody experienced the same thing?
>
>     ------------------------------------------------------------------------------
>     This SF.net email is sponsored by:
>     SourcForge Community
>     SourceForge wants to tell your story.
>     http://p.sf.net/sfu/sf-spreadtheword
>     _______________________________________________
>     Quickfix-developers mailing list
>     Qui...@li...
>     <mailto:Qui...@li...>
>     https://lists.sourceforge.net/lists/listinfo/quickfix-developers
>
>