Re: [Quickfix-developers] C# Memory growing fast

[Bristotti, F. <fer...@cr...>]

Shane,
 
The problem is in the way C++ allocates memory for keeping field order (
struct message_order ):
 
quickfix\src\C++\MessageSorters.cpp:
 
void message_order::setOrder( int size, const int order[] )
{ QF_STACK_PUSH(message_order::operator=)
 
  if(size < 1) return;
  m_largest = m_delim = order[0];
 
  int* fields = new int[ size + 1 ];
  fields[ 1 ] = m_delim;
  // collect all fields and find the largest field number
  int i;
  for ( i = 2; i <= size; ++i )
  {
    int field = order[i-1];
    m_largest = m_largest > field ? m_largest : field;
    fields[ i ] = field;
  }
 
  // populate array with field number as key and position as value
  m_groupOrder = new int[ m_largest + 1 ];
  memset( m_groupOrder, 0, ( m_largest + 1 ) * sizeof( int ) );
  for ( i = 1; i <= size; ++i )
    m_groupOrder[ fields[ i ] ] = i;
  delete [] fields;
 
  QF_STACK_POP
}
 
Let's suppose that you have a group with only two custom fields ( 10000,
10001 ) in a give fix message. To me, it doesn't make any sence to
allocate a int[] vector size of the largest field (10001 in this case)
for each entry of this group. Memory will grow really fast!!!!!
 
Fernando Bristotti

________________________________

From: Shane Trotter [mailto:str...@co...] 
Sent: Tuesday, March 03, 2009 12:15 PM
To: Bristotti, Fernando
Cc: qui...@li...
Subject: Re: [Quickfix-developers] C# Memory growing fast


I believe with the .NET version you must dispose each message after you
have used it.  Is it a requirement in your application to maintain a
reference to all 35,000 trade messages?

--
Shane Trotter
Connamara Systems, LLC


On Tue, Mar 3, 2009 at 9:06 AM, Bristotti, Fernando
<fer...@cr...> wrote:


	QuickFIX Documentation:
http://www.quickfixengine.org/quickfix/doc/html/index.html
	QuickFIX Support: http://www.quickfixengine.org/services.html
	
	
	

	Hi, 

	I am currently using QuickFIX 1.12.4. Memory grows fast when
QuickFIX process a market data snapshot message, for instance the day
I've received a snapshot with about 35000 trades (attached), memory
grows to about 1.0G and my application crashes.

	Taking a look on QuickFIX source code I could see that struct
message_order ( MessageSorters.cpp ) allocates for each group a vector
int[] size of the largest field in the group even if the group contains
only two fields. The largest field in NoMDEntry group for BM&F market
data is a custom field = 6032, thus for processing the message above
QuickFIX allocates 6032 x ( 35000 trades ) x ( 4 bytes ).

	I have changed the code in order to allocate a vector int[] size
of the number of fields in the group, any suggestion? 

	Thanks, 
	Fernando 

	

	
========================================================================
======
	Please access the attached hyperlink for an important electronic
communications disclaimer: 
	
	http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html
	
========================================================================
======

	
------------------------------------------------------------------------
------
	Open Source Business Conference (OSBC), March 24-25, 2009, San
Francisco, CA
	-OSBC tackles the biggest issue in open source: Open Sourcing
the Enterprise
	-Strategies to boost innovation and cut costs with open source
participation
	-Receive a $600 discount off the registration fee with the
source code: SFAD
	http://p.sf.net/sfu/XcvMzF8H
	_______________________________________________
	Quickfix-developers mailing list
	Qui...@li...
	https://lists.sourceforge.net/lists/listinfo/quickfix-developers
	



==============================================================================
Please access the attached hyperlink for an important electronic communications disclaimer: 

http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html
==============================================================================