#1 Memory grows fast

[bristotti]

The problem is in the way C++ allocates memory for keeping field order ( struct message_order ):

quickfix\src\C++\MessageSorters.cpp:

void message_order::setOrder( int size, const int order[] )
{ QF_STACK_PUSH(message_order::operator=)

if(size < 1) return;
m_largest = m_delim = order[0];

int* fields = new int[ size + 1 ];
fields[ 1 ] = m_delim;
// collect all fields and find the largest field number
int i;
for ( i = 2; i <= size; ++i )
{
int field = order[i-1];
m_largest = m_largest > field ? m_largest : field;
fields[ i ] = field;
}

// populate array with field number as key and position as value
m_groupOrder = new int[ m_largest + 1 ];
memset( m_groupOrder, 0, ( m_largest + 1 ) * sizeof( int ) );
for ( i = 1; i <= size; ++i )
m_groupOrder[ fields[ i ] ] = i;
delete [] fields;

QF_STACK_POP
}

Let's suppose that you have a group with only two custom fields ( 10000, 10001 ) in a give fix message. It doesn't make any sence to allocate a int[] vector size of the largest field (10001 in this case) for each entry of this group. Memory will grow really fast!!!!!

Attached an example of a market data snapshot message with more than 35000 trades. QuickFIX will allocate 6032(largest field) x 35000(trades) x 4(size of int) bytes, almost 800Mb.