[Quickfix-users] Possible bug in the Parser when receiving huge amounts through th e socket

[<em...@co...>]

Here is the scenario. 

At the moment the socket buffer in the parser is 4096. So, I have recently
generated a lot of traffic so I constantly receive of 4096 bytes on the
socket and have encountered the following problem.

The end of the incoming buffer (m_buffer) of 4096 is as follows (some of the
values have been substituted to hide trading details):

8=FIX.4.2
9=291
35=8
34=156110
369=31450
52=20041209-15:35:32.687
49=BLA
50=
G
56=D33030N
60=20041209-15:35:32
59=0
55=GE
54=1
48=BLA000060467
107=BLF5
1
67=FUT
44=9740.00
41=0
40=2
39=0
38=10
37=20041209004077
151=10
150=0
20=0
9
717=67960
17=00687120041209093532
14=0
11=00067960
6=0
432=20041209
1=123456
78
10=097
8=F

As we can see, when the parser gets to parsing this part of the buffer, it
will extract the length of the valid message which is 294 (9=291) and in
turn send of the message for processing into the application code, etc.
After, it has processed the message the buffer then looks like:

8=F

This is because the rest of the message is resting on the socket to be read
so a valid FIX message can be composed.

As you can see there is a 8=F at the end which was received into the buffer,
and more data is left over on the socket to be read into the buffer. But, at
the moment what the parser will do is make 8=F into a fix message as this is
where the buffer is ending, and will try and extract length from it, and
this generates a series of exceptions (FieldConvertError, MessageParseError,
FieldNotFound and InvalidMessage).


Once the parser tries to parse this message, and throws its exceptions, it
will get around to reading the socket again and receiving the following into
buffer:

IX.4.2
9=349
35=8
34=156111
369=31450
52=20041209-15:35:32.692
49=BLA
50=G
5
6=D33030N
60=20041209-15:35:32
59=0
55=GE
54=1
48=CME000060467
107=BLF5
167=
FUT
44=9740.00
41=0
40=2
39=2
38=10
37=20041209004077
337=0C0
32=10
31=9740.
00
151=0
150=2
20=0
9717=67960
17=00687220041209093532TN0002843
75=20041209

14=0
11=00067960
375=BLA030A1
6=0
432=20041209
1=12345678
10=178


Once this is received the m_buffer is "8=F" and m_readBuffer is as above.
When the parser appends m_readBuffer to m_buffer the message looks like:

8=FIX.4.2
9=349
35=8
34=156111
369=31450
52=20041209-15:35:32.692
49=BLA
50=
G
56=D33030N
60=20041209-15:35:32
59=0
55=GE
54=1
48=CME000060467
107=BLF5
1
67=FUT
44=9740.00
41=0
40=2
39=2
38=10
37=20041209004077
337=0C0
32=10
31=97
40.00
151=0
150=2
20=0
9717=67960
17=00687220041209093532TN0002843
75=200412
09
14=0
11=00067960
375=BLA030A1
6=0
432=20041209
1=12345678
10=178


a COMPLETE message as it should be.

Now surely, if this kind of scenario is encountered the parser should not
even attempt to parse the message, as it is incomplete. I think that there
needs to be an extra safeguard in the parser that needs to check presence of
field 10 (Standard Trailer) in order to verify that it is a complete
message.

If anyone has any ideas how to solve this, please be my guess as it has
taken my over 2 days just to track this problem down.

Thanks a lot

Emir

---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.766 / Virus Database: 513 - Release Date: 9/17/2004