RE: [Quickfix-users] Possible bug in the Parser when receiving hu ge amounts through th e socket

[<em...@co...>]

I am not quite suire of the version I am using but this is the code that was
giving me trouble. I am not sure of the version becase I have customised the
code heavily, so I have lately taken code from your updated versions and big
fixes and just added, instead of taking a whole new release, sorry about
this ;)...



The following was giving me trouble. I think the problem is that the
'extractLength' below should not be called before 'pos = m_buffer.find(
"\00110=", pos-1 );', this is because you cannot check for the length unless
the 'standard trailer' is part of the message. This is the problem I was
faced with where the only thing I would have in the buffer when trying to
extract the length is '8=FI', and obviously at this point the extractLength
would throw an exception. So, I have just modified it to check for the
standard message trailer before attempting to extract length and have not
seen this problem.

Here is the code below.

bool Parser::readFixMessage( std::string& str )
throw( MessageParseError&, RecvFailed& )
{ QF_STACK_PUSH(Parser::readFixMessage)

  readFromStream();

  std::string::size_type pos = 0;

  if( m_buffer.length() < 2 ) 
	  return false;
  pos = m_buffer.find( "8=" );
  if( pos == std::string::npos ) 
	  return false;


  int pos_end_trailer_ = m_buffer.find( "\00110=");	\\	I HAVE ADDED
THE FOLLOWING 3 LINES
  if(pos_end_trailer_ == std::string::npos)		\\ 	AND HAVE
SEEMED TO SOLVED THE PRBLEM
	  return false;						\\

  if(pos)
	  m_buffer.erase( 0, pos );

  int length = 0;

  try
  {
    if( extractLength(length, pos, m_buffer) )
    {
      pos += length;
      if( m_buffer.size() < pos )
        return false;

      pos = m_buffer.find( "\00110=", pos-1 );
      if( pos == std::string::npos ) return false;
      pos += 4;
      pos = m_buffer.find( "\001", pos );
      if( pos == std::string::npos ) return false;
      pos += 1;

      str = m_buffer.substr( 0, pos );
      m_buffer.erase( 0, pos );
      return true;
    }
  }
  catch( MessageParseError& e )
  {
    m_buffer.erase( 0, pos + length );
    throw e;
  }

  readFromStream();
  return false;

  QF_STACK_POP
}


I hope this helps.

Regards,

Emir

-----Original Message-----
From: Oren Miller [mailto:or...@qu...]
Sent: Friday, December 10, 2004 10:04 PM
To: em...@co...; qui...@li...
Subject: Re: [Quickfix-users] Possible bug in the Parser when receiving
huge amounts through th e socket


QuickFIX Documentation:
http://www.quickfixengine.org/quickfix/doc/html/index.html
QuickFIX FAQ: http://www.quickfixengine.org/wikifix/index.php?QuickFixFAQ
QuickFIX Support: http://www.quickfixengine.org/services.html

I think I do need some more information about the version you are using.  I 
wrote a test and ran it against 1.9.4 and was not able to duplicate the 
behavior described.  The parser has changed over time so it may have been 
resolved somewhere along the way.  What version are you using?

--oren

----- Original Message ----- 
From: <em...@co...>
To: <qui...@li...>
Sent: Thursday, December 09, 2004 10:14 AM
Subject: [Quickfix-users] Possible bug in the Parser when receiving huge 
amounts through th e socket


> QuickFIX Documentation: 
> http://www.quickfixengine.org/quickfix/doc/html/index.html
> QuickFIX FAQ: http://www.quickfixengine.org/wikifix/index.php?QuickFixFAQ
> QuickFIX Support: http://www.quickfixengine.org/services.html
>
> Here is the scenario.
>
> At the moment the socket buffer in the parser is 4096. So, I have recently
> generated a lot of traffic so I constantly receive of 4096 bytes on the
> socket and have encountered the following problem.
>
> The end of the incoming buffer (m_buffer) of 4096 is as follows (some of 
> the
> values have been substituted to hide trading details):
>
>
8=FIX.4.2
9=291
35=8
34=156110
369=31450
52=20041209-15:35:32.687
49=BLA
50=
>
G
56=D33030N
60=20041209-15:35:32
59=0
55=GE
54=1
48=BLA000060467
107=BLF5
1
>
67=FUT
44=9740.00
41=0
40=2
39=0
38=10
37=20041209004077
151=10
150=0
20=0
9
>
717=67960
17=00687120041209093532
14=0
11=00067960
6=0
432=20041209
1=123456
> 78
10=097
8=F
>
> As we can see, when the parser gets to parsing this part of the buffer, it
> will extract the length of the valid message which is 294 (9=291) and in
> turn send of the message for processing into the application code, etc.
> After, it has processed the message the buffer then looks like:
>
> 8=F
>
> This is because the rest of the message is resting on the socket to be 
> read
> so a valid FIX message can be composed.
>
> As you can see there is a 8=F at the end which was received into the 
> buffer,
> and more data is left over on the socket to be read into the buffer. But, 
> at
> the moment what the parser will do is make 8=F into a fix message as this 
> is
> where the buffer is ending, and will try and extract length from it, and
> this generates a series of exceptions (FieldConvertError, 
> MessageParseError,
> FieldNotFound and InvalidMessage).
>
>
> Once the parser tries to parse this message, and throws its exceptions, it
> will get around to reading the socket again and receiving the following 
> into
> buffer:
>
>
IX.4.2
9=349
35=8
34=156111
369=31450
52=20041209-15:35:32.692
49=BLA
50=G
5
>
6=D33030N
60=20041209-15:35:32
59=0
55=GE
54=1
48=CME000060467
107=BLF5
167=
>
FUT
44=9740.00
41=0
40=2
39=2
38=10
37=20041209004077
337=0C0
32=10
31=9740.
>
00
151=0
150=2
20=0
9717=67960
17=00687220041209093532TN0002843
75=20041209

> 14=0
11=00067960
375=BLA030A1
6=0
432=20041209
1=12345678
10=178

>
> Once this is received the m_buffer is "8=F" and m_readBuffer is as above.
> When the parser appends m_readBuffer to m_buffer the message looks like:
>
>
8=FIX.4.2
9=349
35=8
34=156111
369=31450
52=20041209-15:35:32.692
49=BLA
50=
>
G
56=D33030N
60=20041209-15:35:32
59=0
55=GE
54=1
48=CME000060467
107=BLF5
1
>
67=FUT
44=9740.00
41=0
40=2
39=2
38=10
37=20041209004077
337=0C0
32=10
31=97
>
40.00
151=0
150=2
20=0
9717=67960
17=00687220041209093532TN0002843
75=200412
> 09
14=0
11=00067960
375=BLA030A1
6=0
432=20041209
1=12345678
10=178

>
> a COMPLETE message as it should be.
>
> Now surely, if this kind of scenario is encountered the parser should not
> even attempt to parse the message, as it is incomplete. I think that there
> needs to be an extra safeguard in the parser that needs to check presence 
> of
> field 10 (Standard Trailer) in order to verify that it is a complete
> message.
>
> If anyone has any ideas how to solve this, please be my guess as it has
> taken my over 2 days just to track this problem down.
>
> Thanks a lot
>
> Emir
>
> ---
> Outgoing mail is certified Virus Free.
> Checked by AVG anti-virus system (http://www.grisoft.com).
> Version: 6.0.766 / Virus Database: 513 - Release Date: 9/17/2004
>
>
>
> -------------------------------------------------------
> SF email is sponsored by - The IT Product Guide
> Read honest & candid reviews on hundreds of IT Products from real users.
> Discover which products truly live up to the hype. Start reading now.
> http://productguide.itmanagersjournal.com/
> _______________________________________________
> Quickfix-users mailing list
> Qui...@li...
> https://lists.sourceforge.net/lists/listinfo/quickfix-users
> 



-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now. 
http://productguide.itmanagersjournal.com/
_______________________________________________
Quickfix-users mailing list
Qui...@li...
https://lists.sourceforge.net/lists/listinfo/quickfix-users

---
Incoming mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.766 / Virus Database: 513 - Release Date: 9/17/2004
 

---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.766 / Virus Database: 513 - Release Date: 9/17/2004
 

---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.766 / Virus Database: 513 - Release Date: 9/17/2004