quickfix-developers — QuickFIX developers forum

Re: [Quickfix-developers] Fix forSessionID::fromString

[Oren M. <or...@qu...>]

Thanks Emil,

Another user had reported this last week:  
http://www.quickfixengine.org/bugtracker/bug.php?op=show&bugid=69

The patch is available in CVS, and will go out with the next release:  
http://cvs.sourceforge.net/viewcvs.py/quickfix/quickfix/src/C%2B%2B/ 
SessionID.h?r1=1.7&r2=1.8

Thanks for the report!

--oren

On Mar 30, 2005, at 8:12 AM, Emil Vladov wrote:

> QuickFIX Documentation:  
> http://www.quickfixengine.org/quickfix/doc/html/index.html
> QuickFIX FAQ:  
> http://www.quickfixengine.org/wikifix/index.php?QuickFixFAQ
> QuickFIX Support: http://www.quickfixengine.org/services.html
>
> Hi Oren,
>
> here is a small fix for the parsing of the session ID string -  
> basically if the senderCompID had a dash inside, the SenderCompID  
> ended there, and not on the ->.
>
> (this is from 1.9.4)
>
> Emil
>
>
>
>  void fromString( const std::string& str )
>  {
>    std::string::size_type first =
>      str.find_first_of(':');
>    std::string::size_type second =
>      str.find("->");
>    std::string::size_type third =
>      str.find_last_of(':');
>    if( first == std::string::npos )
>      return;
>    if( second == std::string::npos )
>      return;
>    m_beginString = str.substr(0, first);
>    m_senderCompID = str.substr(first+1, second - first - 1);
>    if( first == third )
>    {
>      m_targetCompID = str.substr(second+2);
>      m_sessionQualifier = "";
>    }
>    else
>    {
>      m_targetCompID = str.substr(second+2, third - second - 2);
>      m_sessionQualifier = str.substr(third+1);
>    }
>  }
>
> _________________________________________________________________
> Express yourself instantly with MSN Messenger! Download today it's  
> FREE! http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/
>
>
>
> -------------------------------------------------------
> SF email is sponsored by - The IT Product Guide
> Read honest & candid reviews on hundreds of IT Products from real  
> users.
> Discover which products truly live up to the hype. Start reading now.
> http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
> _______________________________________________
> Quickfix-developers mailing list
> Qui...@li...
> https://lists.sourceforge.net/lists/listinfo/quickfix-developers
>

[Quickfix-developers] Fix forSessionID::fromString

[Emil V. <que...@ho...>]

Hi Oren,

here is a small fix for the parsing of the session ID string - basically if 
the senderCompID had a dash inside, the SenderCompID ended there, and not on 
the ->.

(this is from 1.9.4)

Emil



  void fromString( const std::string& str )
  {
    std::string::size_type first =
      str.find_first_of(':');
    std::string::size_type second =
      str.find("->");
    std::string::size_type third =
      str.find_last_of(':');
    if( first == std::string::npos )
      return;
    if( second == std::string::npos )
      return;
    m_beginString = str.substr(0, first);
    m_senderCompID = str.substr(first+1, second - first - 1);
    if( first == third )
    {
      m_targetCompID = str.substr(second+2);
      m_sessionQualifier = "";
    }
    else
    {
      m_targetCompID = str.substr(second+2, third - second - 2);
      m_sessionQualifier = str.substr(third+1);
    }
  }

_________________________________________________________________
Express yourself instantly with MSN Messenger! Download today it's FREE! 
http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/

[Quickfix-developers] Re: Changing SenderCompID

[Oren M. <or...@qu...>]

BTW, do they also change these ID's on incoming messages?  That would be a 
bigger problem.

--oren

----- Original Message ----- 
From: "Shane Ryan" <Sr...@ty...>
To: "Oren Miller" <or...@qu...>; "James Wiggs" <wi...@wi...>
Cc: <qui...@li...>
Sent: Tuesday, March 29, 2005 10:12 AM
Subject: Changing SenderCompID


Is there a way to easily change the SenderCompID without starting a new
session?

I am doing the ttCert tests and they want me to change the FirmID which
for them means changing the fourth, fifth, and sixth characters of the
SenderCompID.

I've tried setting the value SenderCompID in the header of a new message
and then using a current session to send it (ie
sessionX->send(message)). But the SenderCompID is not changed. I however
don't get an error. It is just not changed.

It would be handy (at least for my purposed) if the send methods would
recognize the SenderCompID in the current message if it has been set and
otherwise use the default one for the session.

--Shane

Re: [Quickfix-developers] Changing SenderCompID

[Oren M. <or...@qu...>]

Yeah, I've seen the CME's naughty behavior of using multiple SenderCompID's 
on a single session.  QuickFIX has safety precautions in place that ensures 
any message send out always has the correct SenderCompID, TargetCompID and 
BeginString.  If you want to disable this you will have to change change the 
Session::fill( Header ) method to do a something like this:

if( !header.isSetField( FIELD::SenderCompID ) )
    header.setField( m_sessionID.getSenderCompID() )

You will then need to make sure you use the SessionID to lookup your 
session, as your message will not have complete information to identify the 
correct session.

This is a *really* specialized case, and I'm not sure the best way to add 
general support for something like this.  I think many people rely on the 
current behavior for multiplexing which is really convenient to not have to 
worry about session ID's when routing your messages.  This may just require 
another sendToTarget signature which allows you to override the sessionID.

--oren

----- Original Message ----- 
From: "Shane Ryan" <Sr...@ty...>
To: "Caleb Epstein" <cal...@gm...>
Cc: "Oren Miller" <or...@qu...>; "James Wiggs" 
<wi...@wi...>; <qui...@li...>
Sent: Tuesday, March 29, 2005 10:41 AM
Subject: RE: [Quickfix-developers] Changing SenderCompID


No

From the CME's perspective the SenderCompID works like this: chars 1-3
are a session ID, chars 4-6, are the FirmID (the clearing house firm id
that is) and char 7 is a single char that is set based on various
conditions (usually P for primary server or B for backup server but
there is a few other cases).

The SenderSubID is for identifying a trader within a firm.

--Shane


-----Original Message-----
From: Caleb Epstein [mailto:cal...@gm...]
Sent: 29 March 2005 16:20
To: Shane Ryan
Cc: Oren Miller; James Wiggs; qui...@li...
Subject: Re: [Quickfix-developers] Changing SenderCompID

On Tue, 29 Mar 2005 17:12:49 +0100, Shane Ryan
<Sr...@ty...> wrote:
> Is there a way to easily change the SenderCompID without starting a
new
> session?
>
> I am doing the ttCert tests and they want me to change the FirmID
which
> for them means changing the fourth, fifth, and sixth characters of the
> SenderCompID.

!?!?!?  Isn't this what SenderSubID is for?

-- 
Caleb Epstein
caleb dot epstein at gmail dot com

RE: [Quickfix-developers] Changing SenderCompID

[Shane R. <Sr...@ty...>]

No

From the CME's perspective the SenderCompID works like this: chars 1-3
are a session ID, chars 4-6, are the FirmID (the clearing house firm id
that is) and char 7 is a single char that is set based on various
conditions (usually P for primary server or B for backup server but
there is a few other cases).

The SenderSubID is for identifying a trader within a firm.

--Shane


-----Original Message-----
From: Caleb Epstein [mailto:cal...@gm...]=20
Sent: 29 March 2005 16:20
To: Shane Ryan
Cc: Oren Miller; James Wiggs; qui...@li...
Subject: Re: [Quickfix-developers] Changing SenderCompID

On Tue, 29 Mar 2005 17:12:49 +0100, Shane Ryan
<Sr...@ty...> wrote:
> Is there a way to easily change the SenderCompID without starting a
new
> session?
>=20
> I am doing the ttCert tests and they want me to change the FirmID
which
> for them means changing the fourth, fifth, and sixth characters of the
> SenderCompID.

!?!?!?  Isn't this what SenderSubID is for?

--=20
Caleb Epstein
caleb dot epstein at gmail dot com

Re: [Quickfix-developers] Changing SenderCompID

[Caleb E. <cal...@gm...>]

On Tue, 29 Mar 2005 17:12:49 +0100, Shane Ryan <Sr...@ty...> wrote:
> Is there a way to easily change the SenderCompID without starting a new
> session?
> 
> I am doing the ttCert tests and they want me to change the FirmID which
> for them means changing the fourth, fifth, and sixth characters of the
> SenderCompID.

!?!?!?  Isn't this what SenderSubID is for?

-- 
Caleb Epstein
caleb dot epstein at gmail dot com

[Quickfix-developers] Changing SenderCompID

[Shane R. <Sr...@ty...>]

Is there a way to easily change the SenderCompID without starting a new
session?

I am doing the ttCert tests and they want me to change the FirmID which
for them means changing the fourth, fifth, and sixth characters of the
SenderCompID.

I've tried setting the value SenderCompID in the header of a new message
and then using a current session to send it (ie
sessionX->send(message)). But the SenderCompID is not changed. I however
don't get an error. It is just not changed.

It would be handy (at least for my purposed) if the send methods would
recognize the SenderCompID in the current message if it has been set and
otherwise use the default one for the session.

--Shane

Re: [Quickfix-developers] OpenFIX Testing Service

[Oren M. <or...@qu...>]

Actually, looking at the actual test case regarding the OriginalSeqNo 
problem, this is what it states:

Test Description:
g.  PossDupFlag set to Y and OrigSendingTime not specified.

Expected Result:
1) Send Reject (session-level) message referencing missing 
OrigSendingTime (>= FIX 4.2: SessionRejectReason = "Required tag 
missing")
2) Increment inbound MsgSeqNum

So it seems to me that QuickFIX is in fact doing the right thing in 
this case.  Ignore the previous code snippet I sent.  It is wrong 
anyway.

--oren

On Mar 25, 2005, at 11:11 AM, James Wiggs wrote:

> QuickFIX Documentation: 
> http://www.quickfixengine.org/quickfix/doc/html/index.html
> QuickFIX FAQ: 
> http://www.quickfixengine.org/wikifix/index.php?QuickFixFAQ
> QuickFIX Support: http://www.quickfixengine.org/services.html
>
>
>  Folks,
>
>    I am currently testing our QuickFIX-based application against
> the OpenFIX (TransactTools) public testing service.  Overall, the
> testing has been going fairly well, but it has also showed up a
> few gaps in my knowledge -- and what look to be QuickFIX bugs.  I
> am using QuickFIX 1.9.2, and running the FIX 4.2 session-level
> tests.  There are a few which fail for reasons that appear to be
> QuickFIX-related.  First, my own failures:
>
> 1) When I send a manual sequence reset, it appears that the value
> of the sequence number in the Session does not get updated.  I had
> thought that I could get around this by using the Session method
> generateSequenceReset() but that is a private method.  Now I think
> I can fix the problem by sending the SequenceReset method, then
> calling setNextSenderMsgSeqNum() to set the value in the Session,
> but should I pass it "NewSeqNo" or "NewSeqNo+1"?
>
> 2) Due to the problem mentioned in (1) my sequence numbers are now
> out of sync and I can no longer successfully log in to the service.
> The Logon attempt occurs and then the Session immediately logs off
> with an error "MsgSeqNum too low...".  Obviously we don't want to
> exchange messages if the sequence numbers are out of synch, but my
> question is: how do you fix the problem if you can't even log on
> to do a SequenceReset?  I'm clearly missing something very obvious
> here.  Can someone fill me in on what the procedure should be in
> this case with QuickFIX?
>
>    Next, the things that appear to be QuickFIX-related:
>
> 1) Test 2t - Invalid First Three Fields.
> OpenFIX testing service creates and sends a TestRequest with tags
> 8, 9 and 35 deliberately in the wrong order.  My application fails
> this test every time, but I can't figure out if it is a problem
> with QuickFIX or with the test.  They state that the expected flow
> is to ignore the message, drop it without incrementing the incoming
> sequence number, and log a warning message.  I see the following
> in my event log:
>
> 20050325-16:47:21 : Invalid message: Header fields out of order
> 20050325-16:47:33 : Sent test request TEST
> 20050325-16:47:33 : MsgSeqNum too high, expecting  37 but received 39
> 20050325-16:47:33 : Sent ResendRequest FROM: 37 TO: 0
> 20050325-16:47:33 : Received SequenceReset FROM: 37 TO: 40
>
>    But the message that comes back is they received a Heartbeat
> message from me and that "This type of message is not allowed at
> this time."  As I see it, this could mean that in fact my app *is*
> doing the right thing, dropping the message, logging the warning,
> and then continuing on as normal, sending heartbeats every 30
> seconds, and they just aren't closing up the test before the next
> heartbeat goes through.  I could be wrong, though.  Has anyone
> successfully run through all of these tests?  Were there any
> gotchas?
>
> 2) Test 2f - Invalid OrigSendingTime on PossDup Message
> The test complains that they received a Session Reject with a
> "Required tag missing" as the reason for the reject, but that I
> should send SessionRejectReason="SendingTime accuracy problem"
> instead, then send a Logout referencing the bad SendingTime
> value, wait for the Logout response (or wait 2 seconds) and do
> a disconnect.  Obviously, the logout/disconnect didn't happen.
> Is this really an error in QuickFIX, or should I be capturing it
> and implementing this logic at the App level instead?
>
> 3) Test 14m - Missing Conditionally Required Field
> They state that at FIX 4.2 or later, this should not generate a
> Session-level Reject, but rather a BusinessMessageReject.  But
> QuickFIX generates a Session Reject.  And what's more, it did
> this even when I was *not* explicitly trying to get the field in
> question in my onMessage handler.  Is this an oversight on my
> part, should I be building this logic into the onMessage handler
> and manually generating BusinessMessageRejects?  And if so, how
> do I prevent the Session level from rejecting it before I even
> get the message into the handler?
>
>    This has been my first really extensive testing outside my
> own systems, and it has been a real learning experience.  I'd
> appreciate any feedback to help me through the last of this
> work.
>
> thanks,
> Jim
>
> -- 
> James Wiggs <wi...@wi...>
>
>
>
> -------------------------------------------------------
> SF email is sponsored by - The IT Product Guide
> Read honest & candid reviews on hundreds of IT Products from real 
> users.
> Discover which products truly live up to the hype. Start reading now.
> http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
> _______________________________________________
> Quickfix-developers mailing list
> Qui...@li...
> https://lists.sourceforge.net/lists/listinfo/quickfix-developers
>

Re: [Quickfix-developers] OpenFIX Testing Service

[Oren M. <or...@qu...>]

> 1) When I send a manual sequence reset, it appears that the value
> of the sequence number in the Session does not get updated.  I had
> thought that I could get around this by using the Session method
> generateSequenceReset() but that is a private method.  Now I think
> I can fix the problem by sending the SequenceReset method, then
> calling setNextSenderMsgSeqNum() to set the value in the Session,
> but should I pass it "NewSeqNo" or "NewSeqNo+1"?

You should pass it NewSeqNo.  NewSeqNo is the next number the 
counterparty should expect to receive.  Note that the method you are 
calling is set*Next*SenderMsgSeqNum, indicating that this is the next 
sequence number that will be used.   We should probably add a public 
generateSequenceReset( NewSeqNo ) convenience method.

> 2) Due to the problem mentioned in (1) my sequence numbers are now
> out of sync and I can no longer successfully log in to the service.
> The Logon attempt occurs and then the Session immediately logs off
> with an error "MsgSeqNum too low...".  Obviously we don't want to
> exchange messages if the sequence numbers are out of synch, but my
> question is: how do you fix the problem if you can't even log on
> to do a SequenceReset?  I'm clearly missing something very obvious
> here.  Can someone fill me in on what the procedure should be in
> this case with QuickFIX?

Set the ResetSeqNum flag = Y in your toAdmin when the logon gets sent 
out.  Also make sure to set both your sequence numbers to 1.

> 1) Test 2t - Invalid First Three Fields.
> OpenFIX testing service creates and sends a TestRequest with tags
> 8, 9 and 35 deliberately in the wrong order.  My application fails
> this test every time, but I can't figure out if it is a problem
> with QuickFIX or with the test.  They state that the expected flow
> is to ignore the message, drop it without incrementing the incoming
> sequence number, and log a warning message.  I see the following
> in my event log:
>
> 20050325-16:47:21 : Invalid message: Header fields out of order
> 20050325-16:47:33 : Sent test request TEST
> 20050325-16:47:33 : MsgSeqNum too high, expecting  37 but received 39
> 20050325-16:47:33 : Sent ResendRequest FROM: 37 TO: 0
> 20050325-16:47:33 : Received SequenceReset FROM: 37 TO: 40
>
>    But the message that comes back is they received a Heartbeat
> message from me and that "This type of message is not allowed at
> this time."  As I see it, this could mean that in fact my app *is*
> doing the right thing, dropping the message, logging the warning,
> and then continuing on as normal, sending heartbeats every 30
> seconds, and they just aren't closing up the test before the next
> heartbeat goes through.  I could be wrong, though.  Has anyone
> successfully run through all of these tests?  Were there any
> gotchas?

Yeah, I can't think of a scenario where a Heartbeat is ever not 
allowed. On the face of it this looks perfectly normal to me.

> 2) Test 2f - Invalid OrigSendingTime on PossDup Message
> The test complains that they received a Session Reject with a
> "Required tag missing" as the reason for the reject, but that I
> should send SessionRejectReason="SendingTime accuracy problem"
> instead, then send a Logout referencing the bad SendingTime
> value, wait for the Logout response (or wait 2 seconds) and do
> a disconnect.  Obviously, the logout/disconnect didn't happen.
> Is this really an error in QuickFIX, or should I be capturing it
> and implementing this logic at the App level instead?

Yeah, we should change this.  I'm looking at the spec and it states 
that if the OrigSendingTime is not available, it should be set to the 
same value as SendingTime.  So we can change the relevant code from 
this:

     if ( !header.isSetField( origSendingTime ) )
     {
       generateReject( msg, 1, origSendingTime.getField() );
       return false;
     }
     header.getField( origSendingTime );

To this:

     if ( !header.isSetField( origSendingTime ) )
       origSendingTime = OrigSendingTime( sendingTime );
     else
       header.getField( origSendingTime );

> 3) Test 14m - Missing Conditionally Required Field
> They state that at FIX 4.2 or later, this should not generate a
> Session-level Reject, but rather a BusinessMessageReject.  But
> QuickFIX generates a Session Reject.  And what's more, it did
> this even when I was *not* explicitly trying to get the field in
> question in my onMessage handler.  Is this an oversight on my
> part, should I be building this logic into the onMessage handler
> and manually generating BusinessMessageRejects?  And if so, how
> do I prevent the Session level from rejecting it before I even
> get the message into the handler?

Yeah, we should be rejecting conditional fields with business level 
rejects.  I'll post a patch for this.  I have no idea why you would be 
rejected before getting into the handler.  What message and field was 
this?

--oren

[Quickfix-developers] OpenFIX Testing Service

[James W. <wi...@wi...>]

 Folks,

   I am currently testing our QuickFIX-based application against
the OpenFIX (TransactTools) public testing service.  Overall, the
testing has been going fairly well, but it has also showed up a
few gaps in my knowledge -- and what look to be QuickFIX bugs.  I
am using QuickFIX 1.9.2, and running the FIX 4.2 session-level
tests.  There are a few which fail for reasons that appear to be
QuickFIX-related.  First, my own failures:

1) When I send a manual sequence reset, it appears that the value
of the sequence number in the Session does not get updated.  I had
thought that I could get around this by using the Session method
generateSequenceReset() but that is a private method.  Now I think
I can fix the problem by sending the SequenceReset method, then
calling setNextSenderMsgSeqNum() to set the value in the Session,
but should I pass it "NewSeqNo" or "NewSeqNo+1"?

2) Due to the problem mentioned in (1) my sequence numbers are now
out of sync and I can no longer successfully log in to the service.
The Logon attempt occurs and then the Session immediately logs off
with an error "MsgSeqNum too low...".  Obviously we don't want to
exchange messages if the sequence numbers are out of synch, but my
question is: how do you fix the problem if you can't even log on
to do a SequenceReset?  I'm clearly missing something very obvious
here.  Can someone fill me in on what the procedure should be in
this case with QuickFIX?

   Next, the things that appear to be QuickFIX-related:

1) Test 2t - Invalid First Three Fields.
OpenFIX testing service creates and sends a TestRequest with tags
8, 9 and 35 deliberately in the wrong order.  My application fails
this test every time, but I can't figure out if it is a problem
with QuickFIX or with the test.  They state that the expected flow
is to ignore the message, drop it without incrementing the incoming
sequence number, and log a warning message.  I see the following
in my event log:

20050325-16:47:21 : Invalid message: Header fields out of order
20050325-16:47:33 : Sent test request TEST
20050325-16:47:33 : MsgSeqNum too high, expecting  37 but received 39
20050325-16:47:33 : Sent ResendRequest FROM: 37 TO: 0
20050325-16:47:33 : Received SequenceReset FROM: 37 TO: 40

   But the message that comes back is they received a Heartbeat
message from me and that "This type of message is not allowed at
this time."  As I see it, this could mean that in fact my app *is*
doing the right thing, dropping the message, logging the warning,
and then continuing on as normal, sending heartbeats every 30
seconds, and they just aren't closing up the test before the next
heartbeat goes through.  I could be wrong, though.  Has anyone
successfully run through all of these tests?  Were there any
gotchas?

2) Test 2f - Invalid OrigSendingTime on PossDup Message
The test complains that they received a Session Reject with a
"Required tag missing" as the reason for the reject, but that I
should send SessionRejectReason="SendingTime accuracy problem"
instead, then send a Logout referencing the bad SendingTime
value, wait for the Logout response (or wait 2 seconds) and do
a disconnect.  Obviously, the logout/disconnect didn't happen.
Is this really an error in QuickFIX, or should I be capturing it
and implementing this logic at the App level instead?

3) Test 14m - Missing Conditionally Required Field
They state that at FIX 4.2 or later, this should not generate a
Session-level Reject, but rather a BusinessMessageReject.  But
QuickFIX generates a Session Reject.  And what's more, it did
this even when I was *not* explicitly trying to get the field in
question in my onMessage handler.  Is this an oversight on my
part, should I be building this logic into the onMessage handler
and manually generating BusinessMessageRejects?  And if so, how
do I prevent the Session level from rejecting it before I even
get the message into the handler?

   This has been my first really extensive testing outside my
own systems, and it has been a real learning experience.  I'd
appreciate any feedback to help me through the last of this
work.

thanks,
Jim

-- 
James Wiggs <wi...@wi...>

[Quickfix-developers] SequenceReset control

[James W. <wi...@wi...>]

 Folks,

   I had coded some control logic in my QuickFIX application to
generate SequenceReset messages and send them directly.  I have
discovered that there is a session-level method to do this, but
it looks like you have no control over whether the GapFill flag
will be set.  Session:generateSequenceReset() looks like it will
*always* set GapFillFlag to Y.  Is there a way to generate such
a message with GapFillFlag set to N or not sent at all?

   I ask because while the SequenceReset message goes out when I
generate and send it manually, the Sequence numbers seem not to
be getting reset in the session, and subsequent traffic ends up
with incorrect sequence numbers; I was hoping that using the
Session method would take care of this problem.

thanks,
Jim

-- 
James Wiggs <wi...@wi...>

[Quickfix-developers] Re: Newbie: Session management in load balanced situation?

[Sol <sol...@ho...>]

Caleb: it seems your answer is oriented towards an Acceptor environment. =
What if I have a farm of Initiator servers?

Currently we have a quickfix server that aggregates messages that are =
sent from multiple users, and sends them using a single session to the =
provider. We would like to scale that up by having multiple quickfix =
servers, each serving a subset of the users.

Can all the servers concurrently log in using the same session =
configuration? Or do we need to set up separate session settings for =
each server?

David: if this is not your situation, sorry for hijacking this thread.

Thanks in advance for any input.
-sol

RE: [Quickfix-developers] SocketInitiator.stop()

[Patrick F. <pat...@ch...>]

Oren, 
      Thanks much for your reply.  I will try to explain your question about
the flag. 
The unmanaged C++ version of Initiator::stop(bool force) sets force to a
default value of false (Initiator.cpp line 184).  The .NET version of
SocketInitiator.stop() (SocketInitiator.h line 111)calls the unmanaged stop
without ever setting this parameter or allowing the user to set it.  This
seems to cause the .NET initiator stop function to wait infinitely. 
 
Also if you call Session.lookupSession(mySession).logout();  to end a
session should you be calling Session.lookupSession(mySession).logon();  to
initiate a session instead of sending the logon message ?
 
Patrick Flannery
 
 
  _____  

From: Oren Miller [mailto:or...@qu...] 
Sent: Wednesday, March 23, 2005 11:26 AM
To: Patrick Flannery; qui...@li...
Cc: joh...@ch...
Subject: Re: [Quickfix-developers] SocketInitiator.stop()
 
Patrick,
 
You shouldn't be sending you're own logon and logout messages.  QuickFIX
does this for you.  Instead you should be sending your applications logon
message in the onLogon callback, which is when the FIX sessions have
succesfully handshaked.  And when you log off, you should use the logoff
call instead of sending your own logoff message.
 
What flag are you refering to?
 
--oren
----- Original Message ----- 
From: Patrick Flannery <mailto:pat...@ch...>  
To: qui...@li...
<mailto:qui...@li...>  
Sent: Friday, March 18, 2005 1:56 PM
Subject: [Quickfix-developers] SocketInitiator.stop()
 
Hello,
         There were some recent messages about the SocketInitiator::stop()
causing a deadlock.  An update was posted in cvs where the code has a flag
allowing the user to force a close.  The C# version of SocketInitiator does
not have this same flag and may be needed because the SocketInitiator.stop()
function deadlocks.  Also, and this is real simple, is this correct way to
start a fix session?
 
void start(){
initiator.start(); 
      
     //Wait for the FIX.onLogon(Session ) to be signaled. 
      WaitForConnect(CONNECT_TIMEOUT);
 
           //Fix logon      
      QuickFix44.Logon fixLogon = new Logon();
      fix.Send(fixLogon);    
 
     //Application Logon
      QuickFix44.UserRequest logon = new QuickFix44.UserRequest();
      QuoterParamaters fields = new QuoterParamaters(credentials);
      Set(logon, new UserRequestType(UserRequestType.LOGONUSER));//Set the
logon flag to true
 
      fix.Send(logon);
}
 
 
With a corresponding stop:
 
void stop()
{
     //Application Logoff
      QuickFix44.UserRequest logoff = new QuickFix44.UserRequest ();
      Set(logoff,  new UserRequestType(UserRequestType.LOGOFFUSER));
      fix.Send(logoff);
                
      //Fix logoff
      if(fix.LoggedOn)
{
            QuickFix44.Logout fixLogout = new Logout();
           fix.Send(fixLogout);
      }
                      
      /**
      * This causes the thread to hang. 
      * */
 
initiator.stop();
}
 
Thanks in advance. 
 
Patrick Flannery 
 
 

Re: [Quickfix-developers] SocketInitiator.stop()

[Oren M. <or...@qu...>]

We have addressed starting and stopping issues in CVS.  Are you also using 
.NET or another API?

--oren

----- Original Message ----- 
From: "John Debay" <joh...@ch...>
To: "Patrick Flannery" <pat...@ch...>; 
<qui...@li...>
Sent: Wednesday, March 23, 2005 11:09 AM
Subject: RE: [Quickfix-developers] SocketInitiator.stop()


> QuickFIX Documentation: 
> http://www.quickfixengine.org/quickfix/doc/html/index.html
> QuickFIX FAQ: http://www.quickfixengine.org/wikifix/index.php?QuickFixFAQ
> QuickFIX Support: http://www.quickfixengine.org/services.html
>
> Has anyone experienced similar behavior? Does anyone have a solution?
>
> We are using QuickFIX 1.9.3.
>
> Thanks,
> John
>
>
> ________________________________
>
> From: qui...@li...
> [mailto:qui...@li...] On Behalf Of
> Patrick Flannery
> Sent: Friday, March 18, 2005 1:57 PM
> To: qui...@li...
> Subject: [Quickfix-developers] SocketInitiator.stop()
>
>
>
> Hello,
>
>             There were some recent messages about the
> SocketInitiator::stop() causing a deadlock.  An update was posted in cvs
> where the code has a flag allowing the user to force a close.  The C#
> version of SocketInitiator does not have this same flag and may be needed
> because the SocketInitiator.stop() function deadlocks.  Also, and this is
> real simple, is this correct way to start a fix session?
>
>
>
> void start(){
>
> initiator.start();
>
>
>
>       //Wait for the FIX.onLogon(Session ) to be signaled.
>
>       WaitForConnect(CONNECT_TIMEOUT);
>
>
>
>      //Fix logon
>
>       QuickFix44.Logon fixLogon = new Logon();
>
>       fix.Send(fixLogon);
>
>
>
>       //Application Logon
>
>       QuickFix44.UserRequest logon = new QuickFix44.UserRequest();
>
>       QuoterParamaters fields = new QuoterParamaters(credentials);
>
>       Set(logon, new
> UserRequestType(UserRequestType.LOGONUSER));//Set the logon flag to true
>
>
>
>       fix.Send(logon);
>
> }
>
>
>
>
>
> With a corresponding stop:
>
>
>
> void stop()
>
> {
>
>       //Application Logoff
>
>       QuickFix44.UserRequest logoff = new QuickFix44.UserRequest ();
>
>       Set(logoff,  new UserRequestType(UserRequestType.LOGOFFUSER));
>
>       fix.Send(logoff);
>
>
>
>       //Fix logoff
>
>       if(fix.LoggedOn)
>
> {
>
>             QuickFix44.Logout fixLogout = new Logout();
>
>             fix.Send(fixLogout);
>
>       }
>
>
>
>       /**
>
>       * This causes the thread to hang.
>
>       * */
>
>
>
> initiator.stop();
>
> }
>
>
>
> Thanks in advance.
>
>
>
> Patrick Flannery
>
>
>
>
>
>
>
> -------------------------------------------------------
> This SF.net email is sponsored by Microsoft Mobile & Embedded DevCon 2005
> Attend MEDC 2005 May 9-12 in Vegas. Learn more about the latest Windows
> Embedded(r) & Windows Mobile(tm) platforms, applications & content. 
> Register
> by 3/29 & save $300 
> http://ads.osdn.com/?ad_id=6883&alloc_id=15149&op=click
> _______________________________________________
> Quickfix-developers mailing list
> Qui...@li...
> https://lists.sourceforge.net/lists/listinfo/quickfix-developers
> 

Re: [Quickfix-developers] SocketInitiator.stop()

[Oren M. <or...@qu...>]

Patrick,

You shouldn't be sending you're own logon and logout messages.  QuickFIX =
does this for you.  Instead you should be sending your applications =
logon message in the onLogon callback, which is when the FIX sessions =
have succesfully handshaked.  And when you log off, you should use the =
logoff call instead of sending your own logoff message.

What flag are you refering to?

--oren
  ----- Original Message -----=20
  From: Patrick Flannery=20
  To: qui...@li...=20
  Sent: Friday, March 18, 2005 1:56 PM
  Subject: [Quickfix-developers] SocketInitiator.stop()


  Hello,

              There were some recent messages about the =
SocketInitiator::stop() causing a deadlock.  An update was posted in cvs =
where the code has a flag allowing the user to force a close.  The C# =
version of SocketInitiator does not have this same flag and may be =
needed because the SocketInitiator.stop() function deadlocks.  Also, and =
this is real simple, is this correct way to start a fix session?

  =20

  void start(){

  initiator.start();=20

       =20

        //Wait for the FIX.onLogon(Session ) to be signaled.=20

        WaitForConnect(CONNECT_TIMEOUT);

  =20

       //Fix logon     =20

        QuickFix44.Logon fixLogon =3D new Logon();

        fix.Send(fixLogon);    =20

  =20

        //Application Logon

        QuickFix44.UserRequest logon =3D new QuickFix44.UserRequest();

        QuoterParamaters fields =3D new QuoterParamaters(credentials);

        Set(logon, new UserRequestType(UserRequestType.LOGONUSER));//Set =
the logon flag to true

  =20

        fix.Send(logon);

  }

  =20

  =20

  With a corresponding stop:

  =20

  void stop()

  {

        //Application Logoff

        QuickFix44.UserRequest logoff =3D new QuickFix44.UserRequest ();

        Set(logoff,  new UserRequestType(UserRequestType.LOGOFFUSER));

        fix.Send(logoff);

                 =20

        //Fix logoff

        if(fix.LoggedOn)

  {

              QuickFix44.Logout fixLogout =3D new Logout();

              fix.Send(fixLogout);

        }

                       =20

        /**

        * This causes the thread to hang.=20

        * */

  =20

  initiator.stop();

  }

  =20

  Thanks in advance.=20

  =20

  Patrick Flannery=20

  =20

  =20

RE: [Quickfix-developers] SocketInitiator.stop()

[John D. <joh...@ch...>]

Has anyone experienced similar behavior? Does anyone have a solution?

We are using QuickFIX 1.9.3.

Thanks,
John 


________________________________

	From: qui...@li...
[mailto:qui...@li...] On Behalf Of
Patrick Flannery
	Sent: Friday, March 18, 2005 1:57 PM
	To: qui...@li...
	Subject: [Quickfix-developers] SocketInitiator.stop()
	
	

	Hello,

	            There were some recent messages about the
SocketInitiator::stop() causing a deadlock.  An update was posted in cvs
where the code has a flag allowing the user to force a close.  The C#
version of SocketInitiator does not have this same flag and may be needed
because the SocketInitiator.stop() function deadlocks.  Also, and this is
real simple, is this correct way to start a fix session?

	 

	void start(){

	initiator.start(); 

	      

	      //Wait for the FIX.onLogon(Session ) to be signaled. 

	      WaitForConnect(CONNECT_TIMEOUT);

	 

	     //Fix logon      

	      QuickFix44.Logon fixLogon = new Logon();

	      fix.Send(fixLogon);     

	 

	      //Application Logon

	      QuickFix44.UserRequest logon = new QuickFix44.UserRequest();

	      QuoterParamaters fields = new QuoterParamaters(credentials);

	      Set(logon, new
UserRequestType(UserRequestType.LOGONUSER));//Set the logon flag to true

	 

	      fix.Send(logon);

	}

	 

	 

	With a corresponding stop:

	 

	void stop()

	{

	      //Application Logoff

	      QuickFix44.UserRequest logoff = new QuickFix44.UserRequest ();

	      Set(logoff,  new UserRequestType(UserRequestType.LOGOFFUSER));

	      fix.Send(logoff);

	                

	      //Fix logoff

	      if(fix.LoggedOn)

	{

	            QuickFix44.Logout fixLogout = new Logout();

	            fix.Send(fixLogout);

	      }

	                      

	      /**

	      * This causes the thread to hang. 

	      * */

	 

	initiator.stop();

	}

	 

	Thanks in advance. 

	 

	Patrick Flannery 

	 

	 

Re: [Quickfix-developers] Intermittent disconnects on Solaris

[Oren M. <or...@qu...>]

I know all of this is due for an overhaul.  I would have actually prefered 
to use one of the libraries dedicated to socket handling, but a couple 
reasons prevented me from doing so:  First licensing issues limited our 
choices (can't use GPL and distribute under our license), and second I felt 
it important that QF have as few third party dependencies as possible to 
make the build process simple.  It was far from assured anyone would bother 
to download and build quickfix, much less have to build an equally 
complicated third party socket library.  In order to encourage adoption, the 
basics had to be encapsulated into the main library.

Well ok, that part of the plan worked well enough, and now we are paying the 
price for that decision.  Fortunately unlike before we have the benefit of 
experienced socket developers to advise on how to correct the inexperienced 
design decisions.

First thing to know is the Parser's use of an std::stream is an experiment 
gone bad.  It's not actually used and should have been removed a long time 
ago.  (At one point, the sockets were wrapped in a custom std::stream.  So 
at the time that was intended to be the abstraction. In practice there were 
some problems with this and it was abandoned.)  The only reason it still 
exists is so there is a way for the unit tests to pass data to the Parser. 
Obviously this reduces the value of the tests somewhat and brings glaring 
attention to the design flaw.  Why was the recv call placed there?  Probably 
a poor attempt to share code between the Socket and ThreadedSocket 
implementations. If we had ever gotten around to implementing any non-socket 
based transports, this would certainly have been addressed sooner.

But this will be easy to fix.  We basically need to pull the functionality 
out of the readFromStream call, and instead pass in a parameter to 
readFixMessage which will contain just the bytes to be processed.

The other problems will be harder to address, although I believe some of you 
would likely benefit from the ThreadedSocket classes. These are what is 
currently recommended for high frequency lines, particularly if you are 
running several of them.  I personally don't want to add threads to the 
standard Socket implementation as that pretty much defeats the existence of 
its purpose.  It is intended for applications which do not want or need to 
deal with threads.  When the SocketInitiator/Acceptor Combined with the 
block() or poll() call, QuickFIX can be run entirely single threaded.  I 
think there is value in this. I also think there is plenty of precedence for 
this if done correctly.

I think the naming of the Socket and ThreadedSocket implementations has 
caused confusion.  Perhaps it would be better to just have classes named 
SocketInitiator/Acceptor and pass in the threading model through the 
constructor or configuration settings.

What I think I would like to have happen is if we can come up with a plan to 
address the current shortcomings within the existing framework, do a 
release, and then focus on a more comprehensive overhaul.  Comments?

--oren 

Re: [Quickfix-developers] Intermittent disconnects on Solaris

[Caleb E. <cal...@gm...>]

On Tue, 22 Mar 2005 12:05:43 -0000, kri...@rb...
<kri...@rb...> wrote:
> Oren, Caleb -
> 
> When debugging this problem, I was quite surprised by QuickFIX's socket handling. Despite the use of select(), non-blocking I/O is not actually used which seems to be why platform-specific ioctl's are used (to figure out how much you can read with out blocking).
> 
> As for using recv()/read() == 0 to determine EOF, that fact that the actual socket reads and writes are scattered among various functions and methods of unrelated classes makes this quite harder to do. It also complicates error reporting (i.e. errno on failed reads/writes) and disconnection reasons that I've seen people rquesting on the list.
> 
> There is a SocketConnection class which 'owns' a connected socket and there are functions in Utility.cpp for doing socket I/O, yet grep'ing for recv() calls returns two calls to the syscall recv(), and neither one is in either SocketConnection or in Utility.cpp. FIX::Parser's readFromStream()'s method seems an especially dubious place to put recv(); the fact that FIX::Parser has a member varible for holding a file descriptor as well a pointer to an std::istream also strikes me as wrong.

I agree that the design of the connections, strategies, etc. is
confusing and hard to follow.  I also agree that the Parser should not
have any knowledge of a file descriptor or iostream.  It should just
be handed char*s or std::strings.

> lsdev2:~/work/quickfix/src/C++ $ grep -n -1 recv *.cpp *.h
> Parser.cpp-144-
> Parser.cpp:145:    size = recv( m_socket, m_readBuffer, m_bufferSize, 0 );
> Parser.cpp-146-    if ( size <= 0 ) throw RecvFailed();
> --
> ThreadedSocketConnection.cpp-96-    buffer = new char[ bytes + 1 ];
> ThreadedSocketConnection.cpp:97:    int result = recv( m_socket, buffer, bytes, 0 );
> ThreadedSocketConnection.cpp-98-    if ( result <= 0 ) { throw std::exception(); }
> 
> During my attempts to recreate the Solaris disconnect problem, I wrote sample QuickFIX apps that would send torrents of messages. It was quite easy to lock up two QuickFIX apps talking to each other during a reconnect, both sides would requests resends from each other and then the SocketInitiator and SocketAcceptor threads in the apps would block on send() when resending the messages to each other, as send() would not return until the other side called recv(), which it could not do until the other side called recv(), etc. The apps would then sleep forever.

I have previously submitted a patch (attached here) that enables
non-blocking I/O by using the MSG_DONTWAIT send/recv flag.  It is
lightly tested, and was made originally against 1.9.0, but I was able
to clear up a lock-up type scenario as you describe with it.

I'm not totally happy with the end-result: it won't work on systems
that don't support MSG_DONTWAIT, and wedges yet another callback into
the Socket Strategy interface (onWriteable).  A better approach to the
first problem would be to use the appropriate ioctl to set the sockets
into non-blocking mode for all operations.  As far as the Strategy
interface goes, I think it is over-designed (look at all the empty
callbacks!) and should probably be scrapped in favor of something
simpler.

> This problem can only be avoided by using non-block I/O or having one thread for reading and another for writing, neither of which SocketInitiator or SocketAcceptor do (I haven't looked at ThreadedSocketInitiator).

I think the overall approach of using select and then calling
send/recv as appropriate is the right one.  I don't think the idea of
another thread per-connection or possibly two is tenable.  There is
already one that is handing off strings to the Parser for processing
(when using ThreadedSocketConnection).  Adding more would be
gratuitous.

The down-side of using select is that you essentially lock yourself
into a certain amount of latency: the timeval you pass to the select
call.  If the SocketMonitor is sitting in select and you have a new fd
to add to the list (e.g. there is suddenly some data to send), you
either have to wait for the select timeout to expire, or you add a
special "signalling" fd to your fd_set that you write to when you want
to force select to break out of its wait.

Anyway, thanks for the analysis and insight Kristofer.  I hope we can
improve QuickFIX as a result of the points raised in this thread.

-- 
Caleb Epstein
caleb dot epstein at gmail dot com

RE: [Quickfix-developers] Intermittent disconnects on Solaris

[Bishop, B. <Bar...@gs...>]

Hi Kristofer,

Firstly, I'd like to say thanks very much for your hard work in finding this
'bug'. The later versions of quickfix are much more graceful when
reconnecting, but it still remains an annoying problem.

Secondly, I would totally agree to all of your comments. I would further
add, that it has always struck me as madness that the Parser class contains
a socket handle. It also contains an istream pointer, so I conclude that
every time a new strategy is devised for collecting bytes to parse a new
member will be added to Parser. Personally, I would have designed Parser to
have a method for processing bytes and separate classes for collecting
bundles of bytes to give to the Parser, whether from a socket, a stream, a
file or whatever.

I would also re-enforce your point about threading and sockets. For
clearness of design and ease of partitioning work, I would absolutely always
design components so that every socket has its own dedicated reading thread
and very often a dedicated writing thread also. These would of course,
always using blocking I/O.

At the moment, I think these improvements would involve significant upheaval
in the design of quickfix.

Oren, is this kind of thing at all likely?

Later in the year I might actually have time to work on some of these ideas,
but in the mean time, thanks again.

Barry Bishop


-----Original Message-----
From: kri...@rb... [mailto:kri...@rb...] 
Sent: Tuesday, March 22, 2005 12:06 PM
To: or...@qu...; cal...@gm...
Cc: qui...@li...; Bishop, Barry
Subject: RE: [Quickfix-developers] Intermittent disconnects on Solaris


Oren, Caleb -

When debugging this problem, I was quite surprised by QuickFIX's socket
handling. Despite the use of select(), non-blocking I/O is not actually used
which seems to be why platform-specific ioctl's are used (to figure out how
much you can read with out blocking).

As for using recv()/read() == 0 to determine EOF, that fact that the actual
socket reads and writes are scattered among various functions and methods of
unrelated classes makes this quite harder to do. It also complicates error
reporting (i.e. errno on failed reads/writes) and disconnection reasons that
I've seen people rquesting on the list.

There is a SocketConnection class which 'owns' a connected socket and there
are functions in Utility.cpp for doing socket I/O, yet grep'ing for recv()
calls returns two calls to the syscall recv(), and neither one is in either
SocketConnection or in Utility.cpp. FIX::Parser's readFromStream()'s method
seems an especially dubious place to put recv(); the fact that FIX::Parser
has a member varible for holding a file descriptor as well a pointer to an
std::istream also strikes me as wrong.

lsdev2:~/work/quickfix/src/C++ $ grep -n -1 recv *.cpp *.h
Parser.cpp-144-
Parser.cpp:145:    size = recv( m_socket, m_readBuffer, m_bufferSize, 0 );
Parser.cpp-146-    if ( size <= 0 ) throw RecvFailed();
--
ThreadedSocketConnection.cpp-96-    buffer = new char[ bytes + 1 ];
ThreadedSocketConnection.cpp:97:    int result = recv( m_socket, buffer,
bytes, 0 );
ThreadedSocketConnection.cpp-98-    if ( result <= 0 ) { throw
std::exception(); }

During my attempts to recreate the Solaris disconnect problem, I wrote
sample QuickFIX apps that would send torrents of messages. It was quite easy
to lock up two QuickFIX apps talking to each other during a reconnect, both
sides would requests resends from each other and then the SocketInitiator
and SocketAcceptor threads in the apps would block on send() when resending
the messages to each other, as send() would not return until the other side
called recv(), which it could not do until the other side called recv(),
etc. The apps would then sleep forever.

This problem can only be avoided by using non-block I/O or having one thread
for reading and another for writing, neither of which SocketInitiator or
SocketAcceptor do (I haven't looked at ThreadedSocketInitiator).

Addressing this would be a fair amount of work, however the patch above has
eliminated the disconnects which were causing us major headaches as they
tended to happen at around market close.

Regards,

- Kris

-----Original Message-----
From: Oren Miller [mailto:or...@qu...]
Sent: 21 March 2005 19:23
To: Caleb Epstein; Peterson, Kristofer
Cc: qui...@li...; Bar...@gs...
Subject: Re: [Quickfix-developers] Intermittent disconnects on Solaris


Yeah, I agree.

--oren

----- Original Message ----- 
From: "Caleb Epstein" <cal...@gm...>
To: <kri...@rb...>
Cc: <qui...@li...>; <Bar...@gs...>
Sent: Monday, March 21, 2005 1:20 PM
Subject: Re: [Quickfix-developers] Intermittent disconnects on Solaris


> QuickFIX Documentation:
> http://www.quickfixengine.org/quickfix/doc/html/index.html
> QuickFIX FAQ: http://www.quickfixengine.org/wikifix/index.php?QuickFixFAQ
> QuickFIX Support: http://www.quickfixengine.org/services.html
>
> On Mon, 21 Mar 2005 19:02:19 -0000, kri...@rb... 
> <kri...@rb...> wrote:
>
>> I have isolated the cause of the intermittent disconnects caused by
>> QuickFIX on Solaris. The problem was due to the use of the I_NREAD ioctl
>> to determine whether a readable socket was EOF or not.
>>
>> In certain circumstances which seem to involve high network traffic 
>> and >> low machine load, I_NREAD will return zero for a readable 
>> socket that >> actually has data. In such cases, QuickFIX would 
>> erroneously close the >> socket.
>>
>> I replaced the I_NREAD code in socket_disconnected() one byte recv() 
>> with
>> the MSG_PEEK flag. This appears to have resolved this rather troublesome
>> issue for us in production.
>>
>
> Shouldn't QuickFIX just rely on recv returning 0 to detect a socket 
> disconnect, instead of relying on this ioctl?  I've never seen a 
> socket-based application using this technique to detect disconnects 
> before.  Clearly its not 100% reliable.
>
> --
> Caleb Epstein
> caleb dot epstein at gmail dot com
>
>
> -------------------------------------------------------
> SF email is sponsored by - The IT Product Guide
> Read honest & candid reviews on hundreds of IT Products from real 
> users. Discover which products truly live up to the hype. Start 
> reading now. http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
> _______________________________________________
> Quickfix-developers mailing list 
> Qui...@li...
> https://lists.sourceforge.net/lists/listinfo/quickfix-developers
> 


<font face="Times New Roman" size="3">
<p>-------------------------------------------------------------------------
-----</p>
<p> This email is intended only for the use of the individual(s) to whom it
is addressed and may be privileged and confidential. Unauthorised use or
disclosure is prohibited. If you receive this e-mail in error, please advise
immediately and delete the original message. This message may have been
altered without your or our knowledge and the sender does not accept any
liability for any errors or omissions in the message.</p>
<p>====================================================</p>
</font>

RE: [Quickfix-developers] Intermittent disconnects on Solaris

[<kri...@rb...>]

Oren, Caleb -

When debugging this problem, I was quite surprised by QuickFIX's socket han=
dling. Despite the use of select(), non-blocking I/O is not actually used w=
hich seems to be why platform-specific ioctl's are used (to figure out how =
much you can read with out blocking).

As for using recv()/read() =3D=3D 0 to determine EOF, that fact that the ac=
tual socket reads and writes are scattered among various functions and meth=
ods of unrelated classes makes this quite harder to do. It also complicates=
 error reporting (i.e. errno on failed reads/writes) and disconnection reas=
ons that I've seen people rquesting on the list.

There is a SocketConnection class which 'owns' a connected socket and there=
 are functions in Utility.cpp for doing socket I/O, yet grep'ing for recv()=
 calls returns two calls to the syscall recv(), and neither one is in eithe=
r SocketConnection or in Utility.cpp. FIX::Parser's readFromStream()'s meth=
od seems an especially dubious place to put recv(); the fact that FIX::Pars=
er has a member varible for holding a file descriptor as well a pointer to =
an std::istream also strikes me as wrong.

lsdev2:~/work/quickfix/src/C++ $ grep -n -1 recv *.cpp *.h
Parser.cpp-144-
Parser.cpp:145:    size =3D recv( m_socket, m_readBuffer, m_bufferSize, 0 );
Parser.cpp-146-    if ( size <=3D 0 ) throw RecvFailed();
--
ThreadedSocketConnection.cpp-96-    buffer =3D new char[ bytes + 1 ];
ThreadedSocketConnection.cpp:97:    int result =3D recv( m_socket, buffer, =
bytes, 0 );
ThreadedSocketConnection.cpp-98-    if ( result <=3D 0 ) { throw std::excep=
tion(); }

During my attempts to recreate the Solaris disconnect problem, I wrote samp=
le QuickFIX apps that would send torrents of messages. It was quite easy to=
 lock up two QuickFIX apps talking to each other during a reconnect, both s=
ides would requests resends from each other and then the SocketInitiator an=
d SocketAcceptor threads in the apps would block on send() when resending t=
he messages to each other, as send() would not return until the other side =
called recv(), which it could not do until the other side called recv(), et=
c. The apps would then sleep forever.

This problem can only be avoided by using non-block I/O or having one threa=
d for reading and another for writing, neither of which SocketInitiator or =
SocketAcceptor do (I haven't looked at ThreadedSocketInitiator).

Addressing this would be a fair amount of work, however the patch above has=
 eliminated the disconnects which were causing us major headaches as they t=
ended to happen at around market close.

Regards,

- Kris

-----Original Message-----
From: Oren Miller [mailto:or...@qu...]
Sent: 21 March 2005 19:23
To: Caleb Epstein; Peterson, Kristofer
Cc: qui...@li...; Bar...@gs...
Subject: Re: [Quickfix-developers] Intermittent disconnects on Solaris


Yeah, I agree.

--oren

----- Original Message -----=20
From: "Caleb Epstein" <cal...@gm...>
To: <kri...@rb...>
Cc: <qui...@li...>; <Bar...@gs...>
Sent: Monday, March 21, 2005 1:20 PM
Subject: Re: [Quickfix-developers] Intermittent disconnects on Solaris


> QuickFIX Documentation:=20
> http://www.quickfixengine.org/quickfix/doc/html/index.html
> QuickFIX FAQ: http://www.quickfixengine.org/wikifix/index.php?QuickFixFAQ
> QuickFIX Support: http://www.quickfixengine.org/services.html
>
> On Mon, 21 Mar 2005 19:02:19 -0000, kri...@rb...
> <kri...@rb...> wrote:
>
>> I have isolated the cause of the intermittent disconnects caused by=20
>> QuickFIX on Solaris. The problem was due to the use of the I_NREAD ioctl=
 >> to determine whether a readable socket was EOF or not.
>>
>> In certain circumstances which seem to involve high network traffic and=
 >> low machine load, I_NREAD will return zero for a readable socket that=
 >> actually has data. In such cases, QuickFIX would erroneously close the=
 >> socket.
>>
>> I replaced the I_NREAD code in socket_disconnected() one byte recv() wit=
h=20
>> the MSG_PEEK flag. This appears to have resolved this rather troublesome=
 >> issue for us in production.
>>
>
> Shouldn't QuickFIX just rely on recv returning 0 to detect a socket
> disconnect, instead of relying on this ioctl?  I've never seen a
> socket-based application using this technique to detect disconnects
> before.  Clearly its not 100% reliable.
>
> --=20
> Caleb Epstein
> caleb dot epstein at gmail dot com
>
>
> -------------------------------------------------------
> SF email is sponsored by - The IT Product Guide
> Read honest & candid reviews on hundreds of IT Products from real users.
> Discover which products truly live up to the hype. Start reading now.
> http://ads.osdn.com/?ad_id=3D6595&alloc_id=3D14396&op=3Dclick
> _______________________________________________
> Quickfix-developers mailing list
> Qui...@li...
> https://lists.sourceforge.net/lists/listinfo/quickfix-developers
>=20


<font face=3D"Times New Roman" size=3D"3">
<p>------------------------------------------------------------------------=
------</p>
<p> This email is intended only for the use of the individual(s) to whom it=
 is addressed and may be privileged and confidential. Unauthorised use or d=
isclosure is prohibited. If you receive this e-mail in error, please advise=
 immediately and delete the original message. This message may have been al=
tered without your or our knowledge and the sender does not accept any liab=
ility for any errors or omissions in the message.</p>
<p>=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D</p>
</font>

Re: [Quickfix-developers] Intermittent disconnects on Solaris

[Oren M. <or...@qu...>]

Yeah, I agree.

--oren

----- Original Message ----- 
From: "Caleb Epstein" <cal...@gm...>
To: <kri...@rb...>
Cc: <qui...@li...>; <Bar...@gs...>
Sent: Monday, March 21, 2005 1:20 PM
Subject: Re: [Quickfix-developers] Intermittent disconnects on Solaris


> QuickFIX Documentation: 
> http://www.quickfixengine.org/quickfix/doc/html/index.html
> QuickFIX FAQ: http://www.quickfixengine.org/wikifix/index.php?QuickFixFAQ
> QuickFIX Support: http://www.quickfixengine.org/services.html
>
> On Mon, 21 Mar 2005 19:02:19 -0000, kri...@rb...
> <kri...@rb...> wrote:
>
>> I have isolated the cause of the intermittent disconnects caused by 
>> QuickFIX on Solaris. The problem was due to the use of the I_NREAD ioctl 
>> to determine whether a readable socket was EOF or not.
>>
>> In certain circumstances which seem to involve high network traffic and 
>> low machine load, I_NREAD will return zero for a readable socket that 
>> actually has data. In such cases, QuickFIX would erroneously close the 
>> socket.
>>
>> I replaced the I_NREAD code in socket_disconnected() one byte recv() with 
>> the MSG_PEEK flag. This appears to have resolved this rather troublesome 
>> issue for us in production.
>>
>
> Shouldn't QuickFIX just rely on recv returning 0 to detect a socket
> disconnect, instead of relying on this ioctl?  I've never seen a
> socket-based application using this technique to detect disconnects
> before.  Clearly its not 100% reliable.
>
> -- 
> Caleb Epstein
> caleb dot epstein at gmail dot com
>
>
> -------------------------------------------------------
> SF email is sponsored by - The IT Product Guide
> Read honest & candid reviews on hundreds of IT Products from real users.
> Discover which products truly live up to the hype. Start reading now.
> http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
> _______________________________________________
> Quickfix-developers mailing list
> Qui...@li...
> https://lists.sourceforge.net/lists/listinfo/quickfix-developers
> 

Re: [Quickfix-developers] Intermittent disconnects on Solaris

[Caleb E. <cal...@gm...>]

On Mon, 21 Mar 2005 19:02:19 -0000, kri...@rb...
<kri...@rb...> wrote:

> I have isolated the cause of the intermittent disconnects caused by QuickFIX on Solaris. The problem was due to the use of the I_NREAD ioctl to determine whether a readable socket was EOF or not.
> 
> In certain circumstances which seem to involve high network traffic and low machine load, I_NREAD will return zero for a readable socket that actually has data. In such cases, QuickFIX would erroneously close the socket.
> 
> I replaced the I_NREAD code in socket_disconnected() one byte recv() with the MSG_PEEK flag. This appears to have resolved this rather troublesome issue for us in production.
> 

Shouldn't QuickFIX just rely on recv returning 0 to detect a socket
disconnect, instead of relying on this ioctl?  I've never seen a
socket-based application using this technique to detect disconnects
before.  Clearly its not 100% reliable.

-- 
Caleb Epstein
caleb dot epstein at gmail dot com

[Quickfix-developers] Intermittent disconnects on Solaris

[<kri...@rb...>]

I have isolated the cause of the intermittent disconnects caused by QuickFI=
X on Solaris. The problem was due to the use of the I_NREAD ioctl to determ=
ine whether a readable socket was EOF or not.

In certain circumstances which seem to involve high network traffic and low=
 machine load, I_NREAD will return zero for a readable socket that actually=
 has data. In such cases, QuickFIX would erroneously close the socket.

I replaced the I_NREAD code in socket_disconnected() one byte recv() with t=
he MSG_PEEK flag. This appears to have resolved this rather troublesome iss=
ue for us in production.

I will send patches against 1.9.4 for this fix, as well as other changes re=
quired to get QuickFIX to compile on Solaris/SunPRO 5.3 w/Roguewave STL, af=
ter I have tested them with SunPRO and the default STL, as well as gcc on S=
olaris.

Regards,

- Kris

Utility.cpp
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
OLD
---
bool socket_disconnected( int s )
{ QF_STACK_PUSH(socket_disconnected)

  unsigned long read;
#ifdef _MSC_VER
  ioctlsocket( s, FIONREAD, &read );
#elif defined(USING_STREAMS)
  ioctl( s, I_NREAD, &read );
#else
  ioctl( s, FIONREAD, &read );
#endif
  return read =3D=3D 0;

  QF_STACK_POP
}
NEW
---
bool socket_disconnected( int s )
{ QF_STACK_PUSH(socket_disconnected)

#if defined(_MSC_VER) || !defined(USING_STREAMS)
  unsigned long read;
  #ifdef _MSC_VER
    ::ioctlsocket( s, FIONREAD, &read );
  #else
    ::ioctl( s, FIONREAD, &read );
  #endif
  return read =3D=3D 0;
#elif defined(USING_STREAMS)
  char byte;
  return ::recv (s, &byte, sizeof (byte), MSG_PEEK) <=3D 0;
#endif

  QF_STACK_POP
}


<font face=3D"Times New Roman" size=3D"3">
<p>------------------------------------------------------------------------=
------</p>
<p> This email is intended only for the use of the individual(s) to whom it=
 is addressed and may be privileged and confidential. Unauthorised use or d=
isclosure is prohibited. If you receive this e-mail in error, please advise=
 immediately and delete the original message. This message may have been al=
tered without your or our knowledge and the sender does not accept any liab=
ility for any errors or omissions in the message.</p>
<p>=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D</p>
</font>

[Quickfix-developers] SocketInitiator.stop()

[Patrick F. <pat...@ch...>]

Hello,
            There were some recent messages about the
SocketInitiator::stop() causing a deadlock.  An update was posted in cvs
where the code has a flag allowing the user to force a close.  The C#
version of SocketInitiator does not have this same flag and may be needed
because the SocketInitiator.stop() function deadlocks.  Also, and this is
real simple, is this correct way to start a fix session?
 
void start(){
initiator.start(); 
      
      //Wait for the FIX.onLogon(Session ) to be signaled. 
      WaitForConnect(CONNECT_TIMEOUT);
 
     //Fix logon      
      QuickFix44.Logon fixLogon = new Logon();
      fix.Send(fixLogon);     
 
      //Application Logon
      QuickFix44.UserRequest logon = new QuickFix44.UserRequest();
      QuoterParamaters fields = new QuoterParamaters(credentials);
      Set(logon, new UserRequestType(UserRequestType.LOGONUSER));//Set the
logon flag to true
 
      fix.Send(logon);
}
 
 
With a corresponding stop:
 
void stop()
{
      //Application Logoff
      QuickFix44.UserRequest logoff = new QuickFix44.UserRequest ();
      Set(logoff,  new UserRequestType(UserRequestType.LOGOFFUSER));
      fix.Send(logoff);
                
      //Fix logoff
      if(fix.LoggedOn)
{
            QuickFix44.Logout fixLogout = new Logout();
            fix.Send(fixLogout);
      }
                      
      /**
      * This causes the thread to hang. 
      * */
 
initiator.stop();
}
 
Thanks in advance. 
 
Patrick Flannery 
 
 

Re: [Quickfix-developers] ResendRequest onMessage

[Oren M. <or...@qu...>]

Well, except that if you have already received it and it is a dup, it will 
not be passed on to the application.  ResendRequests are designed as a 
recovery mechanism, not as a message archive.

--oren

----- Original Message ----- 
From: "Caleb Epstein" <cal...@gm...>
To: "Mo" <mo...@ab...>
Cc: <qui...@li...>
Sent: Tuesday, March 15, 2005 3:09 PM
Subject: Re: [Quickfix-developers] ResendRequest onMessage


> QuickFIX Documentation: 
> http://www.quickfixengine.org/quickfix/doc/html/index.html
> QuickFIX FAQ: http://www.quickfixengine.org/wikifix/index.php?QuickFixFAQ
> QuickFIX Support: http://www.quickfixengine.org/services.html
>
> On Tue, 15 Mar 2005 21:18:49 -0000, Mo <mo...@ab...> wrote:
>
>> Hi evrybody
>> How can i parse all the mesSages i get by doing the resendRequest.
>> When i get the messages, The onMessage function is not called. So i can 
>> not
>> get the Object it self.
>
> Messages which are being processed as a result of a ResendRequest
> should be no different than any other messages.
>
> -- 
> Caleb Epstein
> caleb dot epstein at gmail dot com
>
>
> -------------------------------------------------------
> SF email is sponsored by - The IT Product Guide
> Read honest & candid reviews on hundreds of IT Products from real users.
> Discover which products truly live up to the hype. Start reading now.
> http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
> _______________________________________________
> Quickfix-developers mailing list
> Qui...@li...
> https://lists.sourceforge.net/lists/listinfo/quickfix-developers
>