quickfix-developers

[Quickfix-developers] QuickFIX sending extra Logouts?

[Caleb E. <cal...@gm...>]

I'm working on a Berkeley DB-backed MessageStore, and am
torture-testing it with some small applications.  I've run across some
behavior that I think illustrates a bug in QuickFIX.

I have a slightly modified version of the "executor" example app as my
FIX server.  I modified it to fill MKT orders (just sends a hard-coded
price of 1) and send an ACK before every execution report. 
Additionally, I installed a signal handler to catch SIGINT and SIGTERM
and do a clean shutdown (the "wait" function just detects that the
signal was caught and exits now).

As my client, I wrote a small Application which just reads orders from
a text file and sends them to the server and produces some benchmark
timings.

These are both working fine.  Now I'm trying to break things (to test
the robustness of both QuickFIX and Berkeley DB) and see how well they
can recover.

On the client side, I have my application drop its connection as soon
as it is done sending orders (I am just returning from
Application::run).  I'm trying to cause messages to be "queued up" on
the sending side so they will need to retransmitted on the next
connection, but my client application ends up blocking until all the
messages in response to its order flow have been received.  This is
fine, but while it is blocking (after Application::run has returned),
it appears to send a new Logout message for each message it receives
from the executor, and this causes the executor to log the message
"Logon state is not valid for message" for each of these.

I am able to create a similar scenario by interrupting the executor
when it is in the middle of receiving a number of messages from the
client.  In this case, I shut down the SocketAcceptor and the executor
ends up sending extra Logout messages to the client.  It looks like
the same problem, just reversed.

This scenario is reproducible every time, even if I use the FileStore
instead of my Berkeley DB MessageStore.  I'm attaching some logs that
illustrate it.  The directory "logs.executor-killed" is the first
scenario and "logs.client-exited" is the second.  Please let me know
if more info is needed to help debug this.

-- 
Caleb Epstein
cal...@gm...

Re: [Quickfix-developers] QuickFIX sending extra Logouts?

[Hans B. <han...@ma...>]

On Mon, 2004-09-20 at 23:51, Caleb Epstein wrote:
> I'm working on a Berkeley DB-backed MessageStore, and am
>...........

Good news. If we really gonna use QuickFix, I'll have to make Oracle
support, so I'll see what you did.

BTW: if there is anything about using other database, please let me
know. I've read, that people are using Oracle, but have not contributed
it back to the project.

Hans Bausewein

Re: [Quickfix-developers] QuickFIX sending extra Logouts?

[Joerg T. <Joe...@ma...>]

Hans Bausewein wrote:
> On Mon, 2004-09-20 at 23:51, Caleb Epstein wrote:
> 
>>I'm working on a Berkeley DB-backed MessageStore, and am
>>...........
> 
> 
> Good news. If we really gonna use QuickFix, I'll have to make Oracle
> support, so I'll see what you did.
> 
> BTW: if there is anything about using other database, please let me
> know. I've read, that people are using Oracle, but have not contributed
> it back to the project.

Actually, I shall file a enhancement request on the QF bugtracker to get 
PostgreSQL support and put myself as the assigned person. We use 
PostgreSQL here.

Cheers, Jörg

-- 
Joerg Thoennes
                                http://macd.com
Tel.: +49 (0)241 44597-24      Macdonald Associates GmbH
Fax : +49 (0)241 44597-10      Lothringer Str. 52, D-52070 Aachen

Re: [Quickfix-developers] QuickFIX sending extra Logouts?

[Hans B. <han...@ma...>]

On Tue, 2004-09-21 at 11:26, Joerg Thoennes wrote:
> Actually, I shall file a enhancement request on the QF bugtracker to get 
> PostgreSQL support and put myself as the assigned person. We use 
> PostgreSQL here.

Also a good one. PostgreSQL is a lot closer to Oracle than Berkely DB or
MySQL. I've done some Java projects for both Oracle and PostgreSQL and I
also have a PostgreSQL server.

Hans Bausewein

Re: [Quickfix-developers] QuickFIX sending extra Logouts?

[Joerg T. <Joe...@ma...>]

Hans Bausewein wrote:
> On Tue, 2004-09-21 at 11:26, Joerg Thoennes wrote:
> 
>>Actually, I shall file a enhancement request on the QF bugtracker to get 
>>PostgreSQL support and put myself as the assigned person. We use 
>>PostgreSQL here.
> 
> Also a good one. PostgreSQL is a lot closer to Oracle than Berkely DB or
> MySQL. I've done some Java projects for both Oracle and PostgreSQL and I
> also have a PostgreSQL server.

I already added a RFE to the bugtracker. And committed myself...

Cheers, Jörg

-- 
Joerg Thoennes
                                http://macd.com
Tel.: +49 (0)241 44597-24      Macdonald Associates GmbH
Fax : +49 (0)241 44597-10      Lothringer Str. 52, D-52070 Aachen

[Quickfix-developers] Re: QuickFIX sending extra Logouts?

[Oren M. <or...@qu...>]

A bug report has been filed: 
http://www.quickfixengine.org/bugtracker/bug.php?op=show&bugid=8

We'll take a look at this.

--oren

On Sep 20, 2004, at 4:51 PM, Caleb Epstein wrote:

> I'm working on a Berkeley DB-backed MessageStore, and am
> torture-testing it with some small applications.  I've run across some
> behavior that I think illustrates a bug in QuickFIX.
>
> I have a slightly modified version of the "executor" example app as my
> FIX server.  I modified it to fill MKT orders (just sends a hard-coded
> price of 1) and send an ACK before every execution report.
> Additionally, I installed a signal handler to catch SIGINT and SIGTERM
> and do a clean shutdown (the "wait" function just detects that the
> signal was caught and exits now).
>
> As my client, I wrote a small Application which just reads orders from
> a text file and sends them to the server and produces some benchmark
> timings.
>
> These are both working fine.  Now I'm trying to break things (to test
> the robustness of both QuickFIX and Berkeley DB) and see how well they
> can recover.
>
> On the client side, I have my application drop its connection as soon
> as it is done sending orders (I am just returning from
> Application::run).  I'm trying to cause messages to be "queued up" on
> the sending side so they will need to retransmitted on the next
> connection, but my client application ends up blocking until all the
> messages in response to its order flow have been received.  This is
> fine, but while it is blocking (after Application::run has returned),
> it appears to send a new Logout message for each message it receives
> from the executor, and this causes the executor to log the message
> "Logon state is not valid for message" for each of these.
>
> I am able to create a similar scenario by interrupting the executor
> when it is in the middle of receiving a number of messages from the
> client.  In this case, I shut down the SocketAcceptor and the executor
> ends up sending extra Logout messages to the client.  It looks like
> the same problem, just reversed.
>
> This scenario is reproducible every time, even if I use the FileStore
> instead of my Berkeley DB MessageStore.  I'm attaching some logs that
> illustrate it.  The directory "logs.executor-killed" is the first
> scenario and "logs.client-exited" is the second.  Please let me know
> if more info is needed to help debug this.
>
> -- 
> Caleb Epstein
> cal...@gm...
> <extra-logouts.tar.gz>

Re: [Quickfix-developers] QuickFIX sending extra Logouts?

[Oren M. <or...@qu...>]

Caleb,

I think this may be a simple fix.  In Session::next(), try changing 
this code block:

         if( !m_state.sentLogout() )
           m_state.onEvent( "Initiated logout request" );
         generateLogout();

to this:

         if( !m_state.sentLogout() )
         {
           m_state.onEvent( "Initiated logout request" );
           generateLogout();
         }

Let me know how your tests turn out after making this change.

--oren

On Sep 20, 2004, at 4:51 PM, Caleb Epstein wrote:

> I'm working on a Berkeley DB-backed MessageStore, and am
> torture-testing it with some small applications.  I've run across some
> behavior that I think illustrates a bug in QuickFIX.
>
> I have a slightly modified version of the "executor" example app as my
> FIX server.  I modified it to fill MKT orders (just sends a hard-coded
> price of 1) and send an ACK before every execution report.
> Additionally, I installed a signal handler to catch SIGINT and SIGTERM
> and do a clean shutdown (the "wait" function just detects that the
> signal was caught and exits now).
>
> As my client, I wrote a small Application which just reads orders from
> a text file and sends them to the server and produces some benchmark
> timings.
>
> These are both working fine.  Now I'm trying to break things (to test
> the robustness of both QuickFIX and Berkeley DB) and see how well they
> can recover.
>
> On the client side, I have my application drop its connection as soon
> as it is done sending orders (I am just returning from
> Application::run).  I'm trying to cause messages to be "queued up" on
> the sending side so they will need to retransmitted on the next
> connection, but my client application ends up blocking until all the
> messages in response to its order flow have been received.  This is
> fine, but while it is blocking (after Application::run has returned),
> it appears to send a new Logout message for each message it receives
> from the executor, and this causes the executor to log the message
> "Logon state is not valid for message" for each of these.
>
> I am able to create a similar scenario by interrupting the executor
> when it is in the middle of receiving a number of messages from the
> client.  In this case, I shut down the SocketAcceptor and the executor
> ends up sending extra Logout messages to the client.  It looks like
> the same problem, just reversed.
>
> This scenario is reproducible every time, even if I use the FileStore
> instead of my Berkeley DB MessageStore.  I'm attaching some logs that
> illustrate it.  The directory "logs.executor-killed" is the first
> scenario and "logs.client-exited" is the second.  Please let me know
> if more info is needed to help debug this.
>
> -- 
> Caleb Epstein
> cal...@gm...
> <extra-logouts.tar.gz>

Re: [Quickfix-developers] QuickFIX sending extra Logouts?

[Caleb E. <cal...@gm...>]

This patch seems to fix the problem.  I'm no longer seeing the flood
of logout messages getting generated.

-- 
Caleb Epstein
cal...@gm...