quickfix-developers

[Quickfix-developers] Messages not reaching application (MessageCracker) layer

[Rick L. <ric...@gm...>]

Greetings,

I've recently discovered a very unsettling bug in my trading platform, and I
wanted to ask a fairly general question to see if this has bene seen
anywhere else (I'm assuming it's a bug in *my *code, but it can't hurt to
ask the group)...

When a New Order Single message is generated by the user, I don't send an
acknowledgment of that order until I get the ExecutionReport confirming the
order, in my

onMessage(ExecutionReport message, QuickFix.SessionID session)

MessageCracker message handler.  Whenever I get this confirmation, I add
this particular order to my database, to my order books, etc.

What appears to be happening, in very rare instances, is I'm not getting
this ExecutionReport for some fills.  What this means is, my platform
doesn't know the order exists, but it's still working in the Exchange's eyes
-- so there's not even a way to cancel the order unless I call the exchange
or use their web-based front-end to cancel the order.

We run some automated trading applications that can have upwards of a *million
*orders in a given day (constantly adding/pulling orders, etc.) -- and this
bug occurs, usually, once every two days.  So it's a very low percentage of
the time.

I've spoken with the exchange, and they are showing that ExecutionReports *were
*in fact sent (and QuickFix would issue a resend request if the message
wasn't sent and a subsequent message had a higher sequence number).  Since
this isn't occurring, I'm wondering if there is some way the message might
be received by the QuickFix layer and just isn't sent up to the
MessageCracker layer.  I have try-catch blocks in my handler, and no
exceptions are being thrown, so I don't know how I could be missing this
acknowledgment.

Has anyone seen anything even remotely similar?

Re: [Quickfix-developers] Messages not reaching application (MessageCracker) layer

[Rick L. <ric...@gm...>]

What I meant to say below was:

"What appears to be happening, in very rare instances, is I'm not getting
this ExecutionReport for some *orders *"

On Tue, Dec 2, 2008 at 2:07 PM, Rick Lane <ric...@gm...> wrote:

> Greetings,
>
> I've recently discovered a very unsettling bug in my trading platform, and
> I wanted to ask a fairly general question to see if this has bene seen
> anywhere else (I'm assuming it's a bug in *my *code, but it can't hurt to
> ask the group)...
>
> When a New Order Single message is generated by the user, I don't send an
> acknowledgment of that order until I get the ExecutionReport confirming the
> order, in my
>
> onMessage(ExecutionReport message, QuickFix.SessionID session)
>
> MessageCracker message handler.  Whenever I get this confirmation, I add
> this particular order to my database, to my order books, etc.
>
> What appears to be happening, in very rare instances, is I'm not getting
> this ExecutionReport for some fills.  What this means is, my platform
> doesn't know the order exists, but it's still working in the Exchange's eyes
> -- so there's not even a way to cancel the order unless I call the exchange
> or use their web-based front-end to cancel the order.
>
> We run some automated trading applications that can have upwards of a *million
> *orders in a given day (constantly adding/pulling orders, etc.) -- and
> this bug occurs, usually, once every two days.  So it's a very low
> percentage of the time.
>
> I've spoken with the exchange, and they are showing that ExecutionReports
> *were *in fact sent (and QuickFix would issue a resend request if the
> message wasn't sent and a subsequent message had a higher sequence number).
> Since this isn't occurring, I'm wondering if there is some way the message
> might be received by the QuickFix layer and just isn't sent up to the
> MessageCracker layer.  I have try-catch blocks in my handler, and no
> exceptions are being thrown, so I don't know how I could be missing this
> acknowledgment.
>
> Has anyone seen anything even remotely similar?
>

Re: [Quickfix-developers] Messages not reaching application (MessageCracker) layer

[Rick L. <ric...@gm...>]

Greetings,

I have at least one more piece of the puzzle regarding these "missing"
ExecutionReport messages in my fromApp() callback method.  I enabled a
FileLogFactory, and waited until the bug occurred again, and indeed a 35=8
(ExecutionReport "New" confirmation) message immediately proceeded my 35=D
(NewOrderSingle) message.

What I'm wondering -- without really know too terribly much about what's
going on under the hood in QuickFix -- is if it's still possible that the
FileLogFactory saw the exchanges 35=8 response message but it still didn't
reach the fromApp() callback?

I've gone through my fromApp() callback about a thousand times, and cannot
determine where the code could be breaking (without an exception being
thrown).  One of the first few lines of code in this callback is writing the
execution report to my SQL Server database for audit-trail purposes, and
this line is missing from my database every time.

I'm just wondering if the presence in the FileLog is a guarantee that the
bug occurs *after* the fromApp() callback...

Thanks,
Rick

On Tue, Dec 2, 2008 at 2:11 PM, Rick Lane <ric...@gm...> wrote:

> What I meant to say below was:
>
> "What appears to be happening, in very rare instances, is I'm not getting
> this ExecutionReport for some *orders *"
>
>
> On Tue, Dec 2, 2008 at 2:07 PM, Rick Lane <ric...@gm...> wrote:
>
>> Greetings,
>>
>> I've recently discovered a very unsettling bug in my trading platform, and
>> I wanted to ask a fairly general question to see if this has bene seen
>> anywhere else (I'm assuming it's a bug in *my *code, but it can't hurt to
>> ask the group)...
>>
>> When a New Order Single message is generated by the user, I don't send an
>> acknowledgment of that order until I get the ExecutionReport confirming the
>> order, in my
>>
>> onMessage(ExecutionReport message, QuickFix.SessionID session)
>>
>> MessageCracker message handler.  Whenever I get this confirmation, I add
>> this particular order to my database, to my order books, etc.
>>
>> What appears to be happening, in very rare instances, is I'm not getting
>> this ExecutionReport for some fills.  What this means is, my platform
>> doesn't know the order exists, but it's still working in the Exchange's eyes
>> -- so there's not even a way to cancel the order unless I call the exchange
>> or use their web-based front-end to cancel the order.
>>
>> We run some automated trading applications that can have upwards of a *million
>> *orders in a given day (constantly adding/pulling orders, etc.) -- and
>> this bug occurs, usually, once every two days.  So it's a very low
>> percentage of the time.
>>
>> I've spoken with the exchange, and they are showing that ExecutionReports
>> *were *in fact sent (and QuickFix would issue a resend request if the
>> message wasn't sent and a subsequent message had a higher sequence number).
>> Since this isn't occurring, I'm wondering if there is some way the message
>> might be received by the QuickFix layer and just isn't sent up to the
>> MessageCracker layer.  I have try-catch blocks in my handler, and no
>> exceptions are being thrown, so I don't know how I could be missing this
>> acknowledgment.
>>
>> Has anyone seen anything even remotely similar?
>>
>
>

Re: [Quickfix-developers] Messages not reaching application(MessageCracker) layer

[Rick L. <ric...@gm...>]

John,

Greetings -- I'm actually not using an Acceptor -- I'm using a
ThreadedSocketInitiator; this is merely a client that connects to the
exchange; it's not acting as a QuickFix server.

Does your theory/thought still stand?  Also, while I'm using a
ThreadedSocketInitiator, I only connect to one session.  I thought I read
somewhere, a long time ago, that even if you're using a single session, this
should be used in lieu of SocketInitiator.  I could be mistaken.

Thanks,
Rick

On Tue, Dec 2, 2008 at 2:27 PM, John Haldi <jo...@ha...> wrote:

>  Rick,
>
> Are you using the threaded socket acceptor with multiple sessions?
>
>    jh
>
>  ------------------------------
> *From:* Rick Lane [mailto:ric...@gm...]
> *Sent:* Tuesday, December 02, 2008 3:07 PM
> *To:* qui...@li...
> *Subject:* [Quickfix-developers] Messages not reaching
> application(MessageCracker) layer
>
> Greetings,
>
> I've recently discovered a very unsettling bug in my trading platform, and
> I wanted to ask a fairly general question to see if this has bene seen
> anywhere else (I'm assuming it's a bug in *my *code, but it can't hurt to
> ask the group)...
>
> When a New Order Single message is generated by the user, I don't send an
> acknowledgment of that order until I get the ExecutionReport confirming the
> order, in my
>
> onMessage(ExecutionReport message, QuickFix.SessionID session)
>
> MessageCracker message handler.  Whenever I get this confirmation, I add
> this particular order to my database, to my order books, etc.
>
> What appears to be happening, in very rare instances, is I'm not getting
> this ExecutionReport for some fills.  What this means is, my platform
> doesn't know the order exists, but it's still working in the Exchange's eyes
> -- so there's not even a way to cancel the order unless I call the exchange
> or use their web-based front-end to cancel the order.
>
> We run some automated trading applications that can have upwards of a *million
> *orders in a given day (constantly adding/pulling orders, etc.) -- and
> this bug occurs, usually, once every two days.  So it's a very low
> percentage of the time.
>
> I've spoken with the exchange, and they are showing that ExecutionReports
> *were *in fact sent (and QuickFix would issue a resend request if the
> message wasn't sent and a subsequent message had a higher sequence number).
> Since this isn't occurring, I'm wondering if there is some way the message
> might be received by the QuickFix layer and just isn't sent up to the
> MessageCracker layer.  I have try-catch blocks in my handler, and no
> exceptions are being thrown, so I don't know how I could be missing this
> acknowledgment.
>
> Has anyone seen anything even remotely similar?
>
> No virus found in this incoming message.
> Checked by AVG - http://www.avg.com
> Version: 8.0.176 / Virus Database: 270.9.12/1824 - Release Date: 12/2/2008
> 9:31 AM
>
>