quickfix-developers

[Quickfix-developers] SendToTarget...

[John H. <jr...@ya...>]

I am interested in designing behavior in my application such that I intelligently handle a scenario whereby SendToTarget might return False.  

The scenario I am interested in handling is one where the FIX connection has been lost and I attempt to send a new order to the exchange.  My problem (as I understand it) is that if I call SendToTarget while the connection is down, QuickFix will store the message for later transmission when the session gets reestablished.  If you can imagine a scenario where a user tries to send an order which fails (perhaps because the network connection to the workstation is down), I have a potential issue with the order subsequently being transmitted at a later time when the connection is reestablished.  If the market has moved since, the order may now be so far away (or through) the market that sending it would be a big issue.

Does anybody have any insight as to what circumstances might cause SendToTarget to fail?  Similarly does anybody have any suggestions on how I might construct my application such that if a SendToTarget fails, the attempted NewOrderSingle is discarded and NOT sent on a subsequent reconnection?  (fyi I'm using the generic QF build out of the box in a .NET environment.)

I have constructed my code such that I do check that the session is logged on immediately prior to attempting to send the order, but this leaves the possibility that the session drops immediately after I check isLoggedOn but before I attempt to SendToTarget.

Any advice would be greatly appreciated.

Many thanks,

    John

Re: [Quickfix-developers] SendToTarget...

[Shane T. <str...@co...>]

There is a configuration option:

*PersistMessages* If set to N, no messages will be persisted. This will
force QuickFIX to always send GapFills instead of resending messages. Use
this if you know you never want to resend a message. Useful for market data
streams.

On Tue, Jun 9, 2009 at 10:20 AM, John Haldi <jr...@ya...> wrote:

> QuickFIX Documentation:
> http://www.quickfixengine.org/quickfix/doc/html/index.html
> QuickFIX Support: http://www.quickfixengine.org/services.html
>
>
> I am interested in designing behavior in my application such that I
> intelligently handle a scenario whereby SendToTarget might return False.
>
> The scenario I am interested in handling is one where the FIX connection
> has been lost and I attempt to send a new order to the exchange.  My problem
> (as I understand it) is that if I call SendToTarget while the connection is
> down, QuickFix will store the message for later transmission when the
> session gets reestablished.  If you can imagine a scenario where a user
> tries to send an order which fails (perhaps because the network connection
> to the workstation is down), I have a potential issue with the order
> subsequently being transmitted at a later time when the connection is
> reestablished.  If the market has moved since, the order may now be so far
> away (or through) the market that sending it would be a big issue.
>
> Does anybody have any insight as to what circumstances might cause
> SendToTarget to fail?  Similarly does anybody have any suggestions on how I
> might construct my application such that if a SendToTarget fails, the
> attempted NewOrderSingle is discarded and NOT sent on a subsequent
> reconnection?  (fyi I'm using the generic QF build out of the box in a .NET
> environment.)
>
> I have constructed my code such that I do check that the session is logged
> on immediately prior to attempting to send the order, but this leaves the
> possibility that the session drops immediately after I check isLoggedOn but
> before I attempt to SendToTarget.
>
> Any advice would be greatly appreciated.
>
> Many thanks,
>
>     John
>
>
> ------------------------------------------------------------------------------
> Crystal Reports - New Free Runtime and 30 Day Trial
> Check out the new simplified licensing option that enables unlimited
> royalty-free distribution of the report engine for externally facing
> server and web deployment.
> http://p.sf.net/sfu/businessobjects
> _______________________________________________
> Quickfix-developers mailing list
> Qui...@li...
> https://lists.sourceforge.net/lists/listinfo/quickfix-developers
>



-- 
Shane Trotter
Connamara Systems, LLC

Re: [Quickfix-developers] SendToTarget...

[Brian E. <azz...@ya...>]

John -

As I understand SendToTarget, it's going to
increment the outbound sequence number being used by QuickFix,
regardless of whether the message actually got sent. So when you
reconnect, the login will be using a higher sequence number, which will
cause a "sequence-resend" scenario.

Example: You are at seq #12 when you drop your network connection.
Two new orders get sent to SendToTarget while you are still down,
incrementing the seq # to 14. QuickFIX reestablishes the connection by
relogging in, using seq #15. Your counterparty was expecting #13, so it
asks for everything from 13 to 14. 

At this point, unless
you've turned off PersistMessages in the config file, QuickFIX will
resend the two orders. BUT, you have control over whether they are sent
via inspecting the PossDup flag - the PossDup flag is set on any
message sent due to a resend request, even if it never actually got out
the door the first time.

At that point, I will quote a previous email: 

You can add code like the following to your "toApp" callback:

void ExampleApp::toApp(FIX::Message& message, const FIX::SessionID& sessionID) throw(FIX::DoNotSend)
{
    // Don't send messages that are being resent
    try
    {
        FIX::PossDupFlag possDupFlag;
        message.getHeader().getField( possDupFlag );
        if (possDupFlag) 
            throw FIX::DoNotSend();
    }
    catch ( FIX::FieldNotFound& ) {}    

    // Do any other processing...
}
  
QuickFIX will
set the PossDup flag on any message that is being resent. You can use
that flag to prevent all (or a subset) of messages from being resent by
simply throwing the DoNotSend exception.

My rule of thumb is
never resend messages that increase market risk (orders, amendments),
but it's OK to resend messages that decrease risk (cancels) or report
status (execution reports). 

- Brian Erst
Thynk Software, Inc.


________________________________
From: John Haldi <jr...@ya...>
To: qui...@li...
Sent: Tuesday, June 9, 2009 10:20:02 AM
Subject: [Quickfix-developers] SendToTarget...

QuickFIX Documentation: http://www.quickfixengine.org/quickfix/doc/html/index.html
QuickFIX Support: http://www.quickfixengine.org/services.html

I
am interested in designing behavior in my application such that I
intelligently handle a scenario whereby SendToTarget might return
False.  

The scenario I am interested in handling is one where
the FIX connection has been lost and I attempt to send a new order to
the exchange.  My problem (as I understand it) is that if I call
SendToTarget while the connection is down, QuickFix will store the
message for later transmission when the session gets reestablished.  If
you can imagine a scenario where a user tries to send an order which
fails (perhaps because the network connection to the workstation is
down), I have a potential issue with the order subsequently being
transmitted at a later time when the connection is reestablished.  If
the market has moved since, the order may now be so far away (or
through) the market that sending it would be a big issue.

Does
anybody have any insight as to what circumstances might cause
SendToTarget to fail?  Similarly does anybody have any suggestions on
how I might construct my application such that if a SendToTarget fails,
the attempted NewOrderSingle is discarded and NOT sent on a subsequent
reconnection?  (fyi I'm using the generic QF build out of the box in a
.NET environment.)

I have constructed my code such that I do
check that the session is logged on immediately prior to attempting to
send the order, but this leaves the possibility that the session drops
immediately after I check isLoggedOn but before I attempt to
SendToTarget.

Any advice would be greatly appreciated.

Many thanks,

    John

Re: [Quickfix-developers] SendToTarget...

[Igor S. <se...@tb...>]

Hi,

It might seem like a late follow-up question on this, but let me try 
anyway -- how do you handle such "undelivered FIX messages"?

There should be a business object representing order or similar entry 
that corresponds to the undelivered FIX message the resending was 
prohibited for, hence, it seems like one would have tried to update that 
order somehow (e.g. close it) to reflect its new status.

The problem comes when the other side *did* receive that particular FIX 
message which is anyway asked for retransmission, e.g. because another 
message with lower number is missed.

Scenario:
- client logs in (logon from server is also received)
- server sees gap, and requests for resend of message in range [N..0]
- client sends NewOrderSingle with seqnum N+m -- there is no way for 
client to know that FIX resync is taking place there actually
- server receives &queues NewOrderSingle with seqnum N+m since it is 
performing sync
- client realizes that the NewOrderSingle with seqnum N+m was asked for 
resending... and prohibits its resend using the code snippet below
- client updates internal business object (e.g. closes the order)
- client receives execution report on that NewOrderSingle

Here comes the question: what would be the best handling of that 
situation at client side?

As it is seen in the scenario above FIX client has no true knowledge if 
a particular FIX message was received by other side or not when this FIX 
message is being resent.
So, the decision to update the corresponding business object can't be 
made at this point -- client might end up with closed order which 
receives an execution report (e.g. with trade) from venue later on.

It seems like the only action client can made in addition to suppressed 
resend of NewOrderSingle (here I mean only this type of messages) is to 
issue OrderCancelRequest on it and keep internal business object intact, 
and update the actual business object either on successful ACK from 
venue about deletion (with possibly other execution reports prior to 
that) _or_ on order cancel reject stating failed deletion.

Any advice would be greatly appreciated.

Thanks in advance,
Igor



On 6/9/2009 8:50 PM, Brian Erst wrote:
> John -
>
> As I understand SendToTarget, it's going to increment the outbound 
> sequence number being used by QuickFix, regardless of whether the 
> message actually got sent. So when you reconnect, the login will be 
> using a higher sequence number, which will cause a "sequence-resend" 
> scenario.
>
> Example: You are at seq #12 when you drop your network connection. Two 
> new orders get sent to SendToTarget while you are still down, 
> incrementing the seq # to 14. QuickFIX reestablishes the connection by 
> relogging in, using seq #15. Your counterparty was expecting #13, so 
> it asks for everything from 13 to 14.
>
> At this point, unless you've turned off PersistMessages in the config 
> file, QuickFIX will resend the two orders. BUT, you have control over 
> whether they are sent via inspecting the PossDup flag - the PossDup 
> flag is set on any message sent due to a resend request, even if it 
> never actually got out the door the first time.
>
> At that point, I will quote a previous email:
>
> You can add code like the following to your "toApp" callback:
>
> void ExampleApp::toApp(FIX::Message& message, const FIX::SessionID& 
> sessionID) throw(FIX::DoNotSend)
> {
>     // Don't send messages that are being resent
>     try
>     {
>         FIX::PossDupFlag possDupFlag;
>         message.getHeader().getField( possDupFlag );
>         if (possDupFlag)
>             throw FIX::DoNotSend();
>     }
>     catch ( FIX::FieldNotFound& ) {}
>
>     // Do any other processing...
> }
>
> QuickFIX will set the PossDup flag on any message that is being 
> resent. You can use that flag to prevent all (or a subset) of messages 
> from being resent by simply throwing the DoNotSend exception.
>
> My rule of thumb is never resend messages that increase market risk 
> (orders, amendments), but it's OK to resend messages that decrease 
> risk (cancels) or report status (execution reports).
>
> - Brian Erst
> Thynk Software, Inc.
>
> ------------------------------------------------------------------------
> *From:* John Haldi <jr...@ya...>
> *To:* qui...@li...
> *Sent:* Tuesday, June 9, 2009 10:20:02 AM
> *Subject:* [Quickfix-developers] SendToTarget...
>
> QuickFIX Documentation: 
> http://www.quickfixengine.org/quickfix/doc/html/index.html
> QuickFIX Support: http://www.quickfixengine.org/services.html
>
> I am interested in designing behavior in my application such that I 
> intelligently handle a scenario whereby SendToTarget might return False.
>
> The scenario I am interested in handling is one where the FIX 
> connection has been lost and I attempt to send a new order to the 
> exchange.  My problem (as I understand it) is that if I call 
> SendToTarget while the connection is down, QuickFix will store the 
> message for later transmission when the session gets reestablished.  
> If you can imagine a scenario where a user tries to send an order 
> which fails (perhaps because the network connection to the workstation 
> is down), I have a potential issue with the order subsequently being 
> transmitted at a later time when the connection is reestablished.  If 
> the market has moved since, the order may now be so far away (or 
> through) the market that sending it would be a big issue.
>
> Does anybody have any insight as to what circumstances might cause 
> SendToTarget to fail?  Similarly does anybody have any suggestions on 
> how I might construct my application such that if a SendToTarget 
> fails, the attempted NewOrderSingle is discarded and NOT sent on a 
> subsequent reconnection?  (fyi I'm using the generic QF build out of 
> the box in a .NET environment.)
>
> I have constructed my code such that I do check that the session is 
> logged on immediately prior to attempting to send the order, but this 
> leaves the possibility that the session drops immediately after I 
> check isLoggedOn but before I attempt to SendToTarget.
>
> Any advice would be greatly appreciated.
>
> Many thanks,
>
>     John

[Quickfix-developers] Resend request range for >= FIX 4.2 always specify zero EndSeqNo

[Igor S. <se...@tb...>]

Hi,

I am curious if there is a valid reason in EndSeqNo being always set as zero in resend requests for version FIX 4.2 or above.

The exact implementation of Session::generateResendRequest() looks like:
   ...
  EndSeqNo endSeqNo( msgSeqNum - 1 );
  if ( beginString >= FIX::BeginString_FIX42 )
    endSeqNo = 0;
  else if( beginString <= FIX::BeginString_FIX41 )
    endSeqNo = 999999;
  ...

So, the resolved value (msgSeqNum-1) is always ignored, and in the resulting resend request there will be either 0 or 999999 for EndSeqNo, depending on FIX version.

FIX server having this behavior causes confusion to FIX clients since they are asked for messages _already_ delivered within existing and live FIX session.

Scenario:
- client sends logon 34=N
- server sends logon, OK
- client sends many requests, obtaining numbers from N+1...N+k
- server puts incoming requests into queue since in fact it is out of sync... and issue resend request for range [N-m..0]
- client is confused, because besides the range [N-m..N-1] it is also asked for just sent messages with numbers [N+1...N+k]

The confusion mainly comes due to specific client needs, where some message types are not allowed for resending.. and by omitting some messages during this resend client assumes that these messages were not delivered to FIX server.
There seems no good suggestion for FIX client -- on some message resend it doesn't know if it was ever delivered to the other side, or?

The valid solution would be to change QF behavior for Session::generateResendRequest(), so that in the scenario above the range [N-m..N-1] is requested by FIX server.

Is there something _for_ the existing behavior, with 0 being always set in the range?

Any suggestions, comments?

Thanks,
Igor