Re: [Quickfix-users] QuickFIX/J newbie needs tips on robustness

[Joerg T. <Joe...@ma...>]

On 07/20/06 07:28, Scott Maxwell wrote:
> On 7/19/06, Jerry Westrick <je...@we...> wrote:
>=20
>> The Fix Specs say that the ClOrdID is Unique per Order.
>> I would expect your broker to check this.
>> If the ClOrdID already exists then He should reject the NewOrderSingle.
>=20
> Ah, clever idea.  I hadn't thought of just trying that.

Jerry is right here, the broker should support the OrderStatusRequest(ClO=
rdID) without the need to=20
     give the OrderID. Without this feature, you have to revert to manual=
 operation.

Otherwise the following scenario would work a bit better:

(1) Generate order with new ClOrdID in your database.
(2) Send order to QF; if sendToTarget() returns, the order is in the QF o=
utgoing queue
(3) Update database status: NEW --> SENT_TO_FIX

On recovery, you request the state of all known orders:

	OrderStatusRequest(ClOrdID)

If the crash happens between (1) and (2), the request would return "Unkno=
wn ClOrdID" since the order=20
did not even manage to go to QF.

If the crash happens between (2) and (3) and the order is still in the QF=
 outgoing queue, the=20
OrderStatusRequest is appended to the queue after the order itself. So yo=
u have to wait some time=20
until the order and the request is being processed.

The QF callback toApp() can be helpful here: Every time QF puts a message=
 on the wire, this callback=20
is triggered. If the order is sent the first time, the PossDup flag is fa=
lse. If the order is resent=20
from the queue after a crash or network issue, the PossDup flag is set to=
 true and the field=20
OrigSendingTime is populated. By comparing SendingTime and OrigSendingTim=
e you can identify a time=20
lag between the first try and the current one. For some orders this may b=
e too large, and you may=20
decide to cancel sending this order again by throwing a DoNotSend excepti=
on.

>> 2) Isn't this a rather theoretical case?Won't QuickFix Recovery get th=
is for
>> you?
>=20
> I am of course a newbie, so I can't speak too authoritatively.  Maybe
> you're right.  It just seemed to me that since there's no way to both
> (1) send the message (and guarantee it got to the broker) and (2)
> update the database, all as part of the same guaranteed-atomic
> operation, then you can always construct a scenario where you lose if
> the power goes out at just the wrong moment.

If the crash happens before (2), the OrderStatusRequest will finally upda=
te your database. After=20
(1), there is no way in FIX to guarantee that the message actually arrive=
d at the other sides=20
application layer. It may possibly stall in resend processing forever, ie=
 if your network is very=20
bad. This can be caught by time-outs and should be handled manually.

> You're right that it's a theoretical case, but you know how it is ...
> if you don't deal with it, it'll come back and bite you at the worst
> possible moment.

A good place to look for and discuss such issues are the forums at www.fi=
xprotocol.org. Here you=20
will get more feedback on questions of this more general nature. Except m=
entioning the toApp()=20
callback, my answer should apply to all compliant FIX engines.

Cheers, J=F6rg

--=20
Joerg Thoennes
                                http://macd.com
Tel.: +49 (0)241 44597-24      Macdonald Associates GmbH
Fax : +49 (0)241 44597-10      Lothringer Str. 52, D-52070 Aachen