quickfix-users

[Quickfix-users] Some questions about restarting a quickfix server

[Staffan U. <sta...@mu...>]

Hi all,

I've recently been working a bit on a FIX acceptor for order handling
using QuickFIX.  I'm using quickfixj, and have managed to get the
connection up and the initiators seem happy with the setup.  That is,
they can send orders and execution reports are sent back from my acceptor.

This is all still in the testing stage, however, and I was wondering
about what to do whenever a connection is reconnected after a
disconnect, and how to handle a restart of the server.  How much of
gap filling and resending is handled by the quickfix engine itself?
What do I have to do?  I realize the answer might be too complex for a
short mail reply, but I've not been able to find much documentation
either.  Any pointers?

When the server is restarted, maybe after a crash, one way to find out
if there are any orders whose status is not known by the client is to
read the message log.  Is there any way to do that while using the
quickfix engine's message parsing functions?

Staffan

[Quickfix-users] Some questions about restarting a quickfix server

[Staffan U. <st...@ul...>]

Hi all,

I've recently been working a bit on a FIX acceptor for order handling
using QuickFIX.  I'm using quickfixj, and have managed to get the
connection up and the initiators seem happy with the setup.  That is,
they can send orders and execution reports are sent back from my acceptor.

This is all still in the testing stage, however, and I was wondering
about what to do whenever a connection is reconnected after a
disconnect, and how to handle a restart of the server.  How much of
gap filling and resending is handled by the quickfix engine itself?
What do I have to do?  I realize the answer might be too complex for a
short mail reply, but I've not been able to find much documentation
either.  Any pointers?

When the server is restarted, maybe after a crash, one way to find out
if there are any orders whose status is not known by the client is to
read the message log.  Is there any way to do that while using the
quickfix engine's message parsing functions?

Staffan

Re: [Quickfix-users] Some questions about restarting a quickfix server

[Joerg T. <Joe...@ma...>]

Hi Staffan,

> I've recently been working a bit on a FIX acceptor for order handling
> using QuickFIX.  I'm using quickfixj, and have managed to get the
> connection up and the initiators seem happy with the setup.  That is,
> they can send orders and execution reports are sent back from my accept=
or.
>=20
> This is all still in the testing stage, however, and I was wondering
> about what to do whenever a connection is reconnected after a
> disconnect, and how to handle a restart of the server.  How much of
> gap filling and resending is handled by the quickfix engine itself?
> What do I have to do?  I realize the answer might be too complex for a
> short mail reply, but I've not been able to find much documentation
> either.  Any pointers?

All the session level gap fill/resend processing is done for you by the Q=
uickFIX engine.

But there are some fine points where you can influence the process:

Look at this scenario: You are sending an order to the exchange, but the =
connection breaks=20
down. QuickFIX takes care of your NewOrderSingle and will resend it as so=
on as the=20
connection is up again and the other side requested it as part of a Resen=
dRequest.

BUT: This may take a while and your order would be sent 10 mins too late.

Before sending the NewOrderSingle again, QF will pass it through the toAp=
p() callback.
Here you can check for the PossDup flag and if it is true, you can retrie=
ve the=20
OrigSendingTime, at which the order has been sent the first time. If the =
difference=20
between SendingTime (now) and OrigSendingTime is too large, you can throw=
 a DoNotSend=20
exception and QF will gap fill the message instead of sending the outdate=
d order.

> When the server is restarted, maybe after a crash, one way to find out
> if there are any orders whose status is not known by the client is to
> read the message log.  Is there any way to do that while using the
> quickfix engine's message parsing functions?

If the QF sendToTarget() method returns, the QF engine will make sure tha=
t the message=20
arrives at the client end after a crash or disconnection. So you just hav=
e to record which=20
of your order status related message you have already sent to QF and star=
t at this point.

If you are unsure whether you have already sent some application level me=
ssages to the=20
other side, you may flag messages with the PossResend flag which indicate=
s that the same=20
business information (e.g. a Fill) may have sent before to the other side=
 in another=20
message (different MsgSeqNo). Please do not confuse this with PossDup (it=
 relates to the=20
_same_ MsgSeqNo and is handled by the FIX engine).

Cheers, J=F6rg

--=20
Joerg Thoennes
                                http://macd.com
Tel.: +49 (0)241 44597-24      Macdonald Associates GmbH
Fax : +49 (0)241 44597-10      Lothringer Str. 52, D-52070 Aachen

Re: [Quickfix-users] Some questions about restarting a quickfix server

[Staffan U. <sta...@mu...>]

Hello again, 

Thanks Joerg for the info!

Is there any other source for documentation except the main web site
and the souce code?  I find that many minor points (for example,
is an outgoing FIX message guaranteed to be written to the message
store before it is sent to the client?  When are incoming messages
written to the message store?) are hard to find answers to...

> > When the server is restarted, maybe after a crash, one way to find out
> > if there are any orders whose status is not known by the client is to
> > read the message log.  Is there any way to do that while using the
> > quickfix engine's message parsing functions?
> 
> If the QF sendToTarget() method returns, the QF engine will make sure
> that the message arrives at the client end after a crash or
> disconnection. So you just have to record which of your order status
> related message you have already sent to QF and start at this point.
> 
> If you are unsure whether you have already sent some application level
> messages to the other side, you may flag messages with the PossResend
> flag which indicates that the same business information (e.g. a Fill)
> may have sent before to the other side in another message (different
> MsgSeqNo). Please do not confuse this with PossDup (it relates to the
> _same_ MsgSeqNo and is handled by the FIX engine).

Yes, I realize I could have my own database to record whatever
information has been sent to the counterparties I'm communicating
with.

I was thinking along these lines, however: I'm writing a FIX gateway
for a proprietary exchange API.  Since the proprietary API allows
me to chose the order id, I could let that order id be a
string composed of the FIX client's compid and the FIX client order
id.  Then, at application start, I could read the FIX log to find all
orders that are still active as far as the FIX client knows.  Since
the client order ids are enough to identify the orders at the
exchange, I could just query their state and send out execution
reports with their current status, if different from what has already
been sent.  (Or, just send out all information including trades for
these orders, with the PossResend flag set.)

Anyway, then I have to parse the FIX message log...  Not very hard to
do by hand since there are only a couple of messages of interest, but
it would of course be nice to be able to use all of the type safe
functions in the QuickFIX api...

Staffan

Re: [Quickfix-users] Some questions about restarting a quickfix server

[Joerg T. <Joe...@ma...>]

Staffan Ulfberg wrote:
> Is there any other source for documentation except the main web site
> and the souce code?  I find that many minor points (for example,
> is an outgoing FIX message guaranteed to be written to the message
> store before it is sent to the client?

If the sendToTarget() method returns, the message has been written to the=
 store at least.
QuickFIX takes care to send out the message when appropriate, ie when a F=
IX connection has=20
been established or a ResendRequest is being processed.

If you use a database-backed store, the database will guarantee that the =
message is stored=20
savely. File-based storage may be not so safe.

> When are incoming messages written to the message store?) are hard to f=
ind answers
> to...

There is no need to write them to the store. You just receive them via fr=
omApp().

>>>When the server is restarted, maybe after a crash, one way to find out
>>>if there are any orders whose status is not known by the client is to
>>>read the message log.  Is there any way to do that while using the
>>>quickfix engine's message parsing functions?
>> [...]
>=20
> I was thinking along these lines, however: I'm writing a FIX gateway
> for a proprietary exchange API.  Since the proprietary API allows
> me to chose the order id, I could let that order id be a
> string composed of the FIX client's compid and the FIX client order
> id.  Then, at application start, I could read the FIX log to find all
> orders that are still active as far as the FIX client knows.  Since
> the client order ids are enough to identify the orders at the
> exchange, I could just query their state and send out execution
> reports with their current status, if different from what has already
> been sent.  (Or, just send out all information including trades for
> these orders, with the PossResend flag set.)
>=20
> Anyway, then I have to parse the FIX message log...  Not very hard to
> do by hand since there are only a couple of messages of interest, but
> it would of course be nice to be able to use all of the type safe
> functions in the QuickFIX api...

If I remember right, there has been some discussions about using the mess=
age log or store=20
for session recovery in this list. Please have a look at earlier posts.

Cheers, J=F6rg

--=20
Joerg Thoennes
                                http://macd.com
Tel.: +49 (0)241 44597-24      Macdonald Associates GmbH
Fax : +49 (0)241 44597-10      Lothringer Str. 52, D-52070 Aachen