quickfix-developers

[Quickfix-developers] when does quickfix log messages sent to exchange?

[Rick L. <ric...@gm...>]

Greetings,
In my trade server application, for every action (user-generated and
ExecutionReport generated) I persist this data in a SQL Server database.
 For speed issues, as events occur, I add them to a producer/consumer queue,
which another thread then takes and writes to the underlying database.  I'm
attempting to figure out a way to prevent the following scenario from
occurring:

1) trader requests a new order
2) server sends order request to exchange
3) server adds "new order" entry to the queue to be written to the database
4) SERVER CRASH

In other words, I've sent a message to the exchange and haven't logged it
(I've put it on the queue to be logged, but because I don't want to have to
wait for a SQL Server transaction to occur before sending the order to the
exchange, this is all done on a separate thread).

So when I reconnect, the exchange responds with an execution report (confirm
NewOrderSingle) for an order that I do not know existed.

So then I began wondering how QuickFix logs messages sent to the exchange.
 Do you

1) send to exchange
2) write to the message store

or vice versa?  I would imagine you do it in the order above, otherwise you
would be subject to the speed of writing it to file or database.

Just curious how QuickFix does this, and if anyone has any advice for me and
my problem, I'm all ears.

Much appreciated.

Rick

Re: [Quickfix-developers] when does quickfix log messages sent toexchange?

[Marcus M. <Mar...@wi...>]

Hi Rick,

I had the same sort of questions when designing our system and I ended
up using the following:

1. Message is received.
2. Engine writes message to a persistence layer.
3. Engine sends request to exchange.

The first version of our engine used a multicast socket to send messages
to a set of servers where the database processors lived. This allowed us
to fire off the message quickly without too much overhead. The downside
is that multicast isn't reliable and it's possible to miss messages so I
had to work in sequence number checking routines. Since then I have
reworked the persistence layer to use a JMS messaging system (Progress
Sonic) to support a more transactional scenario.

1. If we have a server problem then we didn't receive the message; so
we're ok as we'll conditionally process it during the re-send request
phase of the Engine start-up.

2. If the message persistence fails then we throw an exception (write
the message to disk) and respond to the client with a NAK. Our
monitoring system is notified and human intervention takes over to
resolve the technical and business issues.

3. If the exchange has a problem and we don't receive a response within
a time period we timeout the order and send a message back to the
client. If we then receive a response after the timeout period it's sent
on to the monitoring system for human intervention.

Hope this helps.

Regards,
Marcus

*** IMPORTANT NOTICE ***

This message (including any attachments) is confidential and is for the intended recipient only. If you are not the intended recipient, please inform the sender and delete any copies from your system. Internet communications are not secure and therefore Winterflood Securities Ltd does not accept legal responsibility for any of its contents (including any attachments) and any view expressed by the sender as these are not necessarily the views of Winterflood Securities Ltd.


Although all emails sent and received by Winterflood Securities Ltd are passed through extensive virus scanning technologies, we cannot guarantee the email (including attachments) is virus free. You should take whatever measures you deem appropriate within your organisation to ensure maximum protection.

Registered Office: The Atrium Building, Cannon Bridge, 25 Dowgate Hill, London, EC4R 2GA. Registered in England No. 2242204.

Winterflood Securities Limited is authorised and regulated by the Financial Services Authority.