Re: [Quickfix-users] Two more questions about QuickFix

[Dan G. <dua...@ya...>]

John,
thanks. here we did a tcp dump and found this  - two orders were sent consecutively back to back and the 2nd order is 6mill seconds after 1st one, on socket level. If that holds, 160 orders/s is the best we can do. we certainly think this is not acceptable. we wonder 
1. if there is any way to improve;
2. if this is what others see;
3. if there is any configuration error on our part.

thanks,
Dan

----- Original Message ----
From: John Haldi <jh...@ca...>
To: Dan Guo <dua...@ya...>
Sent: Friday, February 8, 2008 1:52:04 PM
Subject: RE: [Quickfix-users] Two more questions about QuickFix


Dan,

1.  On sending, you should log when you are actually calling the SendToTarget method and determine how long the delay is between calls within your code.  Once you call SendToTarget, quickfix will take over and manage the socket connection to the broker.  Depending on your line bandwidth and latency, as well as their ability to process things, this could be a bottleneck.  If however you are seeing delays between successive calls to SendToTarget, the delay is in your code.  QuickFix will manage any line delays by handling its own queue internally, so it won't cause any noticeable delays between successive calls to SendToTarget.  You could also consider logging a millisecond timestamp both before you call SendToTarget and immediately after the method returns, in order to get a sense of the time quickfix is taking to release your thread.

2.  When your broker is sending messages to you, quickfix ought to handle any delays in your code by transparently creating and managing an inbound queue on your side.  You clear the queue by handling each onMessage event.  If your code is the bottleneck, you won't react to the onMessage event until your code has finished processing the previous message.  One way to get around this problem (if that is indeed what you are experiencing) is to create your own managed queue.  Essentially you limit your code in the onMessage event to grabbing the message and putting it onto your own managed queue.  Then, on a separate thread, you have a queue processor which clears the queue.  This gives you the flexibility to easily monitor any queue length in your managed queue.

If however your broker is in fact seeing a bottleneck on their side in their ability to send you messages, it is likely a problem with your network connection to the broker.  You might want to investigate the TCP traffic and determine if there is a problem between you and the broker.  Incorrect net card drivers or router configuration can cause the TCP protocol to progressively reduce the frame size in an attempt to determine an optimal sending packet size.  If this shrinks enough, it will manifest itself in seemingly slow network connectivity as the protocol takes each of your messages and breaks them into tiny chunks and then has to reassemble them on the other side before passing them up to your application.

If it helps you to get a benchmark for what to expect, on our systems we typically see about a 1,000 max message per second throughput using the single threaded socket initiator and socket acceptor object, configured for file (not DB) logging and with the .body files turned on.  This is on a system using high speed dedicated lines between us and the exchanges/brokers in question.  We have not gone further in our attempts to maximize throughput yet, as we only do a few hundred thousand orders per day.  On a 4xCPU, 4GB RAM server running right now, I see CPU load at about 2% on our engine and its running about 20 messages per second right now.

Hope this helps,

    John

-----Original Message-----
From: qui...@li... [mailto:qui...@li...] On Behalf Of Dan Guo
Sent: Friday, February 08, 2008 1:03 PM
To: qui...@li...
Subject: [Quickfix-users] Two more questions about QuickFix

QuickFIX Documentation: http://www.quickfixengine.org/quickfix/doc/html/index.html
QuickFIX Support: http://www.quickfixengine.org/services.html

Hi Everyone!

we notice two issues and want to hear whether anyone knows what is going on.

1. when i sent FIX msgs at higher rate (10 msg per second), there seems a longer latency for the msg to hit market.

  i have strong suspecion QuickFix responsible for the longer latency, as i isolated other segments' contributions to the overall latency.

  can anyone comment on this?

2. our broker's network engineers noticed there is TCP back-pressure alert on their site. It basically means the FIX messages got queued on our broker's side. Either because our network connection is not adequate or (and) our QuickFix code cannot process them fast enough.

  did anyone in this list encounter the similiar issue? any comments?

Thank you in advance.

Dan

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Quickfix-users mailing list
Qui...@li...
https://lists.sourceforge.net/lists/listinfo/quickfix-users

No virus found in this incoming message.
Checked by AVG Free Edition.
Version: 7.5.516 / Virus Database: 269.19.21/1266 - Release Date: 2/8/2008 10:06 AM


No virus found in this outgoing message.
Checked by AVG Free Edition.
Version: 7.5.516 / Virus Database: 269.19.21/1266 - Release Date: 2/8/2008 10:06 AM