Re: [Quickfix-users] Two more questions about QuickFix
Brought to you by:
orenmnero
Menu
▾
▴
Re: [Quickfix-users] Two more questions about QuickFix
|
From: <or...@qu...> - 2008-03-28 15:31:36
|
<html><body><div>Do you have any more information about what you are doing? This doesn't really give a whole lot to comment on. How is this test setup?</div> <div> </div> <div>--oren</div> <BLOCKQUOTE style="PADDING-LEFT: 8px; MARGIN-LEFT: 8px; BORDER-LEFT: blue 2px solid" webmail="1">-------- Original Message --------<BR>Subject: Re: [Quickfix-users] Two more questions about QuickFix<BR>From: yihan <yi...@ho...><BR>Date: Fri, March 28, 2008 8:55 am<BR>To: <a href="mailto:qui...@li...">qui...@li...</a><BR><BR>QuickFIX Documentation: <A href="http://www.quickfixengine.org/quickfix/doc/html/index.html" target=_blank><a href="http://www.quickfixengine.org/quickfix/doc/html/index.html">http://www.quickfixengine.org/quickfix/doc/html/index.html</a></A><BR>QuickFIX Support: <A href="http://www.quickfixengine.org/services.html" target=_blank><a href="http://www.quickfixengine.org/services.html">http://www.quickfixengine.org/services.html</a></A><BR><BR><BR>Do you know the performace of quickfix ? How many messages per second ? Right<BR>now I didn't put any code in function fromApp and those callbacks (command<BR>out mine code), but I noticed there is above 4 sec delay once I got traffic<BR>above 400 msg/sec. Any ideas about how to improve the performance of<BR>quickfix?<BR><BR>yihan<BR><BR><BR>Djalma Rosa dos Santos Filho wrote:<BR>> <BR>> QuickFIX Documentation:<BR>> <A href="http://www.quickfixengine.org/quickfix/doc/html/index.html" target=_blank><a href="http://www.quickfixengine.org/quickfix/doc/html/index.html">http://www.quickfixengine.org/quickfix/doc/html/index.html</a></A><BR>> QuickFIX Support: <A href="http://www.quickfixengine.org/services.html" target=_blank><a href="http://www.quickfixengine.org/services.html">http://www.quickfixengine.org/services.html</a></A><BR>> <BR>> <BR>> Yes, I have this kind of problem when dealing with hundreds or thousands<BR>> of<BR>> messages per second and most of the times with applications in C# using<BR>> the<BR>> QuickFIX .NET API.<BR>> <BR>> I agree that 10 msg/sec should not be so hard for QF.<BR>> <BR>> On Feb 8, 2008 8:20 PM, Oren Miller <<A onclick="Popup.composeWindow('pcompose.php?sendto=oren%40quickfixengine.org'); return false;" href="#Compose">oren<B></B>@quickfixengine.org</A>> wrote:<BR>> <BR>>> QuickFIX Documentation:<BR>>> <A href="http://www.quickfixengine.org/quickfix/doc/html/index.html" target=_blank><a href="http://www.quickfixengine.org/quickfix/doc/html/index.html">http://www.quickfixengine.org/quickfix/doc/html/index.html</a></A><BR>>> QuickFIX Support: <A href="http://www.quickfixengine.org/services.html" target=_blank><a href="http://www.quickfixengine.org/services.html">http://www.quickfixengine.org/services.html</a></A><BR>>><BR>>><BR>>> Yeah, please keep in mind that as long as you are in your callback,<BR>>> QuickFIX cannot pull the next message. Everything that goes into fromApp<BR>>> becomes part of the message processing, and if the code in there takes<BR>>> time,<BR>>> then QuickFIX is essentially blocked from processing on that thread. If<BR>>> you<BR>>> must do a lot of processing, then stick your message in a queue and get<BR>>> out<BR>>> of there. In the end this probably won't solve your real problem because<BR>>> it<BR>>> indicates you cannot handle the messages as fast as they are being sent<BR>>> to<BR>>> you. If you are building a real time system, this will be an issues for<BR>>> you<BR>>> regardless.<BR>>> --oren<BR>>><BR>>> On Feb 8, 2008, at 3:18 PM, Ted Graham wrote:<BR>>><BR>>> QuickFIX Documentation:<BR>>> <A href="http://www.quickfixengine.org/quickfix/doc/html/index.html" target=_blank><a href="http://www.quickfixengine.org/quickfix/doc/html/index.html">http://www.quickfixengine.org/quickfix/doc/html/index.html</a></A><BR>>> QuickFIX Support: <A href="http://www.quickfixengine.org/services.html" target=_blank><a href="http://www.quickfixengine.org/services.html">http://www.quickfixengine.org/services.html</a></A><BR>>><BR>>> You are talking about 10 msg/sec. At that sending rate, QFJ will have NO<BR>>> trouble keeping up. Look in your code to find what is wrong, QFJ can<BR>>> handle<BR>>> significantly higher message rates that what you are describing.<BR>>><BR>>><BR>>><BR>>> As for C++ QF vs .NET or Java, I did a fair amount of testing and saw<BR>>> negligible differences in speed between C++ QF w/ JNI and the 100% Java<BR>>> QFJ. QFJ may actually be faster, depending on your scenario.<BR>>><BR>>><BR>>><BR>>> Ted<BR>>><BR>>><BR>>> ------------------------------<BR>>><BR>>> *From:* <A onclick="Popup.composeWindow('pcompose.php?sendto=quickfix-users-bounces%40lists.sourceforge.net'); return false;" href="#Compose">quickfix-users-bounces<B></B>@lists.sourceforge.net</A> [<BR>>> mailto:<A onclick="Popup.composeWindow('pcompose.php?sendto=quickfix-users-bounces%40lists.sourceforge.net'); return false;" href="#Compose">quickfix-users-bounces<B></B>@lists.sourceforge.net</A><<A onclick="Popup.composeWindow('pcompose.php?sendto=quickfix-users-bounces%40lists.sourceforge.net'); return false;" href="#Compose">quickfix-users-bounces<B></B>@lists.sourceforge.net</A>>]<BR>>> *On Behalf Of *ying shi<BR>>> *Sent:* Friday, February 08, 2008 1:11 PM<BR>>> *To:* <A onclick="Popup.composeWindow('pcompose.php?sendto=Quickfix-users%40lists.sourceforge.net'); return false;" href="#Compose">Quickfix-users<B></B>@lists.sourceforge.net</A><BR>>> *Subject:* Re: [Quickfix-users] Two more questions about QuickFix<BR>>><BR>>><BR>>><BR>>> Dan:<BR>>><BR>>><BR>>><BR>>> Djalma's analysis is not bad. And I suggest that if you have enough time<BR>>> to trace the bottleneck, you'd better integrate<BR>>><BR>>> QF's source code into your app and build together, so that you can debug<BR>>> and profile inside its code to find it.<BR>>><BR>>><BR>>><BR>>> I would also suggest use C++ QF lib rather than .Net or Java ones.<BR>>><BR>>><BR>>><BR>>> On 08/02/2008, *Djalma Rosa dos Santos Filho* <<A onclick="Popup.composeWindow('pcompose.php?sendto=drsantosfilho%40gmail.com'); return false;" href="#Compose">drsantosfilho<B></B>@gmail.com</A>><BR>>> wrote:<BR>>><BR>>> QuickFIX Documentation:<BR>>> <A href="http://www.quickfixengine.org/quickfix/doc/html/index.html" target=_blank><a href="http://www.quickfixengine.org/quickfix/doc/html/index.html">http://www.quickfixengine.org/quickfix/doc/html/index.html</a></A><BR>>> QuickFIX Support: <A href="http://www.quickfixengine.org/services.html" target=_blank><a href="http://www.quickfixengine.org/services.html">http://www.quickfixengine.org/services.html</a></A><BR>>><BR>>><BR>>><BR>>> Hi,<BR>>><BR>>> This really looks like the TCP buffer is getting full too fast. If<BR>>> possible, try to set a larger TCP window or find and improve the<BR>>> bottleneck<BR>>> in your application or in QuickFIX.<BR>>><BR>>> With my experience, I've noticed that even with a very fast application<BR>>> routine in FromApp, QuickFIX is always faster when sending and much<BR>>> slower<BR>>> when receiving and I believe that this is the main reason for the issue<BR>>> you<BR>>> are reporting, which is not so uncommon for applications that use QF and<BR>>> ocassionally are required to process at a higher rate.<BR>>><BR>>> In part this is a TCP issue, but I believe that the message receiving<BR>>> process in QuickFIX must be optimized at some point in the long road<BR>>> where<BR>>> the bytes travel in the engine.<BR>>><BR>>> {Recv}->{Parser}->{Validation}->[.NET Interop]->[Message<BR>>> Cracker]->{Application Code}<BR>>><BR>>> I have some suspicions, but without a deeper investigation and some<BR>>> profiling I would not dare to cast the first stone.<BR>>><BR>>> Djalma<BR>>><BR>>> On Feb 8, 2008 4:31 PM, Ajay Kamdar <<A onclick="Popup.composeWindow('pcompose.php?sendto=Ajay.Kamdar%40tradeweb.com'); return false;" href="#Compose">Ajay.Kamdar<B></B>@tradeweb.com</A>> wrote:<BR>>><BR>>> QuickFIX Documentation:<BR>>> <A href="http://www.quickfixengine.org/quickfix/doc/html/index.html" target=_blank><a href="http://www.quickfixengine.org/quickfix/doc/html/index.html">http://www.quickfixengine.org/quickfix/doc/html/index.html</a></A><BR>>> QuickFIX Support: <A href="http://www.quickfixengine.org/services.html" target=_blank><a href="http://www.quickfixengine.org/services.html">http://www.quickfixengine.org/services.html</a></A><BR>>><BR>>><BR>>><BR>>> Dan,<BR>>><BR>>> You should also take a look at the application specific processing logic<BR>>> in the fromApp() method and determine how long it takes to run, because<BR>>> it too could be contributing to the slow processing of FIX messages sent<BR>>> by your counter party.<BR>>><BR>>> - Ajay<BR>>><BR>>><BR>>> -----Original Message-----<BR>>> From: <A onclick="Popup.composeWindow('pcompose.php?sendto=quickfix-users-bounces%40lists.sourceforge.net'); return false;" href="#Compose">quickfix-users-bounces<B></B>@lists.sourceforge.net</A><BR>>> [mailto:<A onclick="Popup.composeWindow('pcompose.php?sendto=quickfix-users-bounces%40lists.sourceforge.net'); return false;" href="#Compose">quickfix-users-bounces<B></B>@lists.sourceforge.net</A>] On Behalf Of Dan<BR>>> Guo<BR>>> Sent: Friday, February 08, 2008 1:03 PM<BR>>> To: <A onclick="Popup.composeWindow('pcompose.php?sendto=quickfix-users%40lists.sourceforge.net'); return false;" href="#Compose">quickfix-users<B></B>@lists.sourceforge.net</A><BR>>> Subject: [Quickfix-users] Two more questions about QuickFix<BR>>><BR>>> QuickFIX Documentation:<BR>>> <A href="http://www.quickfixengine.org/quickfix/doc/html/index.html" target=_blank><a href="http://www.quickfixengine.org/quickfix/doc/html/index.html">http://www.quickfixengine.org/quickfix/doc/html/index.html</a></A><BR>>> QuickFIX Support: <A href="http://www.quickfixengine.org/services.html" target=_blank><a href="http://www.quickfixengine.org/services.html">http://www.quickfixengine.org/services.html</a></A><BR>>><BR>>> Hi Everyone!<BR>>><BR>>> we notice two issues and want to hear whether anyone knows what is going<BR>>> on.<BR>>><BR>>> 1. when i sent FIX msgs at higher rate (10 msg per second), there seems<BR>>> a longer latency for the msg to hit market.<BR>>><BR>>> i have strong suspecion QuickFix responsible for the longer latency,<BR>>> as i isolated other segments' contributions to the overall latency.<BR>>><BR>>> can anyone comment on this?<BR>>><BR>>> 2. our broker's network engineers noticed there is TCP back-pressure<BR>>> alert on their site. It basically means the FIX messages got queued on<BR>>> our broker's side. Either because our network connection is not adequate<BR>>> or (and) our QuickFix code cannot process them fast enough.<BR>>><BR>>> did anyone in this list encounter the similiar issue? any comments?<BR>>><BR>>> Thank you in advance.<BR>>><BR>>> Dan<BR>>><BR>>> ------------------------------------------------------------------------<BR>>> -<BR>>> This <a href="http://SF.net">SF.net</a> email is sponsored by: Microsoft<BR>>> Defy all challenges. Microsoft(R) Visual Studio 2008.<BR>>> <A href="http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/" target=_blank><a href="http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/">http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/</a></A><BR>>> _______________________________________________<BR>>> Quickfix-users mailing list<BR>>> <A onclick="Popup.composeWindow('pcompose.php?sendto=Quickfix-users%40lists.sourceforge.net'); return false;" href="#Compose">Quickfix-users<B></B>@lists.sourceforge.net</A><BR>>> <A href="https://lists.sourceforge.net/lists/listinfo/quickfix-users" target=_blank><a href="https://lists.sourceforge.net/lists/listinfo/quickfix-users">https://lists.sourceforge.net/lists/listinfo/quickfix-users</a></A><BR>>><BR>>> -------------------------------------------------------------------------<BR>>> This <a href="http://SF.net">SF.net</a> email is sponsored by: Microsoft<BR>>> Defy all challenges. Microsoft(R) Visual Studio 2008.<BR>>> <A href="http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/" target=_blank><a href="http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/">http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/</a></A><BR>>> _______________________________________________<BR>>> Quickfix-users mailing list<BR>>> <A onclick="Popup.composeWindow('pcompose.php?sendto=Quickfix-users%40lists.sourceforge.net'); return false;" href="#Compose">Quickfix-users<B></B>@lists.sourceforge.net</A><BR>>> <A href="https://lists.sourceforge.net/lists/listinfo/quickfix-users" target=_blank><a href="https://lists.sourceforge.net/lists/listinfo/quickfix-users">https://lists.sourceforge.net/lists/listinfo/quickfix-users</a></A><BR>>><BR>>><BR>>><BR>>><BR>>> -------------------------------------------------------------------------<BR>>> This <a href="http://SF.net">SF.net</a> email is sponsored by: Microsoft<BR>>> Defy all challenges. Microsoft(R) Visual Studio 2008.<BR>>> <A href="http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/" target=_blank><a href="http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/">http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/</a></A><BR>>> _______________________________________________<BR>>> Quickfix-users mailing list<BR>>> <A onclick="Popup.composeWindow('pcompose.php?sendto=Quickfix-users%40lists.sourceforge.net'); return false;" href="#Compose">Quickfix-users<B></B>@lists.sourceforge.net</A><BR>>> <A href="https://lists.sourceforge.net/lists/listinfo/quickfix-users" target=_blank><a href="https://lists.sourceforge.net/lists/listinfo/quickfix-users">https://lists.sourceforge.net/lists/listinfo/quickfix-users</a></A><BR>>><BR>>><BR>>> -------------------------------------------------------------------------<BR>>> This <a href="http://SF.net">SF.net</a> email is sponsored by: Microsoft<BR>>> Defy all challenges. Microsoft(R) Visual Studio 2008.<BR>>><BR>>> <A href="http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/_______________________________________________" target=_blank><a href="http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/_______________________________________________">http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/_______________________________________________</a></A><BR>>> Quickfix-users mailing list<BR>>> <A onclick="Popup.composeWindow('pcompose.php?sendto=Quickfix-users%40lists.sourceforge.net'); return false;" href="#Compose">Quickfix-users<B></B>@lists.sourceforge.net</A><BR>>> <A href="https://lists.sourceforge.net/lists/listinfo/quickfix-users" target=_blank><a href="https://lists.sourceforge.net/lists/listinfo/quickfix-users">https://lists.sourceforge.net/lists/listinfo/quickfix-users</a></A><BR>>><BR>>><BR>>><BR>>> -------------------------------------------------------------------------<BR>>> This <a href="http://SF.net">SF.net</a> email is sponsored by: Microsoft<BR>>> Defy all challenges. Microsoft(R) Visual Studio 2008.<BR>>> <A href="http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/" target=_blank><a href="http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/">http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/</a></A><BR>>> _______________________________________________<BR>>> Quickfix-users mailing list<BR>>> <A onclick="Popup.composeWindow('pcompose.php?sendto=Quickfix-users%40lists.sourceforge.net'); return false;" href="#Compose">Quickfix-users<B></B>@lists.sourceforge.net</A><BR>>> <A href="https://lists.sourceforge.net/lists/listinfo/quickfix-users" target=_blank><a href="https://lists.sourceforge.net/lists/listinfo/quickfix-users">https://lists.sourceforge.net/lists/listinfo/quickfix-users</a></A><BR>>><BR>>><BR>> <BR>> -------------------------------------------------------------------------<BR>> This <a href="http://SF.net">SF.net</a> email is sponsored by: Microsoft<BR>> Defy all challenges. Microsoft(R) Visual Studio 2008.<BR>> <A href="http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/" target=_blank><a href="http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/">http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/</a></A><BR>> _______________________________________________<BR>> Quickfix-users mailing list<BR>> <A onclick="Popup.composeWindow('pcompose.php?sendto=Quickfix-users%40lists.sourceforge.net'); return false;" href="#Compose">Quickfix-users<B></B>@lists.sourceforge.net</A><BR>> <A href="https://lists.sourceforge.net/lists/listinfo/quickfix-users" target=_blank><a href="https://lists.sourceforge.net/lists/listinfo/quickfix-users">https://lists.sourceforge.net/lists/listinfo/quickfix-users</a></A><BR>> <BR>> <BR><BR>-- <BR>View this message in context: <A href="http://www.nabble.com/Two-more-questions-about-QuickFix-tp15360781p16349797.html" target=_blank><a href="http://www.nabble.com/Two-more-questions-about-QuickFix-tp15360781p16349797.html">http://www.nabble.com/Two-more-questions-about-QuickFix-tp15360781p16349797.html</a></A><BR>Sent from the QuickFIX - User mailing list archive at <a href="http://Nabble.com">Nabble.com</a>.<BR><BR><BR>-------------------------------------------------------------------------<BR>Check out the new <a href="http://SourceForge.net">SourceForge.net</a> Marketplace.<BR>It's the best place to buy or sell services for<BR>just about anything Open Source.<BR><A href="http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace" target=_blank><a href="http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace">http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace</a></A><BR>_______________________________________________<BR>Quickfix-users mailing list<BR><A onclick="Popup.composeWindow('pcompose.php?sendto=Quickfix-users%40lists.sourceforge.net'); return false;" href="#Compose">Quickfix-users<B></B>@lists.sourceforge.net</A><BR><A href="https://lists.sourceforge.net/lists/listinfo/quickfix-users" target=_blank><a href="https://lists.sourceforge.net/lists/listinfo/quickfix-users">https://lists.sourceforge.net/lists/listinfo/quickfix-users</a></A><BR></BLOCKQUOTE></body></html> |
