Thread: [Quickfix-users] Two more questions about QuickFix
Brought to you by:
orenmnero
Menu
▾
▴
quickfix-users
|
From: Dan G. <dua...@ya...> - 2008-02-08 18:03:04
|
Hi Everyone! we notice two issues and want to hear whether anyone knows what is going on. 1. when i sent FIX msgs at higher rate (10 msg per second), there seems a longer latency for the msg to hit market. i have strong suspecion QuickFix responsible for the longer latency, as i isolated other segments' contributions to the overall latency. can anyone comment on this? 2. our broker's network engineers noticed there is TCP back-pressure alert on their site. It basically means the FIX messages got queued on our broker's side. Either because our network connection is not adequate or (and) our QuickFix code cannot process them fast enough. did anyone in this list encounter the similiar issue? any comments? Thank you in advance. Dan |
|
From: Djalma R. d. S. F. <drs...@gm...> - 2008-02-09 17:26:51
|
Yes, I have this kind of problem when dealing with hundreds or thousands of messages per second and most of the times with applications in C# using the QuickFIX .NET API. I agree that 10 msg/sec should not be so hard for QF. On Feb 8, 2008 8:20 PM, Oren Miller <or...@qu...> wrote: > QuickFIX Documentation: > http://www.quickfixengine.org/quickfix/doc/html/index.html > QuickFIX Support: http://www.quickfixengine.org/services.html > > > Yeah, please keep in mind that as long as you are in your callback, > QuickFIX cannot pull the next message. Everything that goes into fromApp > becomes part of the message processing, and if the code in there takes time, > then QuickFIX is essentially blocked from processing on that thread. If you > must do a lot of processing, then stick your message in a queue and get out > of there. In the end this probably won't solve your real problem because it > indicates you cannot handle the messages as fast as they are being sent to > you. If you are building a real time system, this will be an issues for you > regardless. > --oren > > On Feb 8, 2008, at 3:18 PM, Ted Graham wrote: > > QuickFIX Documentation: > http://www.quickfixengine.org/quickfix/doc/html/index.html > QuickFIX Support: http://www.quickfixengine.org/services.html > > You are talking about 10 msg/sec. At that sending rate, QFJ will have NO > trouble keeping up. Look in your code to find what is wrong, QFJ can handle > significantly higher message rates that what you are describing. > > > > As for C++ QF vs .NET or Java, I did a fair amount of testing and saw > negligible differences in speed between C++ QF w/ JNI and the 100% Java > QFJ. QFJ may actually be faster, depending on your scenario. > > > > Ted > > > ------------------------------ > > *From:* qui...@li... [ > mailto:qui...@li...<qui...@li...>] > *On Behalf Of *ying shi > *Sent:* Friday, February 08, 2008 1:11 PM > *To:* Qui...@li... > *Subject:* Re: [Quickfix-users] Two more questions about QuickFix > > > > Dan: > > > > Djalma's analysis is not bad. And I suggest that if you have enough time > to trace the bottleneck, you'd better integrate > > QF's source code into your app and build together, so that you can debug > and profile inside its code to find it. > > > > I would also suggest use C++ QF lib rather than .Net or Java ones. > > > > On 08/02/2008, *Djalma Rosa dos Santos Filho* <drs...@gm...> > wrote: > > QuickFIX Documentation: > http://www.quickfixengine.org/quickfix/doc/html/index.html > QuickFIX Support: http://www.quickfixengine.org/services.html > > > > Hi, > > This really looks like the TCP buffer is getting full too fast. If > possible, try to set a larger TCP window or find and improve the bottleneck > in your application or in QuickFIX. > > With my experience, I've noticed that even with a very fast application > routine in FromApp, QuickFIX is always faster when sending and much slower > when receiving and I believe that this is the main reason for the issue you > are reporting, which is not so uncommon for applications that use QF and > ocassionally are required to process at a higher rate. > > In part this is a TCP issue, but I believe that the message receiving > process in QuickFIX must be optimized at some point in the long road where > the bytes travel in the engine. > > {Recv}->{Parser}->{Validation}->[.NET Interop]->[Message > Cracker]->{Application Code} > > I have some suspicions, but without a deeper investigation and some > profiling I would not dare to cast the first stone. > > Djalma > > On Feb 8, 2008 4:31 PM, Ajay Kamdar <Aja...@tr...> wrote: > > QuickFIX Documentation: > http://www.quickfixengine.org/quickfix/doc/html/index.html > QuickFIX Support: http://www.quickfixengine.org/services.html > > > > Dan, > > You should also take a look at the application specific processing logic > in the fromApp() method and determine how long it takes to run, because > it too could be contributing to the slow processing of FIX messages sent > by your counter party. > > - Ajay > > > -----Original Message----- > From: qui...@li... > [mailto:qui...@li...] On Behalf Of Dan > Guo > Sent: Friday, February 08, 2008 1:03 PM > To: qui...@li... > Subject: [Quickfix-users] Two more questions about QuickFix > > QuickFIX Documentation: > http://www.quickfixengine.org/quickfix/doc/html/index.html > QuickFIX Support: http://www.quickfixengine.org/services.html > > Hi Everyone! > > we notice two issues and want to hear whether anyone knows what is going > on. > > 1. when i sent FIX msgs at higher rate (10 msg per second), there seems > a longer latency for the msg to hit market. > > i have strong suspecion QuickFix responsible for the longer latency, > as i isolated other segments' contributions to the overall latency. > > can anyone comment on this? > > 2. our broker's network engineers noticed there is TCP back-pressure > alert on their site. It basically means the FIX messages got queued on > our broker's side. Either because our network connection is not adequate > or (and) our QuickFix code cannot process them fast enough. > > did anyone in this list encounter the similiar issue? any comments? > > Thank you in advance. > > Dan > > ------------------------------------------------------------------------ > - > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Quickfix-users mailing list > Qui...@li... > https://lists.sourceforge.net/lists/listinfo/quickfix-users > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Quickfix-users mailing list > Qui...@li... > https://lists.sourceforge.net/lists/listinfo/quickfix-users > > > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Quickfix-users mailing list > Qui...@li... > https://lists.sourceforge.net/lists/listinfo/quickfix-users > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/_______________________________________________ > Quickfix-users mailing list > Qui...@li... > https://lists.sourceforge.net/lists/listinfo/quickfix-users > > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Quickfix-users mailing list > Qui...@li... > https://lists.sourceforge.net/lists/listinfo/quickfix-users > > |
|
From: yihan <yi...@ho...> - 2008-03-28 13:55:36
|
Do you know the performace of quickfix ? How many messages per second ? Right now I didn't put any code in function fromApp and those callbacks (command out mine code), but I noticed there is above 4 sec delay once I got traffic above 400 msg/sec. Any ideas about how to improve the performance of quickfix? yihan Djalma Rosa dos Santos Filho wrote: > > QuickFIX Documentation: > http://www.quickfixengine.org/quickfix/doc/html/index.html > QuickFIX Support: http://www.quickfixengine.org/services.html > > > Yes, I have this kind of problem when dealing with hundreds or thousands > of > messages per second and most of the times with applications in C# using > the > QuickFIX .NET API. > > I agree that 10 msg/sec should not be so hard for QF. > > On Feb 8, 2008 8:20 PM, Oren Miller <or...@qu...> wrote: > >> QuickFIX Documentation: >> http://www.quickfixengine.org/quickfix/doc/html/index.html >> QuickFIX Support: http://www.quickfixengine.org/services.html >> >> >> Yeah, please keep in mind that as long as you are in your callback, >> QuickFIX cannot pull the next message. Everything that goes into fromApp >> becomes part of the message processing, and if the code in there takes >> time, >> then QuickFIX is essentially blocked from processing on that thread. If >> you >> must do a lot of processing, then stick your message in a queue and get >> out >> of there. In the end this probably won't solve your real problem because >> it >> indicates you cannot handle the messages as fast as they are being sent >> to >> you. If you are building a real time system, this will be an issues for >> you >> regardless. >> --oren >> >> On Feb 8, 2008, at 3:18 PM, Ted Graham wrote: >> >> QuickFIX Documentation: >> http://www.quickfixengine.org/quickfix/doc/html/index.html >> QuickFIX Support: http://www.quickfixengine.org/services.html >> >> You are talking about 10 msg/sec. At that sending rate, QFJ will have NO >> trouble keeping up. Look in your code to find what is wrong, QFJ can >> handle >> significantly higher message rates that what you are describing. >> >> >> >> As for C++ QF vs .NET or Java, I did a fair amount of testing and saw >> negligible differences in speed between C++ QF w/ JNI and the 100% Java >> QFJ. QFJ may actually be faster, depending on your scenario. >> >> >> >> Ted >> >> >> ------------------------------ >> >> *From:* qui...@li... [ >> mailto:qui...@li...<qui...@li...>] >> *On Behalf Of *ying shi >> *Sent:* Friday, February 08, 2008 1:11 PM >> *To:* Qui...@li... >> *Subject:* Re: [Quickfix-users] Two more questions about QuickFix >> >> >> >> Dan: >> >> >> >> Djalma's analysis is not bad. And I suggest that if you have enough time >> to trace the bottleneck, you'd better integrate >> >> QF's source code into your app and build together, so that you can debug >> and profile inside its code to find it. >> >> >> >> I would also suggest use C++ QF lib rather than .Net or Java ones. >> >> >> >> On 08/02/2008, *Djalma Rosa dos Santos Filho* <drs...@gm...> >> wrote: >> >> QuickFIX Documentation: >> http://www.quickfixengine.org/quickfix/doc/html/index.html >> QuickFIX Support: http://www.quickfixengine.org/services.html >> >> >> >> Hi, >> >> This really looks like the TCP buffer is getting full too fast. If >> possible, try to set a larger TCP window or find and improve the >> bottleneck >> in your application or in QuickFIX. >> >> With my experience, I've noticed that even with a very fast application >> routine in FromApp, QuickFIX is always faster when sending and much >> slower >> when receiving and I believe that this is the main reason for the issue >> you >> are reporting, which is not so uncommon for applications that use QF and >> ocassionally are required to process at a higher rate. >> >> In part this is a TCP issue, but I believe that the message receiving >> process in QuickFIX must be optimized at some point in the long road >> where >> the bytes travel in the engine. >> >> {Recv}->{Parser}->{Validation}->[.NET Interop]->[Message >> Cracker]->{Application Code} >> >> I have some suspicions, but without a deeper investigation and some >> profiling I would not dare to cast the first stone. >> >> Djalma >> >> On Feb 8, 2008 4:31 PM, Ajay Kamdar <Aja...@tr...> wrote: >> >> QuickFIX Documentation: >> http://www.quickfixengine.org/quickfix/doc/html/index.html >> QuickFIX Support: http://www.quickfixengine.org/services.html >> >> >> >> Dan, >> >> You should also take a look at the application specific processing logic >> in the fromApp() method and determine how long it takes to run, because >> it too could be contributing to the slow processing of FIX messages sent >> by your counter party. >> >> - Ajay >> >> >> -----Original Message----- >> From: qui...@li... >> [mailto:qui...@li...] On Behalf Of Dan >> Guo >> Sent: Friday, February 08, 2008 1:03 PM >> To: qui...@li... >> Subject: [Quickfix-users] Two more questions about QuickFix >> >> QuickFIX Documentation: >> http://www.quickfixengine.org/quickfix/doc/html/index.html >> QuickFIX Support: http://www.quickfixengine.org/services.html >> >> Hi Everyone! >> >> we notice two issues and want to hear whether anyone knows what is going >> on. >> >> 1. when i sent FIX msgs at higher rate (10 msg per second), there seems >> a longer latency for the msg to hit market. >> >> i have strong suspecion QuickFix responsible for the longer latency, >> as i isolated other segments' contributions to the overall latency. >> >> can anyone comment on this? >> >> 2. our broker's network engineers noticed there is TCP back-pressure >> alert on their site. It basically means the FIX messages got queued on >> our broker's side. Either because our network connection is not adequate >> or (and) our QuickFix code cannot process them fast enough. >> >> did anyone in this list encounter the similiar issue? any comments? >> >> Thank you in advance. >> >> Dan >> >> ------------------------------------------------------------------------ >> - >> This SF.net email is sponsored by: Microsoft >> Defy all challenges. Microsoft(R) Visual Studio 2008. >> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ >> _______________________________________________ >> Quickfix-users mailing list >> Qui...@li... >> https://lists.sourceforge.net/lists/listinfo/quickfix-users >> >> ------------------------------------------------------------------------- >> This SF.net email is sponsored by: Microsoft >> Defy all challenges. Microsoft(R) Visual Studio 2008. >> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ >> _______________________________________________ >> Quickfix-users mailing list >> Qui...@li... >> https://lists.sourceforge.net/lists/listinfo/quickfix-users >> >> >> >> >> ------------------------------------------------------------------------- >> This SF.net email is sponsored by: Microsoft >> Defy all challenges. Microsoft(R) Visual Studio 2008. >> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ >> _______________________________________________ >> Quickfix-users mailing list >> Qui...@li... >> https://lists.sourceforge.net/lists/listinfo/quickfix-users >> >> >> ------------------------------------------------------------------------- >> This SF.net email is sponsored by: Microsoft >> Defy all challenges. Microsoft(R) Visual Studio 2008. >> >> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/_______________________________________________ >> Quickfix-users mailing list >> Qui...@li... >> https://lists.sourceforge.net/lists/listinfo/quickfix-users >> >> >> >> ------------------------------------------------------------------------- >> This SF.net email is sponsored by: Microsoft >> Defy all challenges. Microsoft(R) Visual Studio 2008. >> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ >> _______________________________________________ >> Quickfix-users mailing list >> Qui...@li... >> https://lists.sourceforge.net/lists/listinfo/quickfix-users >> >> > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Quickfix-users mailing list > Qui...@li... > https://lists.sourceforge.net/lists/listinfo/quickfix-users > > -- View this message in context: http://www.nabble.com/Two-more-questions-about-QuickFix-tp15360781p16349797.html Sent from the QuickFIX - User mailing list archive at Nabble.com. |
×
Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.
Thanks for helping keep SourceForge clean.
X