quickfix-developers

[Quickfix-developers] QuickFix performance?

[Rick L. <ric...@gm...>]

Greetings,

Is there any documentation regarding the performance (e.g. message 
throughput capabilities) of QuickFix?  I'm running into issues where it 
seems that I'm receiving messages too slowly (I emailed the list a 
couple days ago regarding fill notifications -- namely, if I place an 
immediate-fill order for, say, 1000 contracts, it takes nearly 2.5 
seconds to receive all of the fill notifications).  This may have 
something to do with the CME (perhaps on the environment on which I'm 
testing they throttle fill notifications) but I don't believe this is 
the case.  I've pretty much eliminated any other threading/etc. that my 
application is doing to try to isolate the performance solely to 
QuickFix and I'm still seeing the same results.

Has anyone run into this before, and is there some advice as to why 
these notifications might be taking so long to be processed?

Thanks,
RL

Re: [Quickfix-developers] QuickFix performance?

[Djalma R. d. S. F. <drs...@gm...>]

Hi Rick,

I don't know if this is the same, but we are also suffering with
low-throughput message receiving when using quickfix in a C# GUI
application in the moment when there is a remote connection sending
thousands of messages. Because of this, we're planning to modify the
application's architecture by adding some stuff like MVC, queues and
threading pools to try to maximize the throughput. The current
application design is inefficient and I believe in some reasons for
that, things related to the way how Quickfix is working and some TCP
socket issues...
But, in order to know if maybe this is the same with you, let me ask
you some questions for clarification.
When you say that you could isolate quickfix performance, what exactly
do you mean? Nothing in fromApp() ?
Are you processing all messages in the context of the quickfix thread?
How heavy is this message processing?
Are you using the message cracker?

Djalma


On Dec 19, 2007 12:59 PM, Rick Lane <ric...@gm...> wrote:
> QuickFIX Documentation: http://www.quickfixengine.org/quickfix/doc/html/index.html
> QuickFIX Support: http://www.quickfixengine.org/services.html
>
> Greetings,
>
> Is there any documentation regarding the performance (e.g. message
> throughput capabilities) of QuickFix?  I'm running into issues where it
> seems that I'm receiving messages too slowly (I emailed the list a
> couple days ago regarding fill notifications -- namely, if I place an
> immediate-fill order for, say, 1000 contracts, it takes nearly 2.5
> seconds to receive all of the fill notifications).  This may have
> something to do with the CME (perhaps on the environment on which I'm
> testing they throttle fill notifications) but I don't believe this is
> the case.  I've pretty much eliminated any other threading/etc. that my
> application is doing to try to isolate the performance solely to
> QuickFix and I'm still seeing the same results.
>
> Has anyone run into this before, and is there some advice as to why
> these notifications might be taking so long to be processed?
>
> Thanks,
> RL
>
> -------------------------------------------------------------------------
> SF.Net email is sponsored by:
> Check out the new SourceForge.net Marketplace.
> It's the best place to buy or sell services
> for just about anything Open Source.
> http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
> _______________________________________________
> Quickfix-developers mailing list
> Qui...@li...
> https://lists.sourceforge.net/lists/listinfo/quickfix-developers
>

Re: [Quickfix-developers] QuickFix performance?

[Oren M. <or...@qu...>]

One thing to note for the C# API, make sure you are disposing all  
your messages.  The ones you send as well as the ones you receive.   
This will make a BIG difference in high throughput systems.

--oren

On Dec 19, 2007, at 10:05 PM, Djalma Rosa dos Santos Filho wrote:

> QuickFIX Documentation: http://www.quickfixengine.org/quickfix/doc/ 
> html/index.html
> QuickFIX Support: http://www.quickfixengine.org/services.html
>
> Hi Rick,
>
> I don't know if this is the same, but we are also suffering with
> low-throughput message receiving when using quickfix in a C# GUI
> application in the moment when there is a remote connection sending
> thousands of messages. Because of this, we're planning to modify the
> application's architecture by adding some stuff like MVC, queues and
> threading pools to try to maximize the throughput. The current
> application design is inefficient and I believe in some reasons for
> that, things related to the way how Quickfix is working and some TCP
> socket issues...
> But, in order to know if maybe this is the same with you, let me ask
> you some questions for clarification.
> When you say that you could isolate quickfix performance, what exactly
> do you mean? Nothing in fromApp() ?
> Are you processing all messages in the context of the quickfix thread?
> How heavy is this message processing?
> Are you using the message cracker?
>
> Djalma
>
>
> On Dec 19, 2007 12:59 PM, Rick Lane <ric...@gm...> wrote:
>> QuickFIX Documentation: http://www.quickfixengine.org/quickfix/doc/ 
>> html/index.html
>> QuickFIX Support: http://www.quickfixengine.org/services.html
>>
>> Greetings,
>>
>> Is there any documentation regarding the performance (e.g. message
>> throughput capabilities) of QuickFix?  I'm running into issues  
>> where it
>> seems that I'm receiving messages too slowly (I emailed the list a
>> couple days ago regarding fill notifications -- namely, if I place an
>> immediate-fill order for, say, 1000 contracts, it takes nearly 2.5
>> seconds to receive all of the fill notifications).  This may have
>> something to do with the CME (perhaps on the environment on which I'm
>> testing they throttle fill notifications) but I don't believe this is
>> the case.  I've pretty much eliminated any other threading/etc.  
>> that my
>> application is doing to try to isolate the performance solely to
>> QuickFix and I'm still seeing the same results.
>>
>> Has anyone run into this before, and is there some advice as to why
>> these notifications might be taking so long to be processed?
>>
>> Thanks,
>> RL
>>
>> --------------------------------------------------------------------- 
>> ----
>> SF.Net email is sponsored by:
>> Check out the new SourceForge.net Marketplace.
>> It's the best place to buy or sell services
>> for just about anything Open Source.
>> http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/ 
>> marketplace
>> _______________________________________________
>> Quickfix-developers mailing list
>> Qui...@li...
>> https://lists.sourceforge.net/lists/listinfo/quickfix-developers
>>
>
> ---------------------------------------------------------------------- 
> ---
> SF.Net email is sponsored by:
> Check out the new SourceForge.net Marketplace.
> It's the best place to buy or sell services
> for just about anything Open Source.
> http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/ 
> marketplace
> _______________________________________________
> Quickfix-developers mailing list
> Qui...@li...
> https://lists.sourceforge.net/lists/listinfo/quickfix-developers
>

Re: [Quickfix-developers] QuickFix performance?

[Rick L. <ric...@gm...>]

Djalma,

Good question. By "isolating" QuickFix, I mean I simply have a main 
thread (the QuickFix thread) that handles all of the messages.  My first 
thought was that maybe "cracking" the messages was causing overhead, so 
I removed that -- so I basically only have a method that submits the New 
Order Single request, and of course my fromApp() that does nothing other 
than increments a message counter looks to see if the message is an 
OrdStatus.FILLED and it then displays the total time taken, as well as 
the total number of messages.

In my "full" program, I have several threads that handle notifying order 
"listeners" (if I have one program that submits order A and another 
program that submits order B, and order A gets filled, I have a thread 
that properly routes ExecutionReports to the intended recipient), I also 
have market-data threads (that is outside the scope of QuickFix), etc., 
etc., and by isolate, I only meant I had a single QuickFix thread 
processing the messages.

Before I dive any deeper into this issue, I'm going to test on a new 
network because I have a suspicion the network I'm on has some serious 
limitations (for other market-data related projects, we would get a ton 
of missed/dropped packets, etc.).

Thanks for the response -- I'll let the group know what I find.

Rick

Djalma Rosa dos Santos Filho wrote:
> QuickFIX Documentation: http://www.quickfixengine.org/quickfix/doc/html/index.html
> QuickFIX Support: http://www.quickfixengine.org/services.html
>
> Hi Rick,
>
> I don't know if this is the same, but we are also suffering with
> low-throughput message receiving when using quickfix in a C# GUI
> application in the moment when there is a remote connection sending
> thousands of messages. Because of this, we're planning to modify the
> application's architecture by adding some stuff like MVC, queues and
> threading pools to try to maximize the throughput. The current
> application design is inefficient and I believe in some reasons for
> that, things related to the way how Quickfix is working and some TCP
> socket issues...
> But, in order to know if maybe this is the same with you, let me ask
> you some questions for clarification.
> When you say that you could isolate quickfix performance, what exactly
> do you mean? Nothing in fromApp() ?
> Are you processing all messages in the context of the quickfix thread?
> How heavy is this message processing?
> Are you using the message cracker?
>
> Djalma
>
>
> On Dec 19, 2007 12:59 PM, Rick Lane <ric...@gm...> wrote:
>   
>> QuickFIX Documentation: http://www.quickfixengine.org/quickfix/doc/html/index.html
>> QuickFIX Support: http://www.quickfixengine.org/services.html
>>
>> Greetings,
>>
>> Is there any documentation regarding the performance (e.g. message
>> throughput capabilities) of QuickFix?  I'm running into issues where it
>> seems that I'm receiving messages too slowly (I emailed the list a
>> couple days ago regarding fill notifications -- namely, if I place an
>> immediate-fill order for, say, 1000 contracts, it takes nearly 2.5
>> seconds to receive all of the fill notifications).  This may have
>> something to do with the CME (perhaps on the environment on which I'm
>> testing they throttle fill notifications) but I don't believe this is
>> the case.  I've pretty much eliminated any other threading/etc. that my
>> application is doing to try to isolate the performance solely to
>> QuickFix and I'm still seeing the same results.
>>
>> Has anyone run into this before, and is there some advice as to why
>> these notifications might be taking so long to be processed?
>>
>> Thanks,
>> RL
>>
>> -------------------------------------------------------------------------
>> SF.Net email is sponsored by:
>> Check out the new SourceForge.net Marketplace.
>> It's the best place to buy or sell services
>> for just about anything Open Source.
>> http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
>> _______________________________________________
>> Quickfix-developers mailing list
>> Qui...@li...
>> https://lists.sourceforge.net/lists/listinfo/quickfix-developers
>>
>>     
>
> -------------------------------------------------------------------------
> SF.Net email is sponsored by:
> Check out the new SourceForge.net Marketplace.
> It's the best place to buy or sell services
> for just about anything Open Source.
> http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
> _______________________________________________
> Quickfix-developers mailing list
> Qui...@li...
> https://lists.sourceforge.net/lists/listinfo/quickfix-developers
>   

Re: [Quickfix-developers] QuickFix performance?

[JiteshT <ji...@ed...>]

Hi Rick,

I am also looking to optimize my QuickFIX based application for performance.
Does anyone have benchmarks for 

- How many messages per second can QuickFIX handle?
- What the latency is within QuickFIX for sending out one order message, and
how it changes with a load of 10, 50, 100 messages simultaneously?

Jitesh



Rick Lane-2 wrote:
> 
> QuickFIX Documentation:
> http://www.quickfixengine.org/quickfix/doc/html/index.html
> QuickFIX Support: http://www.quickfixengine.org/services.html
> 
> 
> Djalma,
> 
> Good question. By "isolating" QuickFix, I mean I simply have a main 
> thread (the QuickFix thread) that handles all of the messages.  My first 
> thought was that maybe "cracking" the messages was causing overhead, so 
> I removed that -- so I basically only have a method that submits the New 
> Order Single request, and of course my fromApp() that does nothing other 
> than increments a message counter looks to see if the message is an 
> OrdStatus.FILLED and it then displays the total time taken, as well as 
> the total number of messages.
> 
> In my "full" program, I have several threads that handle notifying order 
> "listeners" (if I have one program that submits order A and another 
> program that submits order B, and order A gets filled, I have a thread 
> that properly routes ExecutionReports to the intended recipient), I also 
> have market-data threads (that is outside the scope of QuickFix), etc., 
> etc., and by isolate, I only meant I had a single QuickFix thread 
> processing the messages.
> 
> Before I dive any deeper into this issue, I'm going to test on a new 
> network because I have a suspicion the network I'm on has some serious 
> limitations (for other market-data related projects, we would get a ton 
> of missed/dropped packets, etc.).
> 
> Thanks for the response -- I'll let the group know what I find.
> 
> Rick
> 
> Djalma Rosa dos Santos Filho wrote:
>> QuickFIX Documentation:
>> http://www.quickfixengine.org/quickfix/doc/html/index.html
>> QuickFIX Support: http://www.quickfixengine.org/services.html
>>
>> Hi Rick,
>>
>> I don't know if this is the same, but we are also suffering with
>> low-throughput message receiving when using quickfix in a C# GUI
>> application in the moment when there is a remote connection sending
>> thousands of messages. Because of this, we're planning to modify the
>> application's architecture by adding some stuff like MVC, queues and
>> threading pools to try to maximize the throughput. The current
>> application design is inefficient and I believe in some reasons for
>> that, things related to the way how Quickfix is working and some TCP
>> socket issues...
>> But, in order to know if maybe this is the same with you, let me ask
>> you some questions for clarification.
>> When you say that you could isolate quickfix performance, what exactly
>> do you mean? Nothing in fromApp() ?
>> Are you processing all messages in the context of the quickfix thread?
>> How heavy is this message processing?
>> Are you using the message cracker?
>>
>> Djalma
>>
>>
>> On Dec 19, 2007 12:59 PM, Rick Lane <ric...@gm...> wrote:
>>   
>>> QuickFIX Documentation:
>>> http://www.quickfixengine.org/quickfix/doc/html/index.html
>>> QuickFIX Support: http://www.quickfixengine.org/services.html
>>>
>>> Greetings,
>>>
>>> Is there any documentation regarding the performance (e.g. message
>>> throughput capabilities) of QuickFix?  I'm running into issues where it
>>> seems that I'm receiving messages too slowly (I emailed the list a
>>> couple days ago regarding fill notifications -- namely, if I place an
>>> immediate-fill order for, say, 1000 contracts, it takes nearly 2.5
>>> seconds to receive all of the fill notifications).  This may have
>>> something to do with the CME (perhaps on the environment on which I'm
>>> testing they throttle fill notifications) but I don't believe this is
>>> the case.  I've pretty much eliminated any other threading/etc. that my
>>> application is doing to try to isolate the performance solely to
>>> QuickFix and I'm still seeing the same results.
>>>
>>> Has anyone run into this before, and is there some advice as to why
>>> these notifications might be taking so long to be processed?
>>>
>>> Thanks,
>>> RL
>>>
>>> -------------------------------------------------------------------------
>>> SF.Net email is sponsored by:
>>> Check out the new SourceForge.net Marketplace.
>>> It's the best place to buy or sell services
>>> for just about anything Open Source.
>>> http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
>>> _______________________________________________
>>> Quickfix-developers mailing list
>>> Qui...@li...
>>> https://lists.sourceforge.net/lists/listinfo/quickfix-developers
>>>
>>>     
>>
>> -------------------------------------------------------------------------
>> SF.Net email is sponsored by:
>> Check out the new SourceForge.net Marketplace.
>> It's the best place to buy or sell services
>> for just about anything Open Source.
>> http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
>> _______________________________________________
>> Quickfix-developers mailing list
>> Qui...@li...
>> https://lists.sourceforge.net/lists/listinfo/quickfix-developers
>>   
> 
> -------------------------------------------------------------------------
> SF.Net email is sponsored by:
> Check out the new SourceForge.net Marketplace.
> It's the best place to buy or sell services
> for just about anything Open Source.
> http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
> _______________________________________________
> Quickfix-developers mailing list
> Qui...@li...
> https://lists.sourceforge.net/lists/listinfo/quickfix-developers
> 

-- 
View this message in context: http://www.nabble.com/QuickFix-performance--tp14418589p17348890.html
Sent from the QuickFIX - Dev mailing list archive at Nabble.com.