Re: [Quickfixj-users] Losing time between fromApp calls

[Christoph J. <chr...@ma...>]

BTW, you could also put the received messages into a queue with a worker thread.
That way the processing is completely decoupled and the subsequent message can be processed by QFJ 
immediately.

Cheers,
Chris.


On 10.09.20 23:06, Florian Leu wrote:
> QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
> QuickFIX/J Support: http://www.quickfixj.org/support/
>
>
> Thanks a lot for the hint, it turns out the JdbcStoreFactory was the "culprit". With a 
> MemoryStoreFactory instead, the performance skyrocketed. Now I just need to figure out a way to 
> store the sequence numbers elsewhere to protect against a possible downtime...
>
> On 10.09.2020 19:01, Colin DuPlantis wrote:
>> The recommendation is the same, though, attach a profiler to see where the time is being taken.
>>
>> On 9/10/20 10:00 AM, Florian Leu wrote:
>>> Thanks for the reply. Actually the messages arrive very quickly (thousands inside one second) 
>>> which I can see because the Slf4jLogger logs them immediately to my console. It's just the 
>>> fromApp() method calls that seem to be delayed with this 20ms gap. And btw. it's not always 
>>> exactly 20ms, it varies between 15-30ms, I just picked a nice example with consistent spacing 
>>> between the method calls. So I don't really know what's happening in between since I don't know 
>>> what code is running in this gap and therefore I can't add log statements there.
>>>
>>> On 10.09.2020 18:05, Colin DuPlantis wrote:
>>>> QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
>>>> QuickFIX/J Support: http://www.quickfixj.org/support/
>>>>
>>>>
>>>> We regularly process over 1000 messages/s, so I don't think this is some integral limitation to 
>>>> qfj.
>>>>
>>>> If it were me, I would use a profiling tool like Oracle JMC or the like to see what's taking 
>>>> the most cycles.
>>>>
>>>> At a guess, I would be suspicious of whatever it is that's sending the messages. It's such a 
>>>> regular interval that I wonder if your source isn't sending messages fast enough.
>>>>
>>>> In my experience, the network side of qfj takes microseconds to handle a message.
>>>>
>>>> On 9/10/20 1:33 AM, Florian Leu wrote:
>>>>> QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
>>>>> QuickFIX/J Support: http://www.quickfixj.org/support/
>>>>>
>>>>>
>>>>> Hi everyone
>>>>>
>>>>> I'm a new user of Quickfix/j and so far the implementation is going well, but now that I 
>>>>> started performance testing I realized that it isn't my code in the fromApp calls that is 
>>>>> losing time, but it's actually that between every fromApp call I'm losing around 20ms, which 
>>>>> limits my throughput to under 50msg/s. I see this by logging at the beginning and at the end 
>>>>> of every fromApp call, which shows clearly that the time is lost between the calls and not 
>>>>> during the method execution. Can anyone tell me what I might be doing wrong or how I can get 
>>>>> Quickfix/j not to pause for 20ms between consecutive fromApp calls, but follow one fromApp 
>>>>> call immediately after the last has terminated? Or is there any way to parallelize the fromApp 
>>>>> calls (even though the messages all arrive via the same session)?
>>>>>
>>>>> 2020-09-10 09:00:33.210  INFO 12184 --- [QFJ Message Processor] MyApp    : startFromApp
>>>>> 2020-09-10 09:00:33.210  INFO 12184 --- [QFJ Message Processor] MyApp    : endFromApp
>>>>> 2020-09-10 09:00:33.230  INFO 12184 --- [QFJ Message Processor] MyApp    : startFromApp
>>>>> 2020-09-10 09:00:33.230  INFO 12184 --- [QFJ Message Processor] MyApp    : endFromApp
>>>>> 2020-09-10 09:00:33.250  INFO 12184 --- [QFJ Message Processor] MyApp    : startFromApp
>>>>> 2020-09-10 09:00:33.250  INFO 12184 --- [QFJ Message Processor] MyApp    : endFromApp
>>>>> 2020-09-10 09:00:33.270  INFO 12184 --- [QFJ Message Processor] MyApp    : startFromApp
>>>>> 2020-09-10 09:00:33.270  INFO 12184 --- [QFJ Message Processor] MyApp    : endFromApp
>>>>> 2020-09-10 09:00:33.290  INFO 12184 --- [QFJ Message Processor] MyApp    : startFromApp
>>>>> 2020-09-10 09:00:33.290  INFO 12184 --- [QFJ Message Processor] MyApp    : endFromApp
>>>>>
>>>>> Thanks for your help and best regards,
>>>>> Florian
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Quickfixj-users mailing list
>>>>> Qui...@li...
>>>>> https://lists.sourceforge.net/lists/listinfo/quickfixj-users
>>>>
>
>
> _______________________________________________
> Quickfixj-users mailing list
> Qui...@li...
> https://lists.sourceforge.net/lists/listinfo/quickfixj-users

-- 
Christoph John
Software Engineering
T +49 241 557080-28
chr...@ma...

MACD GmbH
Oppenhoffallee 103
52066 Aachen, Germany
www.macd.com

Amtsgericht Aachen: HRB 8151
Ust.-Id: DE 813021663
Geschäftsführer: George Macdonald