Re: [Quickfixj-users] Losing time between fromApp calls

[Florian L. <fl...@un...>]

Thanks a lot for the hint, it turns out the JdbcStoreFactory was the 
"culprit". With a MemoryStoreFactory instead, the performance 
skyrocketed. Now I just need to figure out a way to store the sequence 
numbers elsewhere to protect against a possible downtime...

On 10.09.2020 19:01, Colin DuPlantis wrote:
> The recommendation is the same, though, attach a profiler to see where 
> the time is being taken.
>
> On 9/10/20 10:00 AM, Florian Leu wrote:
>> Thanks for the reply. Actually the messages arrive very quickly 
>> (thousands inside one second) which I can see because the Slf4jLogger 
>> logs them immediately to my console. It's just the fromApp() method 
>> calls that seem to be delayed with this 20ms gap. And btw. it's not 
>> always exactly 20ms, it varies between 15-30ms, I just picked a nice 
>> example with consistent spacing between the method calls. So I don't 
>> really know what's happening in between since I don't know what code 
>> is running in this gap and therefore I can't add log statements there.
>>
>> On 10.09.2020 18:05, Colin DuPlantis wrote:
>>> QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
>>> QuickFIX/J Support: http://www.quickfixj.org/support/
>>>
>>>
>>> We regularly process over 1000 messages/s, so I don't think this is 
>>> some integral limitation to qfj.
>>>
>>> If it were me, I would use a profiling tool like Oracle JMC or the 
>>> like to see what's taking the most cycles.
>>>
>>> At a guess, I would be suspicious of whatever it is that's sending 
>>> the messages. It's such a regular interval that I wonder if your 
>>> source isn't sending messages fast enough.
>>>
>>> In my experience, the network side of qfj takes microseconds to 
>>> handle a message.
>>>
>>> On 9/10/20 1:33 AM, Florian Leu wrote:
>>>> QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
>>>> QuickFIX/J Support: http://www.quickfixj.org/support/
>>>>
>>>>
>>>> Hi everyone
>>>>
>>>> I'm a new user of Quickfix/j and so far the implementation is going 
>>>> well, but now that I started performance testing I realized that it 
>>>> isn't my code in the fromApp calls that is losing time, but it's 
>>>> actually that between every fromApp call I'm losing around 20ms, 
>>>> which limits my throughput to under 50msg/s. I see this by logging 
>>>> at the beginning and at the end of every fromApp call, which shows 
>>>> clearly that the time is lost between the calls and not during the 
>>>> method execution. Can anyone tell me what I might be doing wrong or 
>>>> how I can get Quickfix/j not to pause for 20ms between consecutive 
>>>> fromApp calls, but follow one fromApp call immediately after the 
>>>> last has terminated? Or is there any way to parallelize the fromApp 
>>>> calls (even though the messages all arrive via the same session)?
>>>>
>>>> 2020-09-10 09:00:33.210  INFO 12184 --- [QFJ Message Processor] 
>>>> MyApp    : startFromApp
>>>> 2020-09-10 09:00:33.210  INFO 12184 --- [QFJ Message Processor] 
>>>> MyApp    : endFromApp
>>>> 2020-09-10 09:00:33.230  INFO 12184 --- [QFJ Message Processor] 
>>>> MyApp    : startFromApp
>>>> 2020-09-10 09:00:33.230  INFO 12184 --- [QFJ Message Processor] 
>>>> MyApp    : endFromApp
>>>> 2020-09-10 09:00:33.250  INFO 12184 --- [QFJ Message Processor] 
>>>> MyApp    : startFromApp
>>>> 2020-09-10 09:00:33.250  INFO 12184 --- [QFJ Message Processor] 
>>>> MyApp    : endFromApp
>>>> 2020-09-10 09:00:33.270  INFO 12184 --- [QFJ Message Processor] 
>>>> MyApp    : startFromApp
>>>> 2020-09-10 09:00:33.270  INFO 12184 --- [QFJ Message Processor] 
>>>> MyApp    : endFromApp
>>>> 2020-09-10 09:00:33.290  INFO 12184 --- [QFJ Message Processor] 
>>>> MyApp    : startFromApp
>>>> 2020-09-10 09:00:33.290  INFO 12184 --- [QFJ Message Processor] 
>>>> MyApp    : endFromApp
>>>>
>>>> Thanks for your help and best regards,
>>>> Florian
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Quickfixj-users mailing list
>>>> Qui...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/quickfixj-users
>>>