quickfixj-users — General mailing list for QuickFIX/J topics.

Re: [Quickfixj-users] Sequence reset time

[philip <ph...@wh...>]

At 9:25 a logout will be sent. Between 9:25 and 9:30 no messages will be 
sent. Then at 9:30 a logon will be sent with 34=1.
This is how FIX is expected to work. Sending sequence resets mid-session 
is not expected behaviour.

You're expected to agree the session window with your counter-party. 
This is a key part of the onboarding process.

Weekly sessions are no different - except because it's a weekly session 
the sequence number resets once a week when the session window ends.

ResetOnLogon mostly exists to deal with a scenario where you don't care 
about missed messages. Again this is agreed with the counter-party. It's 
common in sending live pricing data (rather than say order flow, where a 
missed order/allocation obviously should be re-sent).

On 2020-09-17 15:08, Gaurav Kumar wrote:
> Also I have weekly session (Sunday -Sat), will it work for daily
> sessions only?
> 
> On Thu, 17 Sep 2020 at 19:35, Gaurav Kumar <kum...@gm...>
> wrote:
> 
>> I haven't seen reset message generate between start and end time, i
>> am using quickfix version 1.6.4.
>> but let say start time is 9:30 and end time is 9:25 , are you saying
>> that reset sequence can be sent anytime between 9:25 and 9:30?
>> 
>> On Thu, 17 Sep 2020 at 19:08, philip <ph...@wh...> wrote:
>> 
>>> With a StartTime and EndTime defined, the session *will* reset
>>> between
>>> the EndTime and StartTime.
>>> 
>>> Is this behaviour not doing what you need it to?
>>> 
>>> On 2020-09-17 14:29, Gaurav Kumar wrote:
>>>> QuickFIX/J Documentation:
>>> http://www.quickfixj.org/documentation/
>>>> QuickFIX/J Support: http://www.quickfixj.org/support/
>>>> 
>>>> 
>>>> 
>>>> Hi
>>>> 
>>>> Is there any way in configuration to define the time of sequence
>>> reset
>>>> I know sequence can be rest during logon or logout but  i am
>>> looking
>>>> at the configuration like below.
>>>> 
>>>> Start time : 9:30
>>>> End time : 9:25
>>>> Sequence reset : 9:27
>>>> 
>>>> i don't want to reset the sequence at logon/session start as
>>> in
>>>> case application goes down intraday and comes up again then it
>>> will
>>>> send logon and ask for sequence reset.
>>>> 
>>>> Regards
>>>> Gaurav
>>>> 
>>>> --
>>>> Regards
>>>> Gaurav
>>>> _______________________________________________
>>>> Quickfixj-users mailing list
>>>> Qui...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/quickfixj-users
>> 
>> --
>> Regards
>> Gaurav
> 
> --
> Regards
> Gaurav

Re: [Quickfixj-users] Sequence reset time

[Gaurav K. <kum...@gm...>]

Also I have weekly session (Sunday -Sat), will it work for daily sessions
only?


On Thu, 17 Sep 2020 at 19:35, Gaurav Kumar <kum...@gm...> wrote:

> I haven't seen reset message generate between start and end time, i am
> using quickfix version 1.6.4.
> but let say start time is 9:30 and end time is 9:25 , are you saying that
> reset sequence can be sent anytime between 9:25 and 9:30?
>
> On Thu, 17 Sep 2020 at 19:08, philip <ph...@wh...> wrote:
>
>> With a StartTime and EndTime defined, the session *will* reset between
>> the EndTime and StartTime.
>>
>> Is this behaviour not doing what you need it to?
>>
>> On 2020-09-17 14:29, Gaurav Kumar wrote:
>> > QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
>> > QuickFIX/J Support: http://www.quickfixj.org/support/
>> >
>> >
>> >
>> > Hi
>> >
>> > Is there any way in configuration to define the time of sequence reset
>> > I know sequence can be rest during logon or logout but  i am looking
>> > at the configuration like below.
>> >
>> > Start time : 9:30
>> > End time : 9:25
>> > Sequence reset : 9:27
>> >
>> >    i don't want to reset the sequence at logon/session start as in
>> > case application goes down intraday and comes up again then it will
>> > send logon and ask for sequence reset.
>> >
>> > Regards
>> > Gaurav
>> >
>> > --
>> > Regards
>> > Gaurav
>> > _______________________________________________
>> > Quickfixj-users mailing list
>> > Qui...@li...
>> > https://lists.sourceforge.net/lists/listinfo/quickfixj-users
>>
>
>
> --
> Regards
> Gaurav
>


-- 
Regards
Gaurav

Re: [Quickfixj-users] Sequence reset time

[Gaurav K. <kum...@gm...>]

I haven't seen reset message generate between start and end time, i am
using quickfix version 1.6.4.
but let say start time is 9:30 and end time is 9:25 , are you saying that
reset sequence can be sent anytime between 9:25 and 9:30?

On Thu, 17 Sep 2020 at 19:08, philip <ph...@wh...> wrote:

> With a StartTime and EndTime defined, the session *will* reset between
> the EndTime and StartTime.
>
> Is this behaviour not doing what you need it to?
>
> On 2020-09-17 14:29, Gaurav Kumar wrote:
> > QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
> > QuickFIX/J Support: http://www.quickfixj.org/support/
> >
> >
> >
> > Hi
> >
> > Is there any way in configuration to define the time of sequence reset
> > I know sequence can be rest during logon or logout but  i am looking
> > at the configuration like below.
> >
> > Start time : 9:30
> > End time : 9:25
> > Sequence reset : 9:27
> >
> >    i don't want to reset the sequence at logon/session start as in
> > case application goes down intraday and comes up again then it will
> > send logon and ask for sequence reset.
> >
> > Regards
> > Gaurav
> >
> > --
> > Regards
> > Gaurav
> > _______________________________________________
> > Quickfixj-users mailing list
> > Qui...@li...
> > https://lists.sourceforge.net/lists/listinfo/quickfixj-users
>


-- 
Regards
Gaurav

Re: [Quickfixj-users] Sequence reset time

[philip <ph...@wh...>]

With a StartTime and EndTime defined, the session *will* reset between 
the EndTime and StartTime.

Is this behaviour not doing what you need it to?

On 2020-09-17 14:29, Gaurav Kumar wrote:
> QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
> QuickFIX/J Support: http://www.quickfixj.org/support/
> 
> 
> 
> Hi
> 
> Is there any way in configuration to define the time of sequence reset
> I know sequence can be rest during logon or logout but  i am looking
> at the configuration like below.
> 
> Start time : 9:30
> End time : 9:25
> Sequence reset : 9:27
> 
>    i don't want to reset the sequence at logon/session start as in
> case application goes down intraday and comes up again then it will
> send logon and ask for sequence reset.
> 
> Regards
> Gaurav
> 
> --
> Regards
> Gaurav
> _______________________________________________
> Quickfixj-users mailing list
> Qui...@li...
> https://lists.sourceforge.net/lists/listinfo/quickfixj-users

[Quickfixj-users] Sequence reset time

[Gaurav K. <kum...@gm...>]

Hi

Is there any way in configuration to define the time of sequence reset
I know sequence can be rest during logon or logout but  i am looking at the
configuration like below.

Start time : 9:30
End time : 9:25
Sequence reset : 9:27

   i don't want to reset the sequence at logon/session start as in case
application goes down intraday and comes up again then it will send logon
and ask for sequence reset.

Regards
Gaurav

-- 
Regards
Gaurav

Re: [Quickfixj-users] Losing time between fromApp calls

[Christoph J. <chr...@ma...>]

BTW, you could also put the received messages into a queue with a worker thread.
That way the processing is completely decoupled and the subsequent message can be processed by QFJ 
immediately.

Cheers,
Chris.


On 10.09.20 23:06, Florian Leu wrote:
> QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
> QuickFIX/J Support: http://www.quickfixj.org/support/
>
>
> Thanks a lot for the hint, it turns out the JdbcStoreFactory was the "culprit". With a 
> MemoryStoreFactory instead, the performance skyrocketed. Now I just need to figure out a way to 
> store the sequence numbers elsewhere to protect against a possible downtime...
>
> On 10.09.2020 19:01, Colin DuPlantis wrote:
>> The recommendation is the same, though, attach a profiler to see where the time is being taken.
>>
>> On 9/10/20 10:00 AM, Florian Leu wrote:
>>> Thanks for the reply. Actually the messages arrive very quickly (thousands inside one second) 
>>> which I can see because the Slf4jLogger logs them immediately to my console. It's just the 
>>> fromApp() method calls that seem to be delayed with this 20ms gap. And btw. it's not always 
>>> exactly 20ms, it varies between 15-30ms, I just picked a nice example with consistent spacing 
>>> between the method calls. So I don't really know what's happening in between since I don't know 
>>> what code is running in this gap and therefore I can't add log statements there.
>>>
>>> On 10.09.2020 18:05, Colin DuPlantis wrote:
>>>> QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
>>>> QuickFIX/J Support: http://www.quickfixj.org/support/
>>>>
>>>>
>>>> We regularly process over 1000 messages/s, so I don't think this is some integral limitation to 
>>>> qfj.
>>>>
>>>> If it were me, I would use a profiling tool like Oracle JMC or the like to see what's taking 
>>>> the most cycles.
>>>>
>>>> At a guess, I would be suspicious of whatever it is that's sending the messages. It's such a 
>>>> regular interval that I wonder if your source isn't sending messages fast enough.
>>>>
>>>> In my experience, the network side of qfj takes microseconds to handle a message.
>>>>
>>>> On 9/10/20 1:33 AM, Florian Leu wrote:
>>>>> QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
>>>>> QuickFIX/J Support: http://www.quickfixj.org/support/
>>>>>
>>>>>
>>>>> Hi everyone
>>>>>
>>>>> I'm a new user of Quickfix/j and so far the implementation is going well, but now that I 
>>>>> started performance testing I realized that it isn't my code in the fromApp calls that is 
>>>>> losing time, but it's actually that between every fromApp call I'm losing around 20ms, which 
>>>>> limits my throughput to under 50msg/s. I see this by logging at the beginning and at the end 
>>>>> of every fromApp call, which shows clearly that the time is lost between the calls and not 
>>>>> during the method execution. Can anyone tell me what I might be doing wrong or how I can get 
>>>>> Quickfix/j not to pause for 20ms between consecutive fromApp calls, but follow one fromApp 
>>>>> call immediately after the last has terminated? Or is there any way to parallelize the fromApp 
>>>>> calls (even though the messages all arrive via the same session)?
>>>>>
>>>>> 2020-09-10 09:00:33.210  INFO 12184 --- [QFJ Message Processor] MyApp    : startFromApp
>>>>> 2020-09-10 09:00:33.210  INFO 12184 --- [QFJ Message Processor] MyApp    : endFromApp
>>>>> 2020-09-10 09:00:33.230  INFO 12184 --- [QFJ Message Processor] MyApp    : startFromApp
>>>>> 2020-09-10 09:00:33.230  INFO 12184 --- [QFJ Message Processor] MyApp    : endFromApp
>>>>> 2020-09-10 09:00:33.250  INFO 12184 --- [QFJ Message Processor] MyApp    : startFromApp
>>>>> 2020-09-10 09:00:33.250  INFO 12184 --- [QFJ Message Processor] MyApp    : endFromApp
>>>>> 2020-09-10 09:00:33.270  INFO 12184 --- [QFJ Message Processor] MyApp    : startFromApp
>>>>> 2020-09-10 09:00:33.270  INFO 12184 --- [QFJ Message Processor] MyApp    : endFromApp
>>>>> 2020-09-10 09:00:33.290  INFO 12184 --- [QFJ Message Processor] MyApp    : startFromApp
>>>>> 2020-09-10 09:00:33.290  INFO 12184 --- [QFJ Message Processor] MyApp    : endFromApp
>>>>>
>>>>> Thanks for your help and best regards,
>>>>> Florian
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Quickfixj-users mailing list
>>>>> Qui...@li...
>>>>> https://lists.sourceforge.net/lists/listinfo/quickfixj-users
>>>>
>
>
> _______________________________________________
> Quickfixj-users mailing list
> Qui...@li...
> https://lists.sourceforge.net/lists/listinfo/quickfixj-users

-- 
Christoph John
Software Engineering
T +49 241 557080-28
chr...@ma...

MACD GmbH
Oppenhoffallee 103
52066 Aachen, Germany
www.macd.com

Amtsgericht Aachen: HRB 8151
Ust.-Id: DE 813021663
Geschäftsführer: George Macdonald

Re: [Quickfixj-users] Parsing semantics?

[Christoph J. <chr...@ma...>]

Hi Robert,

if you want to suppress as much validation as possible you should use the following settings: (see 
explanation at https://www.quickfixj.org/usermanual/2.1.0/usage/configuration.html#Validation)
ValidateIncomingMessage = N
ValidateFieldsOutOfOrder = N
ValidateFieldsHaveValues =N
ValidateUserDefinedFields = N
ValidateUnorderedGroupFields = N
AllowUnknownMsgFields = Y

Keep in mind that you still need to use the data dictionary to parse your known repeating groups 
correctly. So in any case set UseDataDictionary=Y.

The latest changes around that part were done with release 1.6.4, so about three years ago:
https://www.quickfixj.org/jira/browse/QFJ-791
https://www.quickfixj.org/jira/browse/QFJ-169

Before these changes the resulting message was truncated on the first failing tag.

But still after these changes, if you have a whole unknown repeating group then you might end up 
with the tags scattered over the message (ordered by tag) since QFJ has no way of knowing that this 
is a repeating group.
But for only one new tag it should work so that the new tag will be placed into the repeating group.

Hope that helps.
Cheers,
Chris.


On 15.09.20 18:50, Robert Nicholson wrote:
> QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
> QuickFIX/J Support: http://www.quickfixj.org/support/
>
>
> I have RejectInvalidMessage=N in order to avoid a vendor adding new tags or new enum values breaking the processing of a message on my side if it’s a tag that I don’t care about.
>
> However, it’s evident that in earlier releases of quickfixj that this doesn’t insulate you from this kind of event because it will effectively stop parsing the message as soon as it finds something it doesn’t like. Even if it’s just going to warn about it.
>
> If somebody adds a new repeating group tag in the message that my FIX dictionary knows nothing about then I’ll still process the message but it’s extremely unlikely that it will parse the content after
> the new repeating group therefore resulting in a partially parsed message being processed.
>
> Q. is this the correct understanding and do all quickfixj releases work this way?
>
> If not all releases work this way when did this behavior change such that it makes a best effort at parsing the message?
>
> What settings allow me to still be able to fully parse the message ignoring any new content that doesn’t align with my dictionary but still end up with a message equivalent to that which didn’t have the new repeating group tags?
>
> _______________________________________________
> Quickfixj-users mailing list
> Qui...@li...
> https://lists.sourceforge.net/lists/listinfo/quickfixj-users

-- 
Christoph John
Software Engineering
T +49 241 557080-28
chr...@ma...

MACD GmbH
Oppenhoffallee 103
52066 Aachen, Germany
www.macd.com

Amtsgericht Aachen: HRB 8151
Ust.-Id: DE 813021663
Geschäftsführer: George Macdonald

Re: [Quickfixj-users] Daily Session start/stop at different time

[Philip W. <ph...@wh...>]

StartTime: 9:30
EndTime: 9:28
Weekdays: Monday, Tuesday, Wednesday, Thursday, Friday

Best,

Philip Whitehouse

> On 16 Sep 2020, at 18:06, Gaurav Kumar <kum...@gm...> wrote:
> 
> QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
> QuickFIX/J Support: http://www.quickfixj.org/support/
> 
> 
> Thanks Philip
> 
> Is there any way via  configuration to start session daily at 9:30 and ends at 9:28 ( Monday to Sat) 
> 
> Session starts on Monday 9:30 and end on Tuesday 9:28
> 
> Starts again on Tuesday at 9:30 and ends at Wednesday 9:28 and so on. 
> 
> 
> 
>> On Wed, 16 Sep 2020, 22:19 Philip Whitehouse, <ph...@wh...> wrote:
>> QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
>> QuickFIX/J Support: http://www.quickfixj.org/support/
>> 
>> 
>> You will need to implement your own SessionSchedule in place of DefaultSessionSchedule if you want it to start at different times on different days as the current configuration does not support this.
>> 
>> I’m surprised that it’s a useful feature - most counter parties have predictable sessions to cover predictable trading hours.
>> 
>> Best,
>> 
>> Philip Whitehouse
>> 
>> > On 16 Sep 2020, at 14:51, Gaurav Kumar <kum...@gm...> wrote:
>> > 
>> > QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
>> > QuickFIX/J Support: http://www.quickfixj.org/support/
>> > 
>> > 
>> > Hi
>> > 
>> > I am trying to setup daily sessions such as 
>> > session starts on Monday 9:30  , ends on Tuesday 9:28
>> > Starts on Tuesday 9:30 , ends on Wednesday 9:28
>> > starts on Wed 10:10 and ends on thursday 10:08
>> > 
>> > can you please tell how to configure sessions like the above.
>> > 
>> > -- 
>> > Regards
>> > Gaurav
>> > _______________________________________________
>> > Quickfixj-users mailing list
>> > Qui...@li...
>> > https://lists.sourceforge.net/lists/listinfo/quickfixj-users
>> 
>> 
>> 
>> _______________________________________________
>> Quickfixj-users mailing list
>> Qui...@li...
>> https://lists.sourceforge.net/lists/listinfo/quickfixj-users
> _______________________________________________
> Quickfixj-users mailing list
> Qui...@li...
> https://lists.sourceforge.net/lists/listinfo/quickfixj-users

Re: [Quickfixj-users] Daily Session start/stop at different time

[Gaurav K. <kum...@gm...>]

Thanks Philip

Is there any way via  configuration to start session daily at 9:30 and ends
at 9:28 ( Monday to Sat)

Session starts on Monday 9:30 and end on Tuesday 9:28

Starts again on Tuesday at 9:30 and ends at Wednesday 9:28 and so on.



On Wed, 16 Sep 2020, 22:19 Philip Whitehouse, <ph...@wh...> wrote:

> QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
> QuickFIX/J <http://www.quickfixj.org/documentation/QuickFIX/J> Support:
> http://www.quickfixj.org/support/
>
>
> You will need to implement your own SessionSchedule in place of
> DefaultSessionSchedule if you want it to start at different times on
> different days as the current configuration does not support this.
>
> I’m surprised that it’s a useful feature - most counter parties have
> predictable sessions to cover predictable trading hours.
>
> Best,
>
> Philip Whitehouse
>
> > On 16 Sep 2020, at 14:51, Gaurav Kumar <kum...@gm...> wrote:
> >
> > QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
> > QuickFIX/J Support: http://www.quickfixj.org/support/
> >
> >
> > Hi
> >
> > I am trying to setup daily sessions such as
> > session starts on Monday 9:30  , ends on Tuesday 9:28
> > Starts on Tuesday 9:30 , ends on Wednesday 9:28
> > starts on Wed 10:10 and ends on thursday 10:08
> >
> > can you please tell how to configure sessions like the above.
> >
> > --
> > Regards
> > Gaurav
> > _______________________________________________
> > Quickfixj-users mailing list
> > Qui...@li...
> > https://lists.sourceforge.net/lists/listinfo/quickfixj-users
>
>
>
> _______________________________________________
> Quickfixj-users mailing list
> Qui...@li...
> https://lists.sourceforge.net/lists/listinfo/quickfixj-users
>

Re: [Quickfixj-users] Daily Session start/stop at different time

[Philip W. <ph...@wh...>]

You will need to implement your own SessionSchedule in place of DefaultSessionSchedule if you want it to start at different times on different days as the current configuration does not support this.

I’m surprised that it’s a useful feature - most counter parties have predictable sessions to cover predictable trading hours.

Best,

Philip Whitehouse

> On 16 Sep 2020, at 14:51, Gaurav Kumar <kum...@gm...> wrote:
> 
> QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
> QuickFIX/J Support: http://www.quickfixj.org/support/
> 
> 
> Hi
> 
> I am trying to setup daily sessions such as 
> session starts on Monday 9:30  , ends on Tuesday 9:28
> Starts on Tuesday 9:30 , ends on Wednesday 9:28
> starts on Wed 10:10 and ends on thursday 10:08
> 
> can you please tell how to configure sessions like the above.
> 
> -- 
> Regards
> Gaurav
> _______________________________________________
> Quickfixj-users mailing list
> Qui...@li...
> https://lists.sourceforge.net/lists/listinfo/quickfixj-users

[Quickfixj-users] Daily Session start/stop at different time

[Gaurav K. <kum...@gm...>]

Hi

I am trying to setup daily sessions such as
session starts on Monday 9:30  , ends on Tuesday 9:28
Starts on Tuesday 9:30 , ends on Wednesday 9:28
starts on Wed 10:10 and ends on thursday 10:08

can you please tell how to configure sessions like the above.

-- 
Regards
Gaurav

Re: [Quickfixj-users] Parsing semantics?

[Grant B. <gbi...@co...>]

Are you sure this vendor doesn't have some kind of mail list or news alert
that will let you know in advance when they make changes to their FIX
interface?

Adding a group is kind of a big deal.  I'd expect such a change to trip up
any QuickFIX engine, no matter what port it is (QF/C++, QF/j, QF/n).

-Grant


On Tue, Sep 15, 2020 at 6:08 PM Robert Nicholson <rob...@gm...>
wrote:

> QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
> QuickFIX/J <http://www.quickfixj.org/documentation/QuickFIX/J> Support:
> http://www.quickfixj.org/support/
>
>
> BTW: In this instance an entire new group was added not just a new tag on
> an existing group.
>
> On Sep 15, 2020, at 12:57 PM, Grant Birchmeier <gbi...@co...>
> wrote:
>
> QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
> QuickFIX/J Support: http://www.quickfixj.org/support/
>
>
> If somebody adds a new repeating group tag without telling you, then it is
> impossible to parse the message.
>
> Remember, repeating groups have no end-delimiter tag.  When the parser
> encounters a tag that's not known to be in the group, then it can only
> assume that the group is over.  Every tag after that is assumed to be out
> of the group, and then things start to go haywire.
>
> Reputable counterparties should be giving you plenty of lead-time notice
> on when they change their DD.  Is it possible that there is a mailing list
> that you need to join?
>
> -Grant
>
> On Tue, Sep 15, 2020 at 11:52 AM Robert Nicholson <
> rob...@gm...> wrote:
>
>> QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
>> QuickFIX/J <http://www.quickfixj.org/documentation/QuickFIX/J> Support:
>> http://www.quickfixj.org/support/
>>
>>
>> I have RejectInvalidMessage=N in order to avoid a vendor adding new tags
>> or new enum values breaking the processing of a message on my side if it’s
>> a tag that I don’t care about.
>>
>> However, it’s evident that in earlier releases of quickfixj that this
>> doesn’t insulate you from this kind of event because it will effectively
>> stop parsing the message as soon as it finds something it doesn’t like.
>> Even if it’s just going to warn about it.
>>
>> If somebody adds a new repeating group tag in the message that my FIX
>> dictionary knows nothing about then I’ll still process the message but it’s
>> extremely unlikely that it will parse the content after
>> the new repeating group therefore resulting in a partially parsed message
>> being processed.
>>
>> Q. is this the correct understanding and do all quickfixj releases work
>> this way?
>>
>> If not all releases work this way when did this behavior change such that
>> it makes a best effort at parsing the message?
>>
>> What settings allow me to still be able to fully parse the message
>> ignoring any new content that doesn’t align with my dictionary but still
>> end up with a message equivalent to that which didn’t have the new
>> repeating group tags?
>>
>> _______________________________________________
>> Quickfixj-users mailing list
>> Qui...@li...
>> https://lists.sourceforge.net/lists/listinfo/quickfixj-users
>>
>
>
-- 
Grant Birchmeier
*Connamara Systems, LLC*
*Made-To-Measure Trading Solutions.*
Exactly what you need. No more. No less.
http://connamara.com

-- 
This email, along with any attachments, is confidential. If you believe you 
received this message in error, please contact the sender immediately and 
delete all copies of the message. Thank you from Connamara Systems, LLC.

Re: [Quickfixj-users] Parsing semantics?

[Robert N. <rob...@gm...>]

BTW: In this instance an entire new group was added not just a new tag on an existing group. 

> On Sep 15, 2020, at 12:57 PM, Grant Birchmeier <gbi...@co...> wrote:
> 
> QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
> QuickFIX/J Support: http://www.quickfixj.org/support/
> 
> 
> If somebody adds a new repeating group tag without telling you, then it is impossible to parse the message.
> 
> Remember, repeating groups have no end-delimiter tag.  When the parser encounters a tag that's not known to be in the group, then it can only assume that the group is over.  Every tag after that is assumed to be out of the group, and then things start to go haywire.
> 
> Reputable counterparties should be giving you plenty of lead-time notice on when they change their DD.  Is it possible that there is a mailing list that you need to join?
> 
> -Grant
> 
> On Tue, Sep 15, 2020 at 11:52 AM Robert Nicholson <rob...@gm... <mailto:rob...@gm...>> wrote:
> QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
> QuickFIX/J <http://www.quickfixj.org/documentation/QuickFIX/J> Support: http://www.quickfixj.org/support/ <http://www.quickfixj.org/support/>
> 
> 
> I have RejectInvalidMessage=N in order to avoid a vendor adding new tags or new enum values breaking the processing of a message on my side if it’s a tag that I don’t care about.
> 
> However, it’s evident that in earlier releases of quickfixj that this doesn’t insulate you from this kind of event because it will effectively stop parsing the message as soon as it finds something it doesn’t like. Even if it’s just going to warn about it.
> 
> If somebody adds a new repeating group tag in the message that my FIX dictionary knows nothing about then I’ll still process the message but it’s extremely unlikely that it will parse the content after
> the new repeating group therefore resulting in a partially parsed message being processed.
> 
> Q. is this the correct understanding and do all quickfixj releases work this way?
> 
> If not all releases work this way when did this behavior change such that it makes a best effort at parsing the message?
> 
> What settings allow me to still be able to fully parse the message ignoring any new content that doesn’t align with my dictionary but still end up with a message equivalent to that which didn’t have the new repeating group tags?
> 
> _______________________________________________
> Quickfixj-users mailing list
> Qui...@li... <mailto:Qui...@li...>
> https://lists.sourceforge.net/lists/listinfo/quickfixj-users <https://lists.sourceforge.net/lists/listinfo/quickfixj-users>
> 
> 
> -- 
> Grant Birchmeier
> Connamara Systems, LLC
> Made-To-Measure Trading Solutions.
> Exactly what you need. No more. No less.
> http://connamara.com <http://connamara.com/>
> 
> This email, along with any attachments, is confidential. If you believe you received this message in error, please contact the sender immediately and delete all copies of the message. Thank you from Connamara Systems, LLC._______________________________________________
> Quickfixj-users mailing list
> Qui...@li...
> https://lists.sourceforge.net/lists/listinfo/quickfixj-users

Re: [Quickfixj-users] Parsing semantics?

[Robert N. <rob...@gm...>]

This was a vendor who occasionally rolls in standard functionality (ie. additional tags/groups) into what we've customized.

Is it fair to say that no version of quickfixj will perform any better in this regard then?

> On Sep 15, 2020, at 12:57 PM, Grant Birchmeier <gbi...@co...> wrote:
> 
> QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
> QuickFIX/J Support: http://www.quickfixj.org/support/
> 
> 
> If somebody adds a new repeating group tag without telling you, then it is impossible to parse the message.
> 
> Remember, repeating groups have no end-delimiter tag.  When the parser encounters a tag that's not known to be in the group, then it can only assume that the group is over.  Every tag after that is assumed to be out of the group, and then things start to go haywire.
> 
> Reputable counterparties should be giving you plenty of lead-time notice on when they change their DD.  Is it possible that there is a mailing list that you need to join?
> 
> -Grant
> 
> On Tue, Sep 15, 2020 at 11:52 AM Robert Nicholson <rob...@gm... <mailto:rob...@gm...>> wrote:
> QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
> QuickFIX/J <http://www.quickfixj.org/documentation/QuickFIX/J> Support: http://www.quickfixj.org/support/ <http://www.quickfixj.org/support/>
> 
> 
> I have RejectInvalidMessage=N in order to avoid a vendor adding new tags or new enum values breaking the processing of a message on my side if it’s a tag that I don’t care about.
> 
> However, it’s evident that in earlier releases of quickfixj that this doesn’t insulate you from this kind of event because it will effectively stop parsing the message as soon as it finds something it doesn’t like. Even if it’s just going to warn about it.
> 
> If somebody adds a new repeating group tag in the message that my FIX dictionary knows nothing about then I’ll still process the message but it’s extremely unlikely that it will parse the content after
> the new repeating group therefore resulting in a partially parsed message being processed.
> 
> Q. is this the correct understanding and do all quickfixj releases work this way?
> 
> If not all releases work this way when did this behavior change such that it makes a best effort at parsing the message?
> 
> What settings allow me to still be able to fully parse the message ignoring any new content that doesn’t align with my dictionary but still end up with a message equivalent to that which didn’t have the new repeating group tags?
> 
> _______________________________________________
> Quickfixj-users mailing list
> Qui...@li... <mailto:Qui...@li...>
> https://lists.sourceforge.net/lists/listinfo/quickfixj-users <https://lists.sourceforge.net/lists/listinfo/quickfixj-users>
> 
> 
> -- 
> Grant Birchmeier
> Connamara Systems, LLC
> Made-To-Measure Trading Solutions.
> Exactly what you need. No more. No less.
> http://connamara.com <http://connamara.com/>
> 
> This email, along with any attachments, is confidential. If you believe you received this message in error, please contact the sender immediately and delete all copies of the message. Thank you from Connamara Systems, LLC._______________________________________________
> Quickfixj-users mailing list
> Qui...@li...
> https://lists.sourceforge.net/lists/listinfo/quickfixj-users

Re: [Quickfixj-users] Parsing semantics?

[Grant B. <gbi...@co...>]

If somebody adds a new repeating group tag without telling you, then it is
impossible to parse the message.

Remember, repeating groups have no end-delimiter tag.  When the parser
encounters a tag that's not known to be in the group, then it can only
assume that the group is over.  Every tag after that is assumed to be out
of the group, and then things start to go haywire.

Reputable counterparties should be giving you plenty of lead-time notice on
when they change their DD.  Is it possible that there is a mailing list
that you need to join?

-Grant

On Tue, Sep 15, 2020 at 11:52 AM Robert Nicholson <
rob...@gm...> wrote:

> QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
> QuickFIX/J <http://www.quickfixj.org/documentation/QuickFIX/J> Support:
> http://www.quickfixj.org/support/
>
>
> I have RejectInvalidMessage=N in order to avoid a vendor adding new tags
> or new enum values breaking the processing of a message on my side if it’s
> a tag that I don’t care about.
>
> However, it’s evident that in earlier releases of quickfixj that this
> doesn’t insulate you from this kind of event because it will effectively
> stop parsing the message as soon as it finds something it doesn’t like.
> Even if it’s just going to warn about it.
>
> If somebody adds a new repeating group tag in the message that my FIX
> dictionary knows nothing about then I’ll still process the message but it’s
> extremely unlikely that it will parse the content after
> the new repeating group therefore resulting in a partially parsed message
> being processed.
>
> Q. is this the correct understanding and do all quickfixj releases work
> this way?
>
> If not all releases work this way when did this behavior change such that
> it makes a best effort at parsing the message?
>
> What settings allow me to still be able to fully parse the message
> ignoring any new content that doesn’t align with my dictionary but still
> end up with a message equivalent to that which didn’t have the new
> repeating group tags?
>
> _______________________________________________
> Quickfixj-users mailing list
> Qui...@li...
> https://lists.sourceforge.net/lists/listinfo/quickfixj-users
>


-- 
Grant Birchmeier
*Connamara Systems, LLC*
*Made-To-Measure Trading Solutions.*
Exactly what you need. No more. No less.
http://connamara.com

-- 
This email, along with any attachments, is confidential. If you believe you 
received this message in error, please contact the sender immediately and 
delete all copies of the message. Thank you from Connamara Systems, LLC.

[Quickfixj-users] Parsing semantics?

[Robert N. <rob...@gm...>]

I have RejectInvalidMessage=N in order to avoid a vendor adding new tags or new enum values breaking the processing of a message on my side if it’s a tag that I don’t care about.

However, it’s evident that in earlier releases of quickfixj that this doesn’t insulate you from this kind of event because it will effectively stop parsing the message as soon as it finds something it doesn’t like. Even if it’s just going to warn about it.

If somebody adds a new repeating group tag in the message that my FIX dictionary knows nothing about then I’ll still process the message but it’s extremely unlikely that it will parse the content after
the new repeating group therefore resulting in a partially parsed message being processed.

Q. is this the correct understanding and do all quickfixj releases work this way?

If not all releases work this way when did this behavior change such that it makes a best effort at parsing the message?

What settings allow me to still be able to fully parse the message ignoring any new content that doesn’t align with my dictionary but still end up with a message equivalent to that which didn’t have the new repeating group tags?

Re: [Quickfixj-users] Losing time between fromApp calls

[Florian L. <fl...@un...>]

Thanks a lot for the hint, it turns out the JdbcStoreFactory was the 
"culprit". With a MemoryStoreFactory instead, the performance 
skyrocketed. Now I just need to figure out a way to store the sequence 
numbers elsewhere to protect against a possible downtime...

On 10.09.2020 19:01, Colin DuPlantis wrote:
> The recommendation is the same, though, attach a profiler to see where 
> the time is being taken.
>
> On 9/10/20 10:00 AM, Florian Leu wrote:
>> Thanks for the reply. Actually the messages arrive very quickly 
>> (thousands inside one second) which I can see because the Slf4jLogger 
>> logs them immediately to my console. It's just the fromApp() method 
>> calls that seem to be delayed with this 20ms gap. And btw. it's not 
>> always exactly 20ms, it varies between 15-30ms, I just picked a nice 
>> example with consistent spacing between the method calls. So I don't 
>> really know what's happening in between since I don't know what code 
>> is running in this gap and therefore I can't add log statements there.
>>
>> On 10.09.2020 18:05, Colin DuPlantis wrote:
>>> QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
>>> QuickFIX/J Support: http://www.quickfixj.org/support/
>>>
>>>
>>> We regularly process over 1000 messages/s, so I don't think this is 
>>> some integral limitation to qfj.
>>>
>>> If it were me, I would use a profiling tool like Oracle JMC or the 
>>> like to see what's taking the most cycles.
>>>
>>> At a guess, I would be suspicious of whatever it is that's sending 
>>> the messages. It's such a regular interval that I wonder if your 
>>> source isn't sending messages fast enough.
>>>
>>> In my experience, the network side of qfj takes microseconds to 
>>> handle a message.
>>>
>>> On 9/10/20 1:33 AM, Florian Leu wrote:
>>>> QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
>>>> QuickFIX/J Support: http://www.quickfixj.org/support/
>>>>
>>>>
>>>> Hi everyone
>>>>
>>>> I'm a new user of Quickfix/j and so far the implementation is going 
>>>> well, but now that I started performance testing I realized that it 
>>>> isn't my code in the fromApp calls that is losing time, but it's 
>>>> actually that between every fromApp call I'm losing around 20ms, 
>>>> which limits my throughput to under 50msg/s. I see this by logging 
>>>> at the beginning and at the end of every fromApp call, which shows 
>>>> clearly that the time is lost between the calls and not during the 
>>>> method execution. Can anyone tell me what I might be doing wrong or 
>>>> how I can get Quickfix/j not to pause for 20ms between consecutive 
>>>> fromApp calls, but follow one fromApp call immediately after the 
>>>> last has terminated? Or is there any way to parallelize the fromApp 
>>>> calls (even though the messages all arrive via the same session)?
>>>>
>>>> 2020-09-10 09:00:33.210  INFO 12184 --- [QFJ Message Processor] 
>>>> MyApp    : startFromApp
>>>> 2020-09-10 09:00:33.210  INFO 12184 --- [QFJ Message Processor] 
>>>> MyApp    : endFromApp
>>>> 2020-09-10 09:00:33.230  INFO 12184 --- [QFJ Message Processor] 
>>>> MyApp    : startFromApp
>>>> 2020-09-10 09:00:33.230  INFO 12184 --- [QFJ Message Processor] 
>>>> MyApp    : endFromApp
>>>> 2020-09-10 09:00:33.250  INFO 12184 --- [QFJ Message Processor] 
>>>> MyApp    : startFromApp
>>>> 2020-09-10 09:00:33.250  INFO 12184 --- [QFJ Message Processor] 
>>>> MyApp    : endFromApp
>>>> 2020-09-10 09:00:33.270  INFO 12184 --- [QFJ Message Processor] 
>>>> MyApp    : startFromApp
>>>> 2020-09-10 09:00:33.270  INFO 12184 --- [QFJ Message Processor] 
>>>> MyApp    : endFromApp
>>>> 2020-09-10 09:00:33.290  INFO 12184 --- [QFJ Message Processor] 
>>>> MyApp    : startFromApp
>>>> 2020-09-10 09:00:33.290  INFO 12184 --- [QFJ Message Processor] 
>>>> MyApp    : endFromApp
>>>>
>>>> Thanks for your help and best regards,
>>>> Florian
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Quickfixj-users mailing list
>>>> Qui...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/quickfixj-users
>>>

Re: [Quickfixj-users] Losing time between fromApp calls

[Florian L. <fl...@un...>]

Thanks for the reply. Actually the messages arrive very quickly 
(thousands inside one second) which I can see because the Slf4jLogger 
logs them immediately to my console. It's just the fromApp() method 
calls that seem to be delayed with this 20ms gap. And btw. it's not 
always exactly 20ms, it varies between 15-30ms, I just picked a nice 
example with consistent spacing between the method calls. So I don't 
really know what's happening in between since I don't know what code is 
running in this gap and therefore I can't add log statements there.

On 10.09.2020 18:05, Colin DuPlantis wrote:
> QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
> QuickFIX/J Support: http://www.quickfixj.org/support/
>
>
> We regularly process over 1000 messages/s, so I don't think this is 
> some integral limitation to qfj.
>
> If it were me, I would use a profiling tool like Oracle JMC or the 
> like to see what's taking the most cycles.
>
> At a guess, I would be suspicious of whatever it is that's sending the 
> messages. It's such a regular interval that I wonder if your source 
> isn't sending messages fast enough.
>
> In my experience, the network side of qfj takes microseconds to handle 
> a message.
>
> On 9/10/20 1:33 AM, Florian Leu wrote:
>> QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
>> QuickFIX/J Support: http://www.quickfixj.org/support/
>>
>>
>> Hi everyone
>>
>> I'm a new user of Quickfix/j and so far the implementation is going 
>> well, but now that I started performance testing I realized that it 
>> isn't my code in the fromApp calls that is losing time, but it's 
>> actually that between every fromApp call I'm losing around 20ms, 
>> which limits my throughput to under 50msg/s. I see this by logging at 
>> the beginning and at the end of every fromApp call, which shows 
>> clearly that the time is lost between the calls and not during the 
>> method execution. Can anyone tell me what I might be doing wrong or 
>> how I can get Quickfix/j not to pause for 20ms between consecutive 
>> fromApp calls, but follow one fromApp call immediately after the last 
>> has terminated? Or is there any way to parallelize the fromApp calls 
>> (even though the messages all arrive via the same session)?
>>
>> 2020-09-10 09:00:33.210  INFO 12184 --- [QFJ Message Processor] 
>> MyApp    : startFromApp
>> 2020-09-10 09:00:33.210  INFO 12184 --- [QFJ Message Processor] 
>> MyApp    : endFromApp
>> 2020-09-10 09:00:33.230  INFO 12184 --- [QFJ Message Processor] 
>> MyApp    : startFromApp
>> 2020-09-10 09:00:33.230  INFO 12184 --- [QFJ Message Processor] 
>> MyApp    : endFromApp
>> 2020-09-10 09:00:33.250  INFO 12184 --- [QFJ Message Processor] 
>> MyApp    : startFromApp
>> 2020-09-10 09:00:33.250  INFO 12184 --- [QFJ Message Processor] 
>> MyApp    : endFromApp
>> 2020-09-10 09:00:33.270  INFO 12184 --- [QFJ Message Processor] 
>> MyApp    : startFromApp
>> 2020-09-10 09:00:33.270  INFO 12184 --- [QFJ Message Processor] 
>> MyApp    : endFromApp
>> 2020-09-10 09:00:33.290  INFO 12184 --- [QFJ Message Processor] 
>> MyApp    : startFromApp
>> 2020-09-10 09:00:33.290  INFO 12184 --- [QFJ Message Processor] 
>> MyApp    : endFromApp
>>
>> Thanks for your help and best regards,
>> Florian
>>
>>
>>
>> _______________________________________________
>> Quickfixj-users mailing list
>> Qui...@li...
>> https://lists.sourceforge.net/lists/listinfo/quickfixj-users
>

Re: [Quickfixj-users] Losing time between fromApp calls

[Colin D. <co...@ma...>]

We regularly process over 1000 messages/s, so I don't think this is some 
integral limitation to qfj.

If it were me, I would use a profiling tool like Oracle JMC or the like 
to see what's taking the most cycles.

At a guess, I would be suspicious of whatever it is that's sending the 
messages. It's such a regular interval that I wonder if your source 
isn't sending messages fast enough.

In my experience, the network side of qfj takes microseconds to handle a 
message.

On 9/10/20 1:33 AM, Florian Leu wrote:
> QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
> QuickFIX/J Support: http://www.quickfixj.org/support/
>
>
> Hi everyone
>
> I'm a new user of Quickfix/j and so far the implementation is going 
> well, but now that I started performance testing I realized that it 
> isn't my code in the fromApp calls that is losing time, but it's 
> actually that between every fromApp call I'm losing around 20ms, which 
> limits my throughput to under 50msg/s. I see this by logging at the 
> beginning and at the end of every fromApp call, which shows clearly 
> that the time is lost between the calls and not during the method 
> execution. Can anyone tell me what I might be doing wrong or how I can 
> get Quickfix/j not to pause for 20ms between consecutive fromApp 
> calls, but follow one fromApp call immediately after the last has 
> terminated? Or is there any way to parallelize the fromApp calls (even 
> though the messages all arrive via the same session)?
>
> 2020-09-10 09:00:33.210  INFO 12184 --- [QFJ Message Processor] 
> MyApp    : startFromApp
> 2020-09-10 09:00:33.210  INFO 12184 --- [QFJ Message Processor] 
> MyApp    : endFromApp
> 2020-09-10 09:00:33.230  INFO 12184 --- [QFJ Message Processor] 
> MyApp    : startFromApp
> 2020-09-10 09:00:33.230  INFO 12184 --- [QFJ Message Processor] 
> MyApp    : endFromApp
> 2020-09-10 09:00:33.250  INFO 12184 --- [QFJ Message Processor] 
> MyApp    : startFromApp
> 2020-09-10 09:00:33.250  INFO 12184 --- [QFJ Message Processor] 
> MyApp    : endFromApp
> 2020-09-10 09:00:33.270  INFO 12184 --- [QFJ Message Processor] 
> MyApp    : startFromApp
> 2020-09-10 09:00:33.270  INFO 12184 --- [QFJ Message Processor] 
> MyApp    : endFromApp
> 2020-09-10 09:00:33.290  INFO 12184 --- [QFJ Message Processor] 
> MyApp    : startFromApp
> 2020-09-10 09:00:33.290  INFO 12184 --- [QFJ Message Processor] 
> MyApp    : endFromApp
>
> Thanks for your help and best regards,
> Florian
>
>
>
> _______________________________________________
> Quickfixj-users mailing list
> Qui...@li...
> https://lists.sourceforge.net/lists/listinfo/quickfixj-users

-- 
Colin DuPlantis
Chief Architect, Marketcetera
Download, Run, Trade
888.868.4884
https://www.marketcetera.com

[Quickfixj-users] Losing time between fromApp calls

[Florian L. <fl...@un...>]

Hi everyone

I'm a new user of Quickfix/j and so far the implementation is going 
well, but now that I started performance testing I realized that it 
isn't my code in the fromApp calls that is losing time, but it's 
actually that between every fromApp call I'm losing around 20ms, which 
limits my throughput to under 50msg/s. I see this by logging at the 
beginning and at the end of every fromApp call, which shows clearly that 
the time is lost between the calls and not during the method execution. 
Can anyone tell me what I might be doing wrong or how I can get 
Quickfix/j not to pause for 20ms between consecutive fromApp calls, but 
follow one fromApp call immediately after the last has terminated? Or is 
there any way to parallelize the fromApp calls (even though the messages 
all arrive via the same session)?

2020-09-10 09:00:33.210  INFO 12184 --- [QFJ Message Processor] MyApp    
: startFromApp
2020-09-10 09:00:33.210  INFO 12184 --- [QFJ Message Processor] MyApp    
: endFromApp
2020-09-10 09:00:33.230  INFO 12184 --- [QFJ Message Processor] MyApp    
: startFromApp
2020-09-10 09:00:33.230  INFO 12184 --- [QFJ Message Processor] MyApp    
: endFromApp
2020-09-10 09:00:33.250  INFO 12184 --- [QFJ Message Processor] MyApp    
: startFromApp
2020-09-10 09:00:33.250  INFO 12184 --- [QFJ Message Processor] MyApp    
: endFromApp
2020-09-10 09:00:33.270  INFO 12184 --- [QFJ Message Processor] MyApp    
: startFromApp
2020-09-10 09:00:33.270  INFO 12184 --- [QFJ Message Processor] MyApp    
: endFromApp
2020-09-10 09:00:33.290  INFO 12184 --- [QFJ Message Processor] MyApp    
: startFromApp
2020-09-10 09:00:33.290  INFO 12184 --- [QFJ Message Processor] MyApp    
: endFromApp

Thanks for your help and best regards,
Florian

Re: [Quickfixj-users] Multiple logons/connections for this session are not allowed

[Christoph J. <chr...@ma...>]

Hi

1. Do you only have Acceptors running or also Initiators?
2. The output of either netstat (see my last mail) or a stack dump at the time of the problem would be very helpful but I guess it is hard to know when the problem appears. 

Cheers 
Chris.

Am 27. August 2020 14:24:35 MESZ schrieb Vipin Chaudhary <vip...@gm...>:
>Hi,
>
>I have some detailed updates on this.
>
>When this issue occur - During the first client LOGON the logon get
>hanged
>(don't no why)
>Following is the state in event log
>Accepting session FIX.4.4: *****Server->****client from /clientIP:
>60868
>
>    Acceptor heartbeat set to 60 seconds
>
>After this the logon process get hanged somewhere and no LOGON message
>is
>sent to client.
>
>When client does not get his LOGON msg response than it disconnect and
>retry.
>=> This disconnect is not detected at our side and when new LOGON
>message
>comes than at *AcceptorIoHandler.java *line no 69
>*qfSession.hasResponder() => return true*
>*But * when i checked *qfSession.getRemoteAddress() *than it return me*
>null. *Also I tried qfSession.logout() which does not do anything.
>
>In case of normal LOGON floe we have following state of event log
>Accepting session FIX.4.4: *****Server->****client from /clientIP:
>60868
> Acceptor heartbeat set to 60 seconds
> Logon contains ResetSeqNumFlag=Y,resetting sequence numbers to 1
> Received logon
> Responding to Logon request
>
>and then I can see the logon message in our outgoing log.
>
>I am not able to replicate this issue on my local machine so I don't
>know
>where it is getting hanged during first logon :(
>
>Do anyone has any solution to this ?
>
>
>Thanks
>Vipin
>
>On Thu, Aug 6, 2020 at 9:41 PM Christoph John <chr...@ma...>
>wrote:
>
>> Hi,
>>
>> when all clients are affected at the same time it could still be a
>network
>> issue, right? :)
>> As said, QFJ does not handle the TCP connection stuff by itself but
>uses
>> MINA which is a stable and mature library. Of course, it still could
>have
>> bugs. But your description rather sounds like all client's connection
>get
>> closed and go into TIME_WAIT or CLOSE_WAIT state.
>>
>>
>https://superuser.com/questions/173535/what-are-close-wait-and-time-wait-states
>>
>> Did you check "netstat" or similar tool when the connection problem
>occurs?
>>
>> If you were to implement the logic that you proposed that would mean
>that
>> someone could do a simple attack against your server which would not
>only
>> end the new malicious session but only the one that is rightfully
>> connected. Are you sure you want to follow that way? ;)
>>
>> Cheers,
>> Chris.
>>
>> On 06.08.20 13:27, Vipin Chaudhary wrote:
>>
>> Hi guys.
>>
>> I am still facing this issue and now the frequency is very high. I am
>also
>> not able to detect any networking issue.
>>
>> One more point is, it is happening with all clients at same time. 
>This
>> does not resolve without the restart of application.
>> One more strange thing is if I don't restart, then the client which
>was
>> disconnected at the time of this issue, when try to connect at their
>> session time also facing this issue.
>>
>> So now I am highly suspecting this as quickfix bug.
>>
>> As per me quickfixj should close old session if it detect that old
>session
>> is still running when it receive new logon. To achieve the same I am
>> thinking to edit *AcceptorIoHandler.java.*
>>
>> Class :  AcceptorIoHandler.java
>> Line no 69
>> if (qfSession.hasResponder()) {
>>              // Session is already bound to another connection
>>              sessionLog.onErrorEvent("Multiple logons/connections for
>this
>> session are not allowed");
>>              protocolSession.closeNow();
>>              *//TODO Close old session here*
>>              return;
>>   }
>>
>> There are many methods to close the session in Session class as
>following
>>
>>    1.    qfSession.close();
>>    2.    qfSession.disconnect("Closing Old session", true);
>>    3.    qfSession.logout("Closing old session to accommodate new
>>    session");
>>    4.    qfSession.generateLogout();
>>    5.    qfSession.reset();
>>
>> Which of the above method I should opt for, that can serve my purpose
>?
>>
>> Thanks
>> Vipin
>>
>>
>> On Mon, May 4, 2020 at 3:47 PM Christoph John
><chr...@ma...>
>> wrote:
>>
>>> Hi,
>>>
>>> if VPN rekeying is not the problem then maybe differing MTU sizes or
>>> asymmetric routing. Or of course it could also be another problem.
>Just
>>> said what the problems mostly were in our case.
>>> But I think if you describe the problem to your and your
>counterparty's
>>> network team (since you only seem to have this problem with some of
>your
>>> sessions) they should be able to debug it. Maybe your team need to
>do some
>>> tcp dumps around the time of the problem.
>>>
>>> It would be nice if you could reply to the user group (not me
>privately)
>>> if you found something out. This could help other users as well.
>>>
>>> Cheers,
>>> Chris.
>>>
>>>
>>> On 04.05.20 08:28, Vipin Chaudhary wrote:
>>>
>>> Hi Christoph,
>>>
>>> I double checked from the network team and find that no initiator
>use VPN
>>> to connect to us.
>>>
>>> In that case, Can you help me on what should I request to network
>team to
>>> fix the network connections?
>>>
>>> Thanks
>>> Vipin Chaudhary
>>>
>>> On Thu, Apr 30, 2020 at 3:49 PM Christoph John
><chr...@ma...>
>>> wrote:
>>>
>>>> The only solution is to fix the network connection. Everything else
>is
>>>> only a workaround.
>>>> You could try to increase socket timeouts on both sides of the
>>>> connection. Maybe it helps (depends on the cause of the problem)
>but as
>>>> said this will only work around the problem.
>>>>
>>>> Cheers,
>>>> Chris.
>>>>
>>>> On 30.04.20 12:11, Vipin Chaudhary wrote:
>>>>
>>>> Hi Christoph,
>>>>
>>>> Thanks for your input,
>>>>
>>>> We have multiple sessions and this problems does not happens with
>all
>>>> sessions simultaneously. Mostly we see this problem with one
>session and
>>>> rarely with other session.
>>>>
>>>> As far as I know initiator connect directly with us over internet
>(have
>>>> ssl). Will double check on this with network team.
>>>>
>>>> Meanwhile any solution you think of this problem ?
>>>>
>>>> Thanks
>>>> Vipin Chaudhary
>>>>
>>>> On Thu, Apr 30, 2020 at 3:31 PM Christoph John
><chr...@ma...>
>>>> wrote:
>>>>
>>>>> Addition: if VPN rekeying is the problem you will probably see
>this
>>>>> message every hour (or whatever the rekey interval is)
>>>>>
>>>>> Chris.
>>>>>
>>>>> On 30.04.20 12:00, Christoph John wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> did you change QFJ version or why do you think it is QFJ related?
>Do
>>>>> you only have one FIX session? If not, do all sessions show this
>behaviour?
>>>>> (apart from that, the TCP/IP stuff is done via the MINA framework
>and
>>>>> not within QFJ itself)
>>>>>
>>>>> This message appears when the initiator side of the connection
>>>>> considers the connection broken and tries to reconnect. But the
>acceptor
>>>>> still considers it a vital connection (probably until the
>connection
>>>>> timeout kicks in).
>>>>>
>>>>> From my experience this happens mostly on VPN connections via
>internet.
>>>>> Reasons for this were one of:
>>>>>  - different MTU sizes on both sides of the connection or on a
>>>>> router/firewall in between
>>>>>  - asymmetric routing
>>>>>  - differing VPN parameters leading to different rekeying
>behaviour on
>>>>> both ends of the connection.
>>>>>
>>>>> Hope that helps,
>>>>> Chris.
>>>>>
>>>>>
>>>>> On 30.04.20 05:51, Vipin Chaudhary wrote:
>>>>>
>>>>> QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
>>>>> QuickFIX/J Support: http://www.quickfixj.org/support/
>>>>>
>>>>>
>>>>> Hi Team,
>>>>>
>>>>> We are facing strange issue with quickfixj.
>>>>>
>>>>> We are SessionAcceptor, sometime when initiator disconnect, then
>>>>> quickfixj is not able to recognize the disconnection. So when
>client logon
>>>>> next time it say
>>>>> " Multiple logons/connections for this session are not allowed".
>>>>>   Although in reality client is disconnected.
>>>>> Earlier it was very rare and happening once in a while but
>nowadays its
>>>>> happening like once in week.
>>>>> Quickfixj is not able to recover from this and we need to restart
>our
>>>>> application
>>>>>
>>>>> *Do anyone have seen/fix this ?*
>>>>>
>>>>> Thanks
>>>>> Vipin Chaudhary
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Quickfixj-users mailing
>lis...@li...://lists.sourceforge.net/lists/listinfo/quickfixj-users
>>>>>
>>>>>
>>>>> --
>>>>> Christoph John
>>>>> Software Engineering
>>>>> T +49 241 557...@ma...
>>>>>
>>>>> MACD GmbH
>>>>> Oppenhoffallee 103
>>>>> 52066 Aachen, Germanywww.macd.com
>>>>>
>>>>> Amtsgericht Aachen: HRB 8151
>>>>> Ust.-Id: DE 813021663
>>>>> Geschäftsführer: George Macdonald
>>>>>
>>>>>
>>>>> --
>>>>> Christoph John
>>>>> Software Engineering
>>>>> T +49 241 557...@ma...
>>>>>
>>>>> MACD GmbH
>>>>> Oppenhoffallee 103
>>>>> 52066 Aachen, Germanywww.macd.com
>>>>>
>>>>> Amtsgericht Aachen: HRB 8151
>>>>> Ust.-Id: DE 813021663
>>>>> Geschäftsführer: George Macdonald
>>>>>
>>>>>
>>>> --
>>>> Christoph John
>>>> Software Engineering
>>>> T +49 241 557...@ma...
>>>>
>>>> MACD GmbH
>>>> Oppenhoffallee 103
>>>> 52066 Aachen, Germanywww.macd.com
>>>>
>>>> Amtsgericht Aachen: HRB 8151
>>>> Ust.-Id: DE 813021663
>>>> Geschäftsführer: George Macdonald
>>>>
>>>>
>>> --
>>> Christoph John
>>> Software Engineering
>>> T +49 241 557...@ma...
>>>
>>> MACD GmbH
>>> Oppenhoffallee 103
>>> 52066 Aachen, Germanywww.macd.com
>>>
>>> Amtsgericht Aachen: HRB 8151
>>> Ust.-Id: DE 813021663
>>> Geschäftsführer: George Macdonald
>>>
>>>
>> --
>> Christoph John
>> Software Engineering
>> T +49 241 557...@ma...
>>
>> MACD GmbH
>> Oppenhoffallee 103
>> 52066 Aachen, Germanywww.macd.com
>>
>> Amtsgericht Aachen: HRB 8151
>> Ust.-Id: DE 813021663
>> Geschäftsführer: George Macdonald
>>
>>

Re: [Quickfixj-users] Multiple logons/connections for this session are not allowed

[Vipin C. <vip...@gm...>]

Hi,

I have some detailed updates on this.

When this issue occur - During the first client LOGON the logon get hanged
(don't no why)
Following is the state in event log
    Accepting session FIX.4.4: *****Server->****client from /clientIP: 60868

    Acceptor heartbeat set to 60 seconds

After this the logon process get hanged somewhere and no LOGON message is
sent to client.

When client does not get his LOGON msg response than it disconnect and
retry.
 => This disconnect is not detected at our side and when new LOGON message
comes than at *AcceptorIoHandler.java *line no 69
*qfSession.hasResponder() => return true*
*But * when i checked *qfSession.getRemoteAddress() *than it return me*
null. *Also I tried qfSession.logout() which does not do anything.

In case of normal LOGON floe we have following state of event log
 Accepting session FIX.4.4: *****Server->****client from /clientIP: 60868
 Acceptor heartbeat set to 60 seconds
 Logon contains ResetSeqNumFlag=Y,resetting sequence numbers to 1
 Received logon
 Responding to Logon request

and then I can see the logon message in our outgoing log.

I am not able to replicate this issue on my local machine so I don't know
where it is getting hanged during first logon :(

Do anyone has any solution to this ?


Thanks
Vipin

On Thu, Aug 6, 2020 at 9:41 PM Christoph John <chr...@ma...>
wrote:

> Hi,
>
> when all clients are affected at the same time it could still be a network
> issue, right? :)
> As said, QFJ does not handle the TCP connection stuff by itself but uses
> MINA which is a stable and mature library. Of course, it still could have
> bugs. But your description rather sounds like all client's connection get
> closed and go into TIME_WAIT or CLOSE_WAIT state.
>
> https://superuser.com/questions/173535/what-are-close-wait-and-time-wait-states
>
> Did you check "netstat" or similar tool when the connection problem occurs?
>
> If you were to implement the logic that you proposed that would mean that
> someone could do a simple attack against your server which would not only
> end the new malicious session but only the one that is rightfully
> connected. Are you sure you want to follow that way? ;)
>
> Cheers,
> Chris.
>
> On 06.08.20 13:27, Vipin Chaudhary wrote:
>
> Hi guys.
>
> I am still facing this issue and now the frequency is very high. I am also
> not able to detect any networking issue.
>
> One more point is, it is happening with all clients at same time.  This
> does not resolve without the restart of application.
> One more strange thing is if I don't restart, then the client which was
> disconnected at the time of this issue, when try to connect at their
> session time also facing this issue.
>
> So now I am highly suspecting this as quickfix bug.
>
> As per me quickfixj should close old session if it detect that old session
> is still running when it receive new logon. To achieve the same I am
> thinking to edit *AcceptorIoHandler.java.*
>
> Class :  AcceptorIoHandler.java
> Line no 69
> if (qfSession.hasResponder()) {
>              // Session is already bound to another connection
>              sessionLog.onErrorEvent("Multiple logons/connections for this
> session are not allowed");
>              protocolSession.closeNow();
>              *//TODO Close old session here*
>              return;
>   }
>
> There are many methods to close the session in Session class as following
>
>    1.    qfSession.close();
>    2.    qfSession.disconnect("Closing Old session", true);
>    3.    qfSession.logout("Closing old session to accommodate new
>    session");
>    4.    qfSession.generateLogout();
>    5.    qfSession.reset();
>
> Which of the above method I should opt for, that can serve my purpose ?
>
> Thanks
> Vipin
>
>
> On Mon, May 4, 2020 at 3:47 PM Christoph John <chr...@ma...>
> wrote:
>
>> Hi,
>>
>> if VPN rekeying is not the problem then maybe differing MTU sizes or
>> asymmetric routing. Or of course it could also be another problem. Just
>> said what the problems mostly were in our case.
>> But I think if you describe the problem to your and your counterparty's
>> network team (since you only seem to have this problem with some of your
>> sessions) they should be able to debug it. Maybe your team need to do some
>> tcp dumps around the time of the problem.
>>
>> It would be nice if you could reply to the user group (not me privately)
>> if you found something out. This could help other users as well.
>>
>> Cheers,
>> Chris.
>>
>>
>> On 04.05.20 08:28, Vipin Chaudhary wrote:
>>
>> Hi Christoph,
>>
>> I double checked from the network team and find that no initiator use VPN
>> to connect to us.
>>
>> In that case, Can you help me on what should I request to network team to
>> fix the network connections?
>>
>> Thanks
>> Vipin Chaudhary
>>
>> On Thu, Apr 30, 2020 at 3:49 PM Christoph John <chr...@ma...>
>> wrote:
>>
>>> The only solution is to fix the network connection. Everything else is
>>> only a workaround.
>>> You could try to increase socket timeouts on both sides of the
>>> connection. Maybe it helps (depends on the cause of the problem) but as
>>> said this will only work around the problem.
>>>
>>> Cheers,
>>> Chris.
>>>
>>> On 30.04.20 12:11, Vipin Chaudhary wrote:
>>>
>>> Hi Christoph,
>>>
>>> Thanks for your input,
>>>
>>> We have multiple sessions and this problems does not happens with all
>>> sessions simultaneously. Mostly we see this problem with one session and
>>> rarely with other session.
>>>
>>> As far as I know initiator connect directly with us over internet (have
>>> ssl). Will double check on this with network team.
>>>
>>> Meanwhile any solution you think of this problem ?
>>>
>>> Thanks
>>> Vipin Chaudhary
>>>
>>> On Thu, Apr 30, 2020 at 3:31 PM Christoph John <chr...@ma...>
>>> wrote:
>>>
>>>> Addition: if VPN rekeying is the problem you will probably see this
>>>> message every hour (or whatever the rekey interval is)
>>>>
>>>> Chris.
>>>>
>>>> On 30.04.20 12:00, Christoph John wrote:
>>>>
>>>> Hi,
>>>>
>>>> did you change QFJ version or why do you think it is QFJ related? Do
>>>> you only have one FIX session? If not, do all sessions show this behaviour?
>>>> (apart from that, the TCP/IP stuff is done via the MINA framework and
>>>> not within QFJ itself)
>>>>
>>>> This message appears when the initiator side of the connection
>>>> considers the connection broken and tries to reconnect. But the acceptor
>>>> still considers it a vital connection (probably until the connection
>>>> timeout kicks in).
>>>>
>>>> From my experience this happens mostly on VPN connections via internet.
>>>> Reasons for this were one of:
>>>>  - different MTU sizes on both sides of the connection or on a
>>>> router/firewall in between
>>>>  - asymmetric routing
>>>>  - differing VPN parameters leading to different rekeying behaviour on
>>>> both ends of the connection.
>>>>
>>>> Hope that helps,
>>>> Chris.
>>>>
>>>>
>>>> On 30.04.20 05:51, Vipin Chaudhary wrote:
>>>>
>>>> QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
>>>> QuickFIX/J Support: http://www.quickfixj.org/support/
>>>>
>>>>
>>>> Hi Team,
>>>>
>>>> We are facing strange issue with quickfixj.
>>>>
>>>> We are SessionAcceptor, sometime when initiator disconnect, then
>>>> quickfixj is not able to recognize the disconnection. So when client logon
>>>> next time it say
>>>> " Multiple logons/connections for this session are not allowed".
>>>>   Although in reality client is disconnected.
>>>> Earlier it was very rare and happening once in a while but nowadays its
>>>> happening like once in week.
>>>> Quickfixj is not able to recover from this and we need to restart our
>>>> application
>>>>
>>>> *Do anyone have seen/fix this ?*
>>>>
>>>> Thanks
>>>> Vipin Chaudhary
>>>>
>>>>
>>>> _______________________________________________
>>>> Quickfixj-users mailing lis...@li...://lists.sourceforge.net/lists/listinfo/quickfixj-users
>>>>
>>>>
>>>> --
>>>> Christoph John
>>>> Software Engineering
>>>> T +49 241 557...@ma...
>>>>
>>>> MACD GmbH
>>>> Oppenhoffallee 103
>>>> 52066 Aachen, Germanywww.macd.com
>>>>
>>>> Amtsgericht Aachen: HRB 8151
>>>> Ust.-Id: DE 813021663
>>>> Geschäftsführer: George Macdonald
>>>>
>>>>
>>>> --
>>>> Christoph John
>>>> Software Engineering
>>>> T +49 241 557...@ma...
>>>>
>>>> MACD GmbH
>>>> Oppenhoffallee 103
>>>> 52066 Aachen, Germanywww.macd.com
>>>>
>>>> Amtsgericht Aachen: HRB 8151
>>>> Ust.-Id: DE 813021663
>>>> Geschäftsführer: George Macdonald
>>>>
>>>>
>>> --
>>> Christoph John
>>> Software Engineering
>>> T +49 241 557...@ma...
>>>
>>> MACD GmbH
>>> Oppenhoffallee 103
>>> 52066 Aachen, Germanywww.macd.com
>>>
>>> Amtsgericht Aachen: HRB 8151
>>> Ust.-Id: DE 813021663
>>> Geschäftsführer: George Macdonald
>>>
>>>
>> --
>> Christoph John
>> Software Engineering
>> T +49 241 557...@ma...
>>
>> MACD GmbH
>> Oppenhoffallee 103
>> 52066 Aachen, Germanywww.macd.com
>>
>> Amtsgericht Aachen: HRB 8151
>> Ust.-Id: DE 813021663
>> Geschäftsführer: George Macdonald
>>
>>
> --
> Christoph John
> Software Engineering
> T +49 241 557...@ma...
>
> MACD GmbH
> Oppenhoffallee 103
> 52066 Aachen, Germanywww.macd.com
>
> Amtsgericht Aachen: HRB 8151
> Ust.-Id: DE 813021663
> Geschäftsführer: George Macdonald
>
>

Re: [Quickfixj-users] Reading QF/J messages

[Christoph J. <chr...@ma...>]

OK, you are sending messages with repeating groups. Did misread that, thought you were receiving 
messages.

Read this:
https://www.quickfixj.org/usermanual/2.1.0/usage/repeating_groups.html

Cheers,
Chris.



On 08.08.20 18:21, Nenge Masoya wrote:
> Hi Christoph,
>
> I'm using the default  data dictionary from QF/J 2.2.0 (fix50sp2) and I am sending message 
> manually,, see the code below,
> Message message = new Message();
> message.getHeader().setString(MsgType.FIELD, "CF");
> message.getHeader().setField(new BeginString("FIXT.1.1"));
> message.setString(1505, requestDto.getPartRequestId());
> message.setString(1506, requestDto.getResponseGroup());
> message.setString(1507, requestDto.getResponseType());
>
> Thanks..
>
>
>
> On Sat, Aug 8, 2020 at 4:03 PM Christoph John <chr...@ma... 
> <mailto:chr...@ma...>> wrote:
>
>     Hi
>
>     Did you make sure to use setting UseDataDictionary=Y? Otherwise repeating groups cannot be
>     parsed correctly. Or else you are having an error in your dictionary.
>
>     Cheers
>     Chris.
>
>     Am 8. August 2020 13:21:08 MESZ schrieb Nenge Masoya <geo...@gm...
>     <mailto:geo...@gm...>>:
>
>         Dear Devs.
>
>         Thank you so much for your help, I was finally able to send party details list request 
>         messages to the server and get the response...
>
>         However, the response message does not come through fromApp method but I can see all data
>         in my message logs and this is because  same tag has been called more than once(according
>         to vendor design), this is the message.... "58=Tag appears more than once,
>         field=1517371=1517372=CG373=1310=232"...
>
>         My question is,, Is there a way to read the messages that does not come through fromApp?
>         because I can see all data in my log but I dont know how to get them apart from getting
>         them in fromApp method.
>
>         Thanks.
>

-- 
Christoph John
Software Engineering
T +49 241 557080-28
chr...@ma...

MACD GmbH
Oppenhoffallee 103
52066 Aachen, Germany
www.macd.com

Amtsgericht Aachen: HRB 8151
Ust.-Id: DE 813021663
Geschäftsführer: George Macdonald

Re: [Quickfixj-users] Reading QF/J messages

[Nenge M. <geo...@gm...>]

Hi Christoph,

I'm using the default  data dictionary from QF/J 2.2.0 (fix50sp2) and I am
sending message manually,, see the code below,

Message message = new Message();
message.getHeader().setString(MsgType.FIELD, "CF");
message.getHeader().setField(new BeginString("FIXT.1.1"));
message.setString(1505, requestDto.getPartRequestId());
message.setString(1506, requestDto.getResponseGroup());
message.setString(1507, requestDto.getResponseType());

Thanks..



On Sat, Aug 8, 2020 at 4:03 PM Christoph John <chr...@ma...>
wrote:

> Hi
>
> Did you make sure to use setting UseDataDictionary=Y? Otherwise repeating
> groups cannot be parsed correctly. Or else you are having an error in your
> dictionary.
>
> Cheers
> Chris.
>
> Am 8. August 2020 13:21:08 MESZ schrieb Nenge Masoya <geo...@gm...
> >:
>>
>> Dear Devs.
>>
>> Thank you so much for your help, I was finally able to send party details
>> list request  messages to the server and get the response...
>>
>> However, the response message does not come through fromApp method but I
>> can see all data in my  message logs and this is because  same tag has been
>> called more than once(according to vendor design), this is the message....
>> "58=Tag appears more than once, field=1517371=1517372=CG373=1310=232"...
>>
>> My question is,, Is there a way to read the messages that does not come
>> through fromApp? because I can see all data in my log but I dont know how
>> to get them apart from getting them in fromApp method.
>>
>> Thanks.
>>
>

Re: [Quickfixj-users] Reading QF/J messages

[Christoph J. <chr...@ma...>]

Hi

Did you make sure to use setting UseDataDictionary=Y? Otherwise repeating groups cannot be parsed correctly. Or else you are having an error in your dictionary. 

Cheers
Chris. 

Am 8. August 2020 13:21:08 MESZ schrieb Nenge Masoya <geo...@gm...>:
>QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
>QuickFIX/J Support: http://www.quickfixj.org/support/