quickfixj-users — General mailing list for QuickFIX/J topics.

Re: [Quickfixj-users] Capturing application message receive time in quickfixj

[Christoph J. <chr...@ma...>]

Hi Ankit,

I was not suggesting to use LoggingFilter but just to build your own filter and take LoggingFilter as an example. 😊

I did not use AspectJ yet. But your approach with the hash map should work. Just bear in mind that you have to treat the message as a String and need to parse it to find the ID which itself will also cost time.

Cheers
Chris


--
Christoph John
Software Engineering

D +49 241 557080 28
chr...@ma...

 

MACD GmbH
Oppenhoffallee 103
52066 Aachen, Germany
www.macd.com

    

Amtsgericht Aachen: HRB 8151
Ust.-Id: DE 813021663
Geschäftsführer: George Macdonald


> Hey Chris,
> 
> Following up on this - we are now thinking of using LoggingFilter and
> manage a hash map that will store the trade execution ID as key and the
> receive timestamp as its value.
> Will access the same time then during the actual message translation layer
> and do further calculations to get network latencies.
>
> AspectJ load weaver option is still open and we might explore that as well.
>
> Just running by you to see if you could share some expert advice or opine
> on a better way to achieve the same.
>
> Thanks,
> Ankit


On Thu, 4 Apr 2024 at 11:42 PM, Ankit Mehta <ank...@gm...>
wrote:

> Thanks for a reply Chris.
> But I am not sure if this will solve what we were looking for.
> For example, I can think of below implementation of the LoggingFilter -
> Note: We are initiators and use SocketInitiator class for the same.
> As soon as we have initialized the SocketInitiator, it can be followed by
> below piece of code -
>
> DefaultIoFilterChainBuilder builderChain = new DefaultIoFilterChainBuilder();
>
> builderCHain.addFirst("logging", new LoggingFilter());
>
> initiator.setIoFilterChainBuilder(builderChain);
>
>
> But from what I understand - all this will do is just start logging in the
> logs, whenever a message is received.
> What we were looking for instead is to capture that time somewhere in the
> message itself, which can be later utilized for some analysis (which we do
> after doing calculations between tag 52 and this newly added field on the
> message, which at present we do inside the fromApp callback)
>
> Maybe my understanding is wrong and I am missing something from what you
> were proposing. Please correct me if that is the case.
>
> Another alternative to achieve this that we have been thinking about is to
> use AspectJ and then do load weavering to amend the message and add the new
> receive time field inside the EventStrategy class under the onMessage
> method.
> Appreciate your thoughts on this approach or if you can propose a better
> alternative for our requirement.
>
> Regards,
> Ankit

Re: [Quickfixj-users] Capturing application message receive time in quickfixj

[Ankit M. <ank...@gm...>]

Hey Chris,

Following up on this - we are now thinking of using LoggingFilter and
manage a hash map that will store the trade execution ID as key and the
receive timestamp as its value.
Will access the same time then during the actual message translation layer
and do further calculations to get network latencies.

AspectJ load weaver option is still open and we might explore that as well.

Just running by you to see if you could share some expert advice or opine
on a better way to achieve the same.

Thanks,
Ankit


On Thu, 4 Apr 2024 at 11:42 PM, Ankit Mehta <ank...@gm...>
wrote:

> Thanks for a reply Chris.
> But I am not sure if this will solve what we were looking for.
> For example, I can think of below implementation of the LoggingFilter -
> Note: We are initiators and use SocketInitiator class for the same.
> As soon as we have initialized the SocketInitiator, it can be followed by
> below piece of code -
>
> DefaultIoFilterChainBuilder builderChain = new DefaultIoFilterChainBuilder();
>
> builderCHain.addFirst("logging", new LoggingFilter());
>
> initiator.setIoFilterChainBuilder(builderChain);
>
>
> But from what I understand - all this will do is just start logging in the
> logs, whenever a message is received.
> What we were looking for instead is to capture that time somewhere in the
> message itself, which can be later utilized for some analysis (which we do
> after doing calculations between tag 52 and this newly added field on the
> message, which at present we do inside the fromApp callback)
>
> Maybe my understanding is wrong and I am missing something from what you
> were proposing. Please correct me if that is the case.
>
> Another alternative to achieve this that we have been thinking about is to
> use AspectJ and then do load weavering to amend the message and add the new
> receive time field inside the EventStrategy class under the onMessage
> method.
> Appreciate your thoughts on this approach or if you can propose a better
> alternative for our requirement.
>
> Regards,
> Ankit
>
>
> On Thu, Apr 4, 2024 at 3:39 AM Christoph John <chr...@ma...>
> wrote:
>
>> QFJ Documentation: http://www.quickfixj.org/documentation/
>> QFJ Support: http://www.quickfixj.org/support/
>>
>>
>> Hi
>>
>> You could try to implement it as a MINA filter and add it as first filter
>> in the chain.
>>
>>
>> https://mina.apache.org/mina-project/userguide/ch5-filters/ch5-filters.html
>>
>> Have a look at e.g. the logging filter.
>>
>> The filter can then be added to your connector via
>> setIoFilterChainBuilder() on the connector.
>>
>> Hope that helps.
>>
>> Cheers
>> Chris
>>
>> _______________________________________________
>> Quickfixj-users mailing list
>> Qui...@li...
>> https://lists.sourceforge.net/lists/listinfo/quickfixj-users
>>
>

Re: [Quickfixj-users] Capturing application message receive time in quickfixj

[Ankit M. <ank...@gm...>]

Thanks for a reply Chris.
But I am not sure if this will solve what we were looking for.
For example, I can think of below implementation of the LoggingFilter -
Note: We are initiators and use SocketInitiator class for the same.
As soon as we have initialized the SocketInitiator, it can be followed by
below piece of code -

DefaultIoFilterChainBuilder builderChain = new DefaultIoFilterChainBuilder();

builderCHain.addFirst("logging", new LoggingFilter());

initiator.setIoFilterChainBuilder(builderChain);


But from what I understand - all this will do is just start logging in the
logs, whenever a message is received.
What we were looking for instead is to capture that time somewhere in the
message itself, which can be later utilized for some analysis (which we do
after doing calculations between tag 52 and this newly added field on the
message, which at present we do inside the fromApp callback)

Maybe my understanding is wrong and I am missing something from what you
were proposing. Please correct me if that is the case.

Another alternative to achieve this that we have been thinking about is to
use AspectJ and then do load weavering to amend the message and add the new
receive time field inside the EventStrategy class under the onMessage
method.
Appreciate your thoughts on this approach or if you can propose a better
alternative for our requirement.

Regards,
Ankit


On Thu, Apr 4, 2024 at 3:39 AM Christoph John <chr...@ma...>
wrote:

> QFJ Documentation: http://www.quickfixj.org/documentation/
> QFJ Support: http://www.quickfixj.org/support/
>
>
> Hi
>
> You could try to implement it as a MINA filter and add it as first filter
> in the chain.
>
> https://mina.apache.org/mina-project/userguide/ch5-filters/ch5-filters.html
>
> Have a look at e.g. the logging filter.
>
> The filter can then be added to your connector via
> setIoFilterChainBuilder() on the connector.
>
> Hope that helps.
>
> Cheers
> Chris
>
> _______________________________________________
> Quickfixj-users mailing list
> Qui...@li...
> https://lists.sourceforge.net/lists/listinfo/quickfixj-users
>

Re: [Quickfixj-users] Capturing application message receive time in quickfixj

[Christoph J. <chr...@ma...>]

Hi

You could try to implement it as a MINA filter and add it as first filter in the chain.

https://mina.apache.org/mina-project/userguide/ch5-filters/ch5-filters.html

Have a look at e.g. the logging filter.

The filter can then be added to your connector via setIoFilterChainBuilder() on the connector.

Hope that helps.

Cheers
Chris

Re: [Quickfixj-users] Capturing application message receive time in quickfixj

[Ankit M. <ank...@gm...>]

hello, wondering if anyone got a chance to have a look at this.

Thanks

On Wed, Apr 3, 2024 at 12:24 AM Ankit Mehta <ank...@gm...>
wrote:

> Hello all,
> We are using quickfixj to receive trade execution reports for various
> venues.
> We are interested in capturing certain network latency stats which is
> ideally the difference between the venue sending time which is stamped in
> tag 52 and the time message actually hit our application.
>
> Currently we stamp it under the fromApp method of the quickfixj
> application.
>
> The question I have is whether there is a better alternate way to capture
> the message received time somewhere in the socket layer, which can give us
> a more accurate picture of network latency.
>
> When I ran the application in DEBUG mode, I noticed that in the
> FixMesageDecoder class when the message is first parsed, the time at which
> the parsed message is printed is much earlier than the time it hits the
> fromApp method.
>
> That's why it's important for us to capture the receive time as early as
> possible so that we can figure out the network latency much more
> appropriately if not accurately.
>
> Eagerly looking forward to hearing some thoughts from you on this.
>
> Regards,
> Ankit
>

[Quickfixj-users] Capturing application message receive time in quickfixj

[Ankit M. <ank...@gm...>]

Hello all,
We are using quickfixj to receive trade execution reports for various
venues.
We are interested in capturing certain network latency stats which is
ideally the difference between the venue sending time which is stamped in
tag 52 and the time message actually hit our application.

Currently we stamp it under the fromApp method of the quickfixj application.

The question I have is whether there is a better alternate way to capture
the message received time somewhere in the socket layer, which can give us
a more accurate picture of network latency.

When I ran the application in DEBUG mode, I noticed that in the
FixMesageDecoder class when the message is first parsed, the time at which
the parsed message is printed is much earlier than the time it hits the
fromApp method.

That's why it's important for us to capture the receive time as early as
possible so that we can figure out the network latency much more
appropriately if not accurately.

Eagerly looking forward to hearing some thoughts from you on this.

Regards,
Ankit

Re: [Quickfixj-users] Is there any plan to support Simple Binary Encoding (SBE)

[Christoph J. <chr...@ma...>]

There is a discussion for this now:
https://github.com/quickfix-j/quickfixj/discussions/784

Our just leave comments on this mail thread.

I've heard a few people over the last few months asking if QFJ supported SBE. My opinion at that time was that QFJ does not need to support it since there are other open source (Silverflash, Aeron) and commercial alternatives which do support SBE.

However it seems that people want to use SBE from within QFJ. But why? Do they expect QFJ to map the messages to the well-known tag-value representation? At the time I was asked I obviously forgot to ask that question.

Cheers
Chris

Re: [Quickfixj-users] Seeking Advice: Handling Message Processing Errors and Pausing FIX Messages in QuickFIX/J

[Felipe W. <fel...@gm...>]

Hello Chris,

Thanks a lot for your advice.

Cheers,
Felipe


Em dom., 17 de mar. de 2024 16:30, Christoph John <chr...@ma...>
escreveu:

> QFJ Documentation: http://www.quickfixj.org/documentation/
> QFJ Support: http://www.quickfixj.org/support/
>
>
> Hi Felipe,
>
> I think the most natural and built in way would be to throw the exception
> which will not increment the seqnum and hence on the next incoming message
> a gap will be detected. You could optionally log off there if you want to
> sort out the issue first.
>
> Directly putting the message onto a queue and/or persisting it outside of
> QFJ is another option. It would also ensure that you spend as little time as
> possible in the fromApp callback, minimising blocking.
>
> Cheers
> Chris
>
>
>
>
> From: Felipe Windmoller <fel...@gm...>
> Sent: Sunday, March 17, 2024 2:24:53 PM
> To: qui...@li... <
> qui...@li...>
> Subject: [Quickfixj-users] Seeking Advice: Handling Message Processing
> Errors and Pausing FIX Messages in QuickFIX/J
>
>
> Dear QuickFIX/J community,
>
> I am seeking advice on how to manage messages from a drop copy session.
> Each business message must be processed by a separate application, and I
> need to ensure that every message is processed without fail.
>
> I am considering how to handle scenarios where, during the execution of the
> fromApp() method, calling this external application results in an error. Is
> it feasible to pause the processing of my FIX messages until the external
> call is successful? In such a scenario, the pending messages would
> accumulate. Would this approach be advisable? Additionally, I am curious
> about the mechanisms available for halting QFJ processing in response to an
> error encountered within the fromApp() method. Specifically, how can I
> effectively stop the processing of messages by QFJ at this juncture? Would
> it involve rejecting the problematic FIX message, thereby necessitating its
> resending by the FIX counterparty?
>
> I have contemplated the following strategies:
>
>    1.
>
>    1. Utilizing the QFJ Incoming Log:
>
>    My idea is to configure the QFJ application to solely receive and log
>    FIX messages. A separate application would then periodically (every 1
>    minute) check this log, guided by a control table that tracks the last
>    sequence number processed. It would then process the messages
> accordingly.
>    Point of Attention:
>
>    As Christoph John has elucidated on StackOverflow
>    <https://stackoverflow.com/a/78167504/11110455>, *"QFJ logs messages
>    upon receipt at AbstractIoHandler.java
>    <
> https://github.com/quickfix-j/quickfixj/blob/d7c9fe52fabdc56f28ace4063f6ac78fe422b511/quickfixj-core/src/main/java/quickfix/mina/AbstractIoHandler.java#L142
> >.
>    Message parsing and subsequent processing occur later; only then is the
>    message conveyed to the Session class where it undergoes validation,
>    leading to a potential call to fromApp() if validation is successful."*
>    Consequently, there's an inherent uncertainty about processing solely
> valid
>    messages.
>
>    2.
>
>
>    3.
>
>    2. Employing a Message Broker:
>
>    Another approach is to leverage a message broker as an intermediary in
>    the fromApp() method to queue the received messages. This would
>    essentially serve as a buffer, allowing another application to process
>    these messages asynchronously.
>    Point of Attention:
>
>    A critical consideration is the strategy to adopt if the message broker
>    becomes unavailable during the execution of a fromApp() method call.
>
>    4.
>
>
>    5.
>
>    3. Storing Pending Messages in a Database Table:
>
>    Similar to using a message broker, this method involves storing the
>    messages received in the fromApp() method in a database table. This
>    table would then act as a buffer until the messages can be processed.
>
>
> I greatly appreciate your insights and suggestions on these approaches or
> any other potential solutions you might recommend.
>
> Thank you, Felipe Windmoller
>
>
>
> _______________________________________________
> Quickfixj-users mailing list
> Qui...@li...
> https://lists.sourceforge.net/lists/listinfo/quickfixj-users
>

Re: [Quickfixj-users] Seeking Advice: Handling Message Processing Errors and Pausing FIX Messages in QuickFIX/J

[Philip W. <Phi...@fl...>]

Hi,

You are almost certainly better off storing the messages you receive and then ensuring that each FIX message is processed within your app. If an individual message has an error you can reject it (but this would not normally lead to a resend).

While you can re-request messages repeatedly it's a fairly hostile behaviour and many brokers may chose not to resend older, more ephemeral data (market data)

Sent from Outlook for iOS<https://aka.ms/o0ukef>
________________________________
From: Felipe Windmoller <fel...@gm...>
Sent: Sunday, March 17, 2024 1:23:10 PM
To: qui...@li... <qui...@li...>
Subject: [Quickfixj-users] Seeking Advice: Handling Message Processing Errors and Pausing FIX Messages in QuickFIX/J

ATTENTION: This email was sent from someone outside of FlexTrade. Always use caution when opening attachments or clicking links.

QFJ Documentation: http://www.quickfixj.org/documentation/
QFJ Support: http://www.quickfixj.org/support/




This communication is for informational purposes only. The contents of this transmission are confidential and are intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error, please notify the sender by return email and delete this message from your system. FlexTrade Systems Inc., its subsidiaries and affiliates do not guarantee the completeness and accuracy of this transmission's contents. Moreover, FlexTrade Systems Inc., its subsidiaries and affiliates do not guarantee this communication to be free of viruses and accept no liability for any damage caused thereof.

Re: [Quickfixj-users] Seeking Advice: Handling Message Processing Errors and Pausing FIX Messages in QuickFIX/J

[Christoph J. <chr...@ma...>]

Hi Felipe,

I think the most natural and built in way would be to throw the exception which will not increment the seqnum and hence on the next incoming message a gap will be detected. You could optionally log off there if you want to sort out the issue first.

Directly putting the message onto a queue and/or persisting it outside of QFJ is another option. It would also ensure that you spend as little time as possible in the fromApp callback, minimising blocking.

Cheers
Chris




From: Felipe Windmoller <fel...@gm...>
Sent: Sunday, March 17, 2024 2:24:53 PM
To: qui...@li... <qui...@li...>
Subject: [Quickfixj-users] Seeking Advice: Handling Message Processing Errors and Pausing FIX Messages in QuickFIX/J


Dear QuickFIX/J community,

I am seeking advice on how to manage messages from a drop copy session.
Each business message must be processed by a separate application, and I
need to ensure that every message is processed without fail.

I am considering how to handle scenarios where, during the execution of the
fromApp() method, calling this external application results in an error. Is
it feasible to pause the processing of my FIX messages until the external
call is successful? In such a scenario, the pending messages would
accumulate. Would this approach be advisable? Additionally, I am curious
about the mechanisms available for halting QFJ processing in response to an
error encountered within the fromApp() method. Specifically, how can I
effectively stop the processing of messages by QFJ at this juncture? Would
it involve rejecting the problematic FIX message, thereby necessitating its
resending by the FIX counterparty?

I have contemplated the following strategies:

   1.

   1. Utilizing the QFJ Incoming Log:

   My idea is to configure the QFJ application to solely receive and log
   FIX messages. A separate application would then periodically (every 1
   minute) check this log, guided by a control table that tracks the last
   sequence number processed. It would then process the messages accordingly.
   Point of Attention:

   As Christoph John has elucidated on StackOverflow
   <https://stackoverflow.com/a/78167504/11110455>, *"QFJ logs messages
   upon receipt at AbstractIoHandler.java
   <https://github.com/quickfix-j/quickfixj/blob/d7c9fe52fabdc56f28ace4063f6ac78fe422b511/quickfixj-core/src/main/java/quickfix/mina/AbstractIoHandler.java#L142>.
   Message parsing and subsequent processing occur later; only then is the
   message conveyed to the Session class where it undergoes validation,
   leading to a potential call to fromApp() if validation is successful."*
   Consequently, there's an inherent uncertainty about processing solely valid
   messages.

   2.


   3.

   2. Employing a Message Broker:

   Another approach is to leverage a message broker as an intermediary in
   the fromApp() method to queue the received messages. This would
   essentially serve as a buffer, allowing another application to process
   these messages asynchronously.
   Point of Attention:

   A critical consideration is the strategy to adopt if the message broker
   becomes unavailable during the execution of a fromApp() method call.

   4.


   5.

   3. Storing Pending Messages in a Database Table:

   Similar to using a message broker, this method involves storing the
   messages received in the fromApp() method in a database table. This
   table would then act as a buffer until the messages can be processed.


I greatly appreciate your insights and suggestions on these approaches or
any other potential solutions you might recommend.

Thank you, Felipe Windmoller

Re: [Quickfixj-users] Seeking Advice: Handling Message Processing Errors and Pausing FIX Messages in QuickFIX/J

[Felipe W. <fel...@gm...>]

Hello Robert,

Thank you very much for sharing your insights.

Based on your advice, I’ve decided to implement a message broker for
processing my FIX messages. This approach seems to align well with ensuring
that our system remains efficient and responsive, particularly during heavy
processing periods.

I have a query that may have been covered in the QFJ documentation, but I
would appreciate your expertise on it. Could you advise on the recommended
procedure for gracefully logging off a session if the message broker
becomes unavailable?

To manage situations where the message broker is down, I'm considering the
following procedure within the `fromApp()` callback:
1. Attempt to enqueue the FIX message onto the broker.
2. If the attempt fails, retry a predefined number of times (the exact
number of retries would be configurable).
3. Should these attempts to place the message on the broker fail, I plan to
initiate a logoff from the FIX session and then throw an exception to
ensure that QFJ does not mark this message sequence as successfully
processed in its session control.
4. Additionally, trigger an alert to notify our team that there is an issue
with the message broker. This will ensure that we are aware of the problem
immediately and can take action to resolve it as quickly as possible.

Do you think this approach is practical? I'm aiming to strike a balance
between robustness and maintaining session integrity, ensuring we don’t
lose messages or unnecessarily disrupt the session with our FIX
counterparties.

Thank you again for your valuable feedback.

Best regards,
Felipe Windmoller

Em dom., 17 de mar. de 2024 14:02, Robert Nicholson <ro...@el...>
escreveu:

> QFJ Documentation: http://www.quickfixj.org/documentation/
> QFJ Support: http://www.quickfixj.org/support/
>
>
> If you cannot heartbeat in time because your main thread is processing the
> message then you will have problems. You want to minimize the need to
> logoff, logon again and send resend requests. The other end isn’t going to
> tolerate that much.
>
> In general when you listen on FIX and you have expensive processing to do
> you should be using a messaging layer and not doing expensive progressing
> in the handlers of your FIX callbacks. You want that code to be as light as
> possible. In general you want to be able to continue listening to FIX
> messages even when you have resource issues on your side related to
> processing the messages. Worse case you have to logoff and potentially
> sleep forever if you cannot “queue” the message for processing later. The
> idea being that your alerting catches that and procedures are in place to
> resume processing later. What you don’t want to do is miss messages because
> the reset takes place whilst you’re down.
>
>
> On Mar 17, 2024, at 8:23 AM, Felipe Windmoller <fel...@gm...>
> wrote:
>
> QFJ Documentation: http://www.quickfixj.org/documentation/
> QFJ Support: http://www.quickfixj.org/support/
>
>
> Dear QuickFIX/J community,
>
> I am seeking advice on how to manage messages from a drop copy session.
> Each business message must be processed by a separate application, and I
> need to ensure that every message is processed without fail.
>
> I am considering how to handle scenarios where, during the execution of
> the fromApp() method, calling this external application results in an
> error. Is it feasible to pause the processing of my FIX messages until the
> external call is successful? In such a scenario, the pending messages would
> accumulate. Would this approach be advisable? Additionally, I am curious
> about the mechanisms available for halting QFJ processing in response to an
> error encountered within the fromApp() method. Specifically, how can I
> effectively stop the processing of messages by QFJ at this juncture? Would
> it involve rejecting the problematic FIX message, thereby necessitating its
> resending by the FIX counterparty?
>
> I have contemplated the following strategies:
>
>    1. 1. Utilizing the QFJ Incoming Log:
>
>    My idea is to configure the QFJ application to solely receive and log
>    FIX messages. A separate application would then periodically (every 1
>    minute) check this log, guided by a control table that tracks the last
>    sequence number processed. It would then process the messages accordingly.
>    Point of Attention:
>
>    As Christoph John has elucidated on StackOverflow
>    <https://stackoverflow.com/a/78167504/11110455>, *"QFJ logs messages
>    upon receipt at AbstractIoHandler.java
>    <https://github.com/quickfix-j/quickfixj/blob/d7c9fe52fabdc56f28ace4063f6ac78fe422b511/quickfixj-core/src/main/java/quickfix/mina/AbstractIoHandler.java#L142>.
>    Message parsing and subsequent processing occur later; only then is the
>    message conveyed to the Session class where it undergoes validation,
>    leading to a potential call to fromApp() if validation is successful."*
>    Consequently, there's an inherent uncertainty about processing solely valid
>    messages.
>
>
>    2.
>    3. 2. Employing a Message Broker:
>
>    Another approach is to leverage a message broker as an intermediary in
>    the fromApp() method to queue the received messages. This would
>    essentially serve as a buffer, allowing another application to process
>    these messages asynchronously.
>    Point of Attention:
>
>    A critical consideration is the strategy to adopt if the message
>    broker becomes unavailable during the execution of a fromApp() method
>    call.
>
>
>    4.
>    5. 3. Storing Pending Messages in a Database Table:
>
>    Similar to using a message broker, this method involves storing the
>    messages received in the fromApp() method in a database table. This
>    table would then act as a buffer until the messages can be processed.
>
>
>
> I greatly appreciate your insights and suggestions on these approaches or
> any other potential solutions you might recommend.
>
> Thank you, Felipe Windmoller
> _______________________________________________
> Quickfixj-users mailing list
> Qui...@li...
> https://lists.sourceforge.net/lists/listinfo/quickfixj-users
>
>
> _______________________________________________
> Quickfixj-users mailing list
> Qui...@li...
> https://lists.sourceforge.net/lists/listinfo/quickfixj-users
>

Re: [Quickfixj-users] Seeking Advice: Handling Message Processing Errors and Pausing FIX Messages in QuickFIX/J

[Robert N. <ro...@el...>]

If you cannot heartbeat in time because your main thread is processing the message then you will have problems. You want to minimize the need to logoff, logon again and send resend requests. The other end isn’t going to tolerate that much.

In general when you listen on FIX and you have expensive processing to do you should be using a messaging layer and not doing expensive progressing in the handlers of your FIX callbacks. You want that code to be as light as possible. In general you want to be able to continue listening to FIX messages even when you have resource issues on your side related to processing the messages. Worse case you have to logoff and potentially sleep forever if you cannot “queue” the message for processing later. The idea being that your alerting catches that and procedures are in place to resume processing later. What you don’t want to do is miss messages because the reset takes place whilst you’re down.


> On Mar 17, 2024, at 8:23 AM, Felipe Windmoller <fel...@gm...> wrote:
> 
> QFJ Documentation: http://www.quickfixj.org/documentation/
> QFJ Support: http://www.quickfixj.org/support/
> 
> 
> Dear QuickFIX/J community,
> 
> I am seeking advice on how to manage messages from a drop copy session. Each business message must be processed by a separate application, and I need to ensure that every message is processed without fail.
> 
> I am considering how to handle scenarios where, during the execution of the fromApp() method, calling this external application results in an error. Is it feasible to pause the processing of my FIX messages until the external call is successful? In such a scenario, the pending messages would accumulate. Would this approach be advisable? Additionally, I am curious about the mechanisms available for halting QFJ processing in response to an error encountered within the fromApp() method. Specifically, how can I effectively stop the processing of messages by QFJ at this juncture? Would it involve rejecting the problematic FIX message, thereby necessitating its resending by the FIX counterparty?
> 
> I have contemplated the following strategies:
> 
> 1. Utilizing the QFJ Incoming Log:
> My idea is to configure the QFJ application to solely receive and log FIX messages. A separate application would then periodically (every 1 minute) check this log, guided by a control table that tracks the last sequence number processed. It would then process the messages accordingly.
> Point of Attention:
> As Christoph John has elucidated on StackOverflow <https://stackoverflow.com/a/78167504/11110455>, "QFJ logs messages upon receipt at AbstractIoHandler.java <https://github.com/quickfix-j/quickfixj/blob/d7c9fe52fabdc56f28ace4063f6ac78fe422b511/quickfixj-core/src/main/java/quickfix/mina/AbstractIoHandler.java#L142>. Message parsing and subsequent processing occur later; only then is the message conveyed to the Session class where it undergoes validation, leading to a potential call to fromApp() if validation is successful." Consequently, there's an inherent uncertainty about processing solely valid messages.
> 
> 
> 2. Employing a Message Broker:
> Another approach is to leverage a message broker as an intermediary in the fromApp() method to queue the received messages. This would essentially serve as a buffer, allowing another application to process these messages asynchronously.
> Point of Attention:
> A critical consideration is the strategy to adopt if the message broker becomes unavailable during the execution of a fromApp() method call.
> 
> 
> 3. Storing Pending Messages in a Database Table:
> Similar to using a message broker, this method involves storing the messages received in the fromApp() method in a database table. This table would then act as a buffer until the messages can be processed.
> 
> I greatly appreciate your insights and suggestions on these approaches or any other potential solutions you might recommend.
> 
> Thank you, Felipe Windmoller
> _______________________________________________
> Quickfixj-users mailing list
> Qui...@li...
> https://lists.sourceforge.net/lists/listinfo/quickfixj-users

[Quickfixj-users] Seeking Advice: Handling Message Processing Errors and Pausing FIX Messages in QuickFIX/J

[Felipe W. <fel...@gm...>]

Dear QuickFIX/J community,

I am seeking advice on how to manage messages from a drop copy session.
Each business message must be processed by a separate application, and I
need to ensure that every message is processed without fail.

I am considering how to handle scenarios where, during the execution of the
fromApp() method, calling this external application results in an error. Is
it feasible to pause the processing of my FIX messages until the external
call is successful? In such a scenario, the pending messages would
accumulate. Would this approach be advisable? Additionally, I am curious
about the mechanisms available for halting QFJ processing in response to an
error encountered within the fromApp() method. Specifically, how can I
effectively stop the processing of messages by QFJ at this juncture? Would
it involve rejecting the problematic FIX message, thereby necessitating its
resending by the FIX counterparty?

I have contemplated the following strategies:

   1.

   1. Utilizing the QFJ Incoming Log:

   My idea is to configure the QFJ application to solely receive and log
   FIX messages. A separate application would then periodically (every 1
   minute) check this log, guided by a control table that tracks the last
   sequence number processed. It would then process the messages accordingly.
   Point of Attention:

   As Christoph John has elucidated on StackOverflow
   <https://stackoverflow.com/a/78167504/11110455>, *"QFJ logs messages
   upon receipt at AbstractIoHandler.java
   <https://github.com/quickfix-j/quickfixj/blob/d7c9fe52fabdc56f28ace4063f6ac78fe422b511/quickfixj-core/src/main/java/quickfix/mina/AbstractIoHandler.java#L142>.
   Message parsing and subsequent processing occur later; only then is the
   message conveyed to the Session class where it undergoes validation,
   leading to a potential call to fromApp() if validation is successful."*
   Consequently, there's an inherent uncertainty about processing solely valid
   messages.

   2.


   3.

   2. Employing a Message Broker:

   Another approach is to leverage a message broker as an intermediary in
   the fromApp() method to queue the received messages. This would
   essentially serve as a buffer, allowing another application to process
   these messages asynchronously.
   Point of Attention:

   A critical consideration is the strategy to adopt if the message broker
   becomes unavailable during the execution of a fromApp() method call.

   4.


   5.

   3. Storing Pending Messages in a Database Table:

   Similar to using a message broker, this method involves storing the
   messages received in the fromApp() method in a database table. This
   table would then act as a buffer until the messages can be processed.


I greatly appreciate your insights and suggestions on these approaches or
any other potential solutions you might recommend.

Thank you, Felipe Windmoller

Re: [Quickfixj-users] Issue receiving messages in the fromApp method after sending a Quote Request Reject (35=AG)

[Felipe W. <fel...@gm...>]

I collaborate with @Adan Vinicius
Troubleshooting the Issue

During our investigation, we discovered the following sequence of events:

   1. We sent a *35=AG* (Request Reject) message.
   2. In response, we received a *35=Z* (Quote Cancel) message.
   3. While processing the *35=Z* message, our application encountered a
   NullPointerException. Consequently, QuickFIX/J (QFJ) was unable to store
   this sequence number message in its FIX session control.

Handling the Missing Message

When we subsequently received a *35=R* (Quote Request) message, QFJ
automatically sent a *35=2* (Resend Request). From QFJ’s perspective, a
message was missing.

Interestingly, the *35=R* message was successfully stored in the QFJ log.
However, our fromApp() method in our Application object was not being
executed. This discrepancy caused confusion.
QFJ Flow and Handling Invalid Messages

We made an important observation: when QFJ identifies an invalid message
(such as an incorrect sequence number), it still persists that message in
the QFJ log. However, the fromApp() method is not executed.


This answer is available on StackOverflow

https://stackoverflow.com/questions/78063637/issue-receiving-messages-in-the-fromapp-method-after-sending-a-quote-request-rej


Felipe Windmoller

On Tue, Feb 27, 2024 at 1:18 PM Grant Birchmeier <gbi...@co...>
wrote:

> QFJ Documentation: http://www.quickfixj.org/documentation/
> QFJ Support: http://www.quickfixj.org/support/
>
>
> If an incoming message is not handled by fromApp, then I suspect it's
> being rejected at the transport level.  In that case, I believe you'd see
> an outgoing 35=3 message in your outgoing message log.
>
>
>
>
> On Tue, Feb 27, 2024 at 9:02 AM Vinicius Da Silva Leitao, Adan <
> avi...@mi...> wrote:
>
>> QFJ Documentation: http://www.quickfixj.org/documentation/
>> QFJ Support: http://www.quickfixj.org/support/
>>
>>
>> I have an application that uses QuickFix version 4.4. In this
>> application, we receive a Quote Request (35-R) and perform some
>> validations. If any of the validations are not met, the application issues
>> a Quote Request Reject (35-AG). The problem arises when this scenario
>> occurs: subsequent Quote Requests (35-R) are not received by the fromApp
>> method for processing, as if the application were blocked (although it
>> continues to return administrative messages normally: fromAdmin and
>> toAdmin). After a certain period, which can vary from 1 to 10 minutes, the
>> application resumes normal processing. It's important to note that the
>> Quote Request messages that are not processed by the fromApp method are
>> still recorded in the database (JdbcLogIncomingTable) without any issues.
>>
>> Sincerely,
>>
>> Adan Vinícius.
>>
>> ------------------------------
>>
>> Este correo electrónico y, en su caso, cualquier fichero anexo al mismo,
>> contiene información de carácter confidencial exclusivamente dirigida a su
>> destinatario o destinatarios. Si no es vd. el destinatario indicado, queda
>> notificado que la lectura, utilización, divulgación y/o copia sin
>> autorización está prohibida en virtud de la legislación vigente. En el caso
>> de haber recibido este correo electrónico por error, se ruega notificar
>> inmediatamente esta circunstancia mediante reenvío a la dirección
>> electrónica del remitente.
>> Evite imprimir este mensaje si no es estrictamente necesario.
>>
>> This email and any file attached to it (when applicable) contain(s)
>> confidential information that is exclusively addressed to its recipient(s).
>> If you are not the indicated recipient, you are informed that reading,
>> using, disseminating and/or copying it without authorisation is forbidden
>> in accordance with the legislation in effect. If you have received this
>> email by mistake, please immediately notify the sender of the situation by
>> resending it to their email address.
>> Avoid printing this message if it is not absolutely necessary.
>> _______________________________________________
>> Quickfixj-users mailing list
>> Qui...@li...
>> https://lists.sourceforge.net/lists/listinfo/quickfixj-users
>>
>
>
> --
> <https://www.connamara.com>
>
> Grant Birchmeier
>
> Sr. Software Engineer, Connamara
>
> gbi...@co...
>
> This email, along with any attachments, is confidential. If you believe
> you received this message in error, please contact the sender immediately
> and delete all copies of the message. Thank you from Connamara Systems, LLC.
> _______________________________________________
> Quickfixj-users mailing list
> Qui...@li...
> https://lists.sourceforge.net/lists/listinfo/quickfixj-users
>

Re: [Quickfixj-users] Issue receiving messages in the fromApp method after sending a Quote Request Reject (35=AG)

[Grant B. <gbi...@co...>]

If an incoming message is not handled by fromApp, then I suspect it's being
rejected at the transport level.  In that case, I believe you'd see an
outgoing 35=3 message in your outgoing message log.




On Tue, Feb 27, 2024 at 9:02 AM Vinicius Da Silva Leitao, Adan <
avi...@mi...> wrote:

> QFJ Documentation: http://www.quickfixj.org/documentation/
> QFJ Support: http://www.quickfixj.org/support/
>
>
> I have an application that uses QuickFix version 4.4. In this application,
> we receive a Quote Request (35-R) and perform some validations. If any of
> the validations are not met, the application issues a Quote Request Reject
> (35-AG). The problem arises when this scenario occurs: subsequent Quote
> Requests (35-R) are not received by the fromApp method for processing, as
> if the application were blocked (although it continues to return
> administrative messages normally: fromAdmin and toAdmin). After a certain
> period, which can vary from 1 to 10 minutes, the application resumes normal
> processing. It's important to note that the Quote Request messages that are
> not processed by the fromApp method are still recorded in the database
> (JdbcLogIncomingTable) without any issues.
>
> Sincerely,
>
> Adan Vinícius.
>
> ------------------------------
>
> Este correo electrónico y, en su caso, cualquier fichero anexo al mismo,
> contiene información de carácter confidencial exclusivamente dirigida a su
> destinatario o destinatarios. Si no es vd. el destinatario indicado, queda
> notificado que la lectura, utilización, divulgación y/o copia sin
> autorización está prohibida en virtud de la legislación vigente. En el caso
> de haber recibido este correo electrónico por error, se ruega notificar
> inmediatamente esta circunstancia mediante reenvío a la dirección
> electrónica del remitente.
> Evite imprimir este mensaje si no es estrictamente necesario.
>
> This email and any file attached to it (when applicable) contain(s)
> confidential information that is exclusively addressed to its recipient(s).
> If you are not the indicated recipient, you are informed that reading,
> using, disseminating and/or copying it without authorisation is forbidden
> in accordance with the legislation in effect. If you have received this
> email by mistake, please immediately notify the sender of the situation by
> resending it to their email address.
> Avoid printing this message if it is not absolutely necessary.
> _______________________________________________
> Quickfixj-users mailing list
> Qui...@li...
> https://lists.sourceforge.net/lists/listinfo/quickfixj-users
>


-- 
<https://www.connamara.com>

Grant Birchmeier

Sr. Software Engineer, Connamara

gbi...@co...

-- 
This email, along with any attachments, is confidential. If you believe you 
received this message in error, please contact the sender immediately and 
delete all copies of the message. Thank you from Connamara Systems, LLC.

[Quickfixj-users] Issue receiving messages in the fromApp method after sending a Quote Request Reject (35=AG)

[Vinicius Da S. L. A. <avi...@mi...>]

I have an application that uses QuickFix version 4.4. In this application, we receive a Quote Request (35-R) and perform some validations. If any of the validations are not met, the application issues a Quote Request Reject (35-AG). The problem arises when this scenario occurs: subsequent Quote Requests (35-R) are not received by the fromApp method for processing, as if the application were blocked (although it continues to return administrative messages normally: fromAdmin and toAdmin). After a certain period, which can vary from 1 to 10 minutes, the application resumes normal processing. It's important to note that the Quote Request messages that are not processed by the fromApp method are still recorded in the database (JdbcLogIncomingTable) without any issues.

Sincerely,

Adan Vinícius.

________________________________

Este correo electrónico y, en su caso, cualquier fichero anexo al mismo, contiene información de carácter confidencial exclusivamente dirigida a su destinatario o destinatarios. Si no es vd. el destinatario indicado, queda notificado que la lectura, utilización, divulgación y/o copia sin autorización está prohibida en virtud de la legislación vigente. En el caso de haber recibido este correo electrónico por error, se ruega notificar inmediatamente esta circunstancia mediante reenvío a la dirección electrónica del remitente.
Evite imprimir este mensaje si no es estrictamente necesario.

This email and any file attached to it (when applicable) contain(s) confidential information that is exclusively addressed to its recipient(s). If you are not the indicated recipient, you are informed that reading, using, disseminating and/or copying it without authorisation is forbidden in accordance with the legislation in effect. If you have received this email by mistake, please immediately notify the sender of the situation by resending it to their email address.
Avoid printing this message if it is not absolutely necessary.

Re: [Quickfixj-users] Is there any plan to support Simple Binary Encoding (SBE)

[Christoph J. <chr...@ma...>]

Hi,

as said, QFJ is no commercial FIX engine. It is open source. I don’t get any extra money to implement features or fix bugs. Everyone can contribute to the project. Of course a company could sponsor a certain feature with money or contribute code.

As far as I remember you are the first person to request that QFJ should support SBE. Given that there are open source implementations for FIXP and SBE, I don’t think that someone will come forward to take the time and develop this for QFJ (if it is even technically feasible, I haven’t checked it).
Maybe you want to give it a go and create a PR for it? I’d be happy to review it then. 😊

Thank you and best regards
Chris


From: JinJin Huang <jin...@gm...>
Sent: 11 January 2024 23:04
To: qui...@li...
Subject: Re: [Quickfixj-users] Is there any plan to support Simple Binary Encoding (SBE)

Sie erhalten nicht oft eine E-Mail von jin...@gm.... Erfahren Sie, warum dies wichtig ist<https://aka.ms/LearnAboutSenderIdentification>
Yes, FIXP is a session layer (OSI layer 5) protocol for better performance compared to FIXT, there is already a reference implementation using SBE (simple binary encoding) in https://github.com/FIXTradingCommunity/silverflash

But it will be good if quickfixj (as an application layer implementation) also supports SBE, like other commercial products as mentioned above.

This diagram (steal from web)  will be good to illustrate the above idea

(picture removed)

Regards
JH


On Thu, 11 Jan 2024 at 21:20, Laurent Danesi <ld...@sm...<mailto:ld...@sm...>> wrote:
QFJ Documentation: http://www.quickfixj.org/documentation/
QFJ Support: http://www.quickfixj.org/support/

Hi,

I think it is related to the "new" FIXP (https://www.fixtrading.org/standards/fixp/) standard from the FIX Trading Community.
The idea is to normalize a fast and binary (instead of text) protocol keeping the FIX semantic.
It is inspired from soupTCP or other binary protocol tentative from some exchange.
The control is very light but the protocol is very different than FIX4.x or FIX5 and Application messages are custom for each exchange: no way to use FIX messages and getField(xxx) as we do.

The messages and codecs are generated from a sbe schema file.

Regards,
Lo

Le jeu. 11 janv. 2024, 19:52, Colin DuPlantis <co...@ma...<mailto:co...@ma...>> a écrit :
QFJ Documentation: http://www.quickfixj.org/documentation/
QFJ Support: http://www.quickfixj.org/support/


Out of curiosity, what are SBEs in this context and what is the use case for FIX supporting them?

> On Jan 11, 2024, at 8:34 AM, Christoph John <chr...@ma...<mailto:chr...@ma...>> wrote:
>
> QFJ Documentation: http://www.quickfixj.org/documentation/
> QFJ Support: http://www.quickfixj.org/support/
>
>
> Hi,
>
> There are currently no such plans. Please keep in mind that we are no commercial FIX engine but an open source one. 😊 So if anyone has added this on top of QFJ they would be welcome to contribute it back to the project.
> I could imagine that it is possible to implement it in some kind of MINA filter on top of QFJ but I cannot say how much effort that will be.
> However, there is a reference implementation for SBE on github here: https://github.com/real-logic/simple-binary-encoding
>
> Cheers
> Chris
>
> --
> Christoph John
> Software Engineering
>
> D +49 241 557080 28
> chr...@ma...<mailto:chr...@ma...>
>
>
>
> MACD GmbH
> Oppenhoffallee 103
> 52066 Aachen, Germany
> www.macd.com<http://www.macd.com/>
>
>
>
> Amtsgericht Aachen: HRB 8151
> Ust.-Id: DE 813021663
> Geschäftsführer: George Macdonald
>
>
>
> -----Original Message-----
> From: JinJin Huang <jin...@gm...<mailto:jin...@gm...>>
> Sent: 11 January 2024 00:49
> To: qui...@li...<mailto:qui...@li...>
> Subject: [Quickfixj-users] Is there any plan to support Simple Binary Encoding (SBE)
>
>
>
> Hi, Chris and Quickfixj experts
>
> Is there any plan to support SBE in your current or further development?
> Like other commercial FIX engines, such as Chronicle and Onixs.
>
> Kind Regards
> JH
>
> _______________________________________________
> Quickfixj-users mailing list
> Qui...@li...<mailto:Qui...@li...>
> https://lists.sourceforge.net/lists/listinfo/quickfixj-users



_______________________________________________
Quickfixj-users mailing list
Qui...@li...<mailto:Qui...@li...>
https://lists.sourceforge.net/lists/listinfo/quickfixj-users

[cid:~WRD0001.jpg]<https://smart-trade.net/>
Linkedin<https://www.linkedin.com/company/smart-trade-technologies> | Twitter<https://twitter.com/smarttrade_tech> | Facebook<https://www.facebook.com/smartTradeTechnologies/>
The content of this e-mail (including any attachments) is strictly confidential and is for the sole use of the intended addressee(s). If you are not the intended recipient of this e-mail please notify the sender immediately and delete this e-mail from your system. Any disclosure, copying, dissemination or use of its content (including any attachments) is strictly prohibited.

_______________________________________________
Quickfixj-users mailing list
Qui...@li...<mailto:Qui...@li...>
https://lists.sourceforge.net/lists/listinfo/quickfixj-users

Re: [Quickfixj-users] Is there any plan to support Simple Binary Encoding (SBE)

[JinJin H. <jin...@gm...>]

Yes, FIXP is a session layer (OSI layer 5) protocol for better performance
compared to FIXT, there is already a reference implementation using SBE
(simple binary encoding) in
https://github.com/FIXTradingCommunity/silverflash

But it will be good if quickfixj (as an application layer implementation)
also supports SBE, like other commercial products as mentioned above.

This diagram (steal from web)  will be good to illustrate the above idea

[image: image.png]

Regards
JH


On Thu, 11 Jan 2024 at 21:20, Laurent Danesi <ld...@sm...>
wrote:

> QFJ Documentation: http://www.quickfixj.org/documentation/
> QFJ Support: http://www.quickfixj.org/support/
>
>
> Hi,
>
> I think it is related to the "new" FIXP (
> https://www.fixtrading.org/standards/fixp/) standard from the FIX Trading
> Community.
> The idea is to normalize a fast and binary (instead of text) protocol
> keeping the FIX semantic.
> It is inspired from soupTCP or other binary protocol tentative from some
> exchange.
> The control is very light but the protocol is very different than FIX4.x
> or FIX5 and Application messages are custom for each exchange: no way to
> use FIX messages and getField(xxx) as we do.
>
> The messages and codecs are generated from a sbe schema file.
>
> Regards,
> Lo
>
> Le jeu. 11 janv. 2024, 19:52, Colin DuPlantis <co...@ma...> a
> écrit :
>
>> QFJ Documentation: http://www.quickfixj.org/documentation/
>> QFJ Support: http://www.quickfixj.org/support/
>>
>>
>> Out of curiosity, what are SBEs in this context and what is the use case
>> for FIX supporting them?
>>
>> > On Jan 11, 2024, at 8:34 AM, Christoph John <chr...@ma...>
>> wrote:
>> >
>> > QFJ Documentation: http://www.quickfixj.org/documentation/
>> > QFJ Support: http://www.quickfixj.org/support/
>> >
>> >
>> > Hi,
>> >
>> > There are currently no such plans. Please keep in mind that we are no
>> commercial FIX engine but an open source one. 😊 So if anyone has added
>> this on top of QFJ they would be welcome to contribute it back to the
>> project.
>> > I could imagine that it is possible to implement it in some kind of
>> MINA filter on top of QFJ but I cannot say how much effort that will be.
>> > However, there is a reference implementation for SBE on github here:
>> https://github.com/real-logic/simple-binary-encoding
>> >
>> > Cheers
>> > Chris
>> >
>> > --
>> > Christoph John
>> > Software Engineering
>> >
>> > D +49 241 557080 28
>> > chr...@ma...
>> >
>> >
>> >
>> > MACD GmbH
>> > Oppenhoffallee 103
>> > 52066 Aachen, Germany
>> > www.macd.com
>> >
>> >
>> >
>> > Amtsgericht Aachen: HRB 8151
>> > Ust.-Id: DE 813021663
>> > Geschäftsführer: George Macdonald
>> >
>> >
>> >
>> > -----Original Message-----
>> > From: JinJin Huang <jin...@gm...>
>> > Sent: 11 January 2024 00:49
>> > To: qui...@li...
>> > Subject: [Quickfixj-users] Is there any plan to support Simple Binary
>> Encoding (SBE)
>> >
>> >
>> >
>> > Hi, Chris and Quickfixj experts
>> >
>> > Is there any plan to support SBE in your current or further development?
>> > Like other commercial FIX engines, such as Chronicle and Onixs.
>> >
>> > Kind Regards
>> > JH
>> >
>> > _______________________________________________
>> > Quickfixj-users mailing list
>> > Qui...@li...
>> > https://lists.sourceforge.net/lists/listinfo/quickfixj-users
>>
>>
>>
>> _______________________________________________
>> Quickfixj-users mailing list
>> Qui...@li...
>> https://lists.sourceforge.net/lists/listinfo/quickfixj-users
>>
>
> [image: https://smart-trade.net] <https://smart-trade.net>
> *Linkedin* <https://www.linkedin.com/company/smart-trade-technologies> |
> *Twitter* <https://twitter.com/smarttrade_tech> | *Facebook*
> <https://www.facebook.com/smartTradeTechnologies/>
> *The content of this e-mail (including any attachments) is strictly
> confidential and is for the sole use of the intended addressee(s). If you
> are not the intended recipient of this e-mail please notify the sender
> immediately and delete this e-mail from your system. Any disclosure,
> copying, dissemination or use of its content (including any attachments) is
> strictly prohibited.*
>
> _______________________________________________
> Quickfixj-users mailing list
> Qui...@li...
> https://lists.sourceforge.net/lists/listinfo/quickfixj-users
>

Re: [Quickfixj-users] Is there any plan to support Simple Binary Encoding (SBE)

[Colin D. <co...@ma...>]

Thanks very much!

> On Jan 11, 2024, at 12:17 PM, Laurent Danesi <ld...@sm...> wrote:
> 
> QFJ Documentation: http://www.quickfixj.org/documentation/
> QFJ Support: http://www.quickfixj.org/support/
> 
> 
> Hi,
> 
> I think it is related to the "new" FIXP (https://www.fixtrading.org/standards/fixp/) standard from the FIX Trading Community.
> The idea is to normalize a fast and binary (instead of text) protocol keeping the FIX semantic.
> It is inspired from soupTCP or other binary protocol tentative from some exchange.
> The control is very light but the protocol is very different than FIX4.x or FIX5 and Application messages are custom for each exchange: no way to use FIX messages and getField(xxx) as we do.
> 
> The messages and codecs are generated from a sbe schema file.
> 
> Regards,
> Lo
> 
> Le jeu. 11 janv. 2024, 19:52, Colin DuPlantis <co...@ma... <mailto:co...@ma...>> a écrit :
>> QFJ Documentation: http://www.quickfixj.org/documentation/
>> QFJ Support: http://www.quickfixj.org/support/
>> 
>> 
>> Out of curiosity, what are SBEs in this context and what is the use case for FIX supporting them?
>> 
>> > On Jan 11, 2024, at 8:34 AM, Christoph John <chr...@ma... <mailto:chr...@ma...>> wrote:
>> > 
>> > QFJ Documentation: http://www.quickfixj.org/documentation/
>> > QFJ Support: http://www.quickfixj.org/support/
>> > 
>> > 
>> > Hi,
>> > 
>> > There are currently no such plans. Please keep in mind that we are no commercial FIX engine but an open source one. 😊 So if anyone has added this on top of QFJ they would be welcome to contribute it back to the project.
>> > I could imagine that it is possible to implement it in some kind of MINA filter on top of QFJ but I cannot say how much effort that will be.
>> > However, there is a reference implementation for SBE on github here: https://github.com/real-logic/simple-binary-encoding
>> > 
>> > Cheers
>> > Chris
>> > 
>> > --
>> > Christoph John
>> > Software Engineering
>> > 
>> > D +49 241 557080 28
>> > chr...@ma... <mailto:chr...@ma...>
>> > 
>> > 
>> > 
>> > MACD GmbH
>> > Oppenhoffallee 103
>> > 52066 Aachen, Germany
>> > www.macd.com <http://www.macd.com/>
>> > 
>> > 
>> > 
>> > Amtsgericht Aachen: HRB 8151
>> > Ust.-Id: DE 813021663
>> > Geschäftsführer: George Macdonald
>> > 
>> > 
>> > 
>> > -----Original Message-----
>> > From: JinJin Huang <jin...@gm... <mailto:jin...@gm...>> 
>> > Sent: 11 January 2024 00:49
>> > To: qui...@li... <mailto:qui...@li...>
>> > Subject: [Quickfixj-users] Is there any plan to support Simple Binary Encoding (SBE)
>> > 
>> > 
>> > 
>> > Hi, Chris and Quickfixj experts
>> > 
>> > Is there any plan to support SBE in your current or further development?
>> > Like other commercial FIX engines, such as Chronicle and Onixs.
>> > 
>> > Kind Regards
>> > JH
>> > 
>> > _______________________________________________
>> > Quickfixj-users mailing list
>> > Qui...@li... <mailto:Qui...@li...>
>> > https://lists.sourceforge.net/lists/listinfo/quickfixj-users
>> 
>> 
>> 
>> _______________________________________________
>> Quickfixj-users mailing list
>> Qui...@li... <mailto:Qui...@li...>
>> https://lists.sourceforge.net/lists/listinfo/quickfixj-users
> 
>  <https://smart-trade.net/>
> Linkedin <https://www.linkedin.com/company/smart-trade-technologies> | Twitter <https://twitter.com/smarttrade_tech> | Facebook <https://www.facebook.com/smartTradeTechnologies/>
> The content of this e-mail (including any attachments) is strictly confidential and is for the sole use of the intended addressee(s). If you are not the intended recipient of this e-mail please notify the sender immediately and delete this e-mail from your system. Any disclosure, copying, dissemination or use of its content (including any attachments) is strictly prohibited.
> 
> _______________________________________________
> Quickfixj-users mailing list
> Qui...@li...
> https://lists.sourceforge.net/lists/listinfo/quickfixj-users

Re: [Quickfixj-users] Is there any plan to support Simple Binary Encoding (SBE)

[Laurent D. <ld...@sm...>]

Hi,

I think it is related to the "new" FIXP (
https://www.fixtrading.org/standards/fixp/) standard from the FIX Trading
Community.
The idea is to normalize a fast and binary (instead of text) protocol
keeping the FIX semantic.
It is inspired from soupTCP or other binary protocol tentative from some
exchange.
The control is very light but the protocol is very different than FIX4.x or
FIX5 and Application messages are custom for each exchange: no way to use
FIX messages and getField(xxx) as we do.

The messages and codecs are generated from a sbe schema file.

Regards,
Lo

Le jeu. 11 janv. 2024, 19:52, Colin DuPlantis <co...@ma...> a
écrit :

> QFJ Documentation: http://www.quickfixj.org/documentation/
> QFJ Support: http://www.quickfixj.org/support/
>
>
> Out of curiosity, what are SBEs in this context and what is the use case
> for FIX supporting them?
>
> > On Jan 11, 2024, at 8:34 AM, Christoph John <chr...@ma...>
> wrote:
> >
> > QFJ Documentation: http://www.quickfixj.org/documentation/
> > QFJ Support: http://www.quickfixj.org/support/
> >
> >
> > Hi,
> >
> > There are currently no such plans. Please keep in mind that we are no
> commercial FIX engine but an open source one. 😊 So if anyone has added
> this on top of QFJ they would be welcome to contribute it back to the
> project.
> > I could imagine that it is possible to implement it in some kind of MINA
> filter on top of QFJ but I cannot say how much effort that will be.
> > However, there is a reference implementation for SBE on github here:
> https://github.com/real-logic/simple-binary-encoding
> >
> > Cheers
> > Chris
> >
> > --
> > Christoph John
> > Software Engineering
> >
> > D +49 241 557080 28
> > chr...@ma...
> >
> >
> >
> > MACD GmbH
> > Oppenhoffallee 103
> > 52066 Aachen, Germany
> > www.macd.com
> >
> >
> >
> > Amtsgericht Aachen: HRB 8151
> > Ust.-Id: DE 813021663
> > Geschäftsführer: George Macdonald
> >
> >
> >
> > -----Original Message-----
> > From: JinJin Huang <jin...@gm...>
> > Sent: 11 January 2024 00:49
> > To: qui...@li...
> > Subject: [Quickfixj-users] Is there any plan to support Simple Binary
> Encoding (SBE)
> >
> >
> >
> > Hi, Chris and Quickfixj experts
> >
> > Is there any plan to support SBE in your current or further development?
> > Like other commercial FIX engines, such as Chronicle and Onixs.
> >
> > Kind Regards
> > JH
> >
> > _______________________________________________
> > Quickfixj-users mailing list
> > Qui...@li...
> > https://lists.sourceforge.net/lists/listinfo/quickfixj-users
>
>
>
> _______________________________________________
> Quickfixj-users mailing list
> Qui...@li...
> https://lists.sourceforge.net/lists/listinfo/quickfixj-users
>

-- 
 <https://smart-trade.net>

*Linkedin* 
<https://www.linkedin.com/company/smart-trade-technologies> | *Twitter* 
<https://twitter.com/smarttrade_tech> | *Facebook* 
<https://www.facebook.com/smartTradeTechnologies/>

_The content of this 
e-mail (including any attachments) is strictly confidential and is for the 
sole use of the intended addressee(s). If you are not the intended 
recipient of this e-mail please notify the sender immediately and delete 
this e-mail from your system. Any disclosure, copying, dissemination or use 
of its content (including any attachments) is strictly prohibited.
_
_
_

Re: [Quickfixj-users] Is there any plan to support Simple Binary Encoding (SBE)

[Colin D. <co...@ma...>]

Out of curiosity, what are SBEs in this context and what is the use case for FIX supporting them?

> On Jan 11, 2024, at 8:34 AM, Christoph John <chr...@ma...> wrote:
> 
> QFJ Documentation: http://www.quickfixj.org/documentation/
> QFJ Support: http://www.quickfixj.org/support/
> 
> 
> Hi,
> 
> There are currently no such plans. Please keep in mind that we are no commercial FIX engine but an open source one. 😊 So if anyone has added this on top of QFJ they would be welcome to contribute it back to the project.
> I could imagine that it is possible to implement it in some kind of MINA filter on top of QFJ but I cannot say how much effort that will be.
> However, there is a reference implementation for SBE on github here: https://github.com/real-logic/simple-binary-encoding
> 
> Cheers
> Chris
> 
> --
> Christoph John
> Software Engineering
> 
> D +49 241 557080 28
> chr...@ma...
> 
> 
> 
> MACD GmbH
> Oppenhoffallee 103
> 52066 Aachen, Germany
> www.macd.com
> 
> 
> 
> Amtsgericht Aachen: HRB 8151
> Ust.-Id: DE 813021663
> Geschäftsführer: George Macdonald
> 
> 
> 
> -----Original Message-----
> From: JinJin Huang <jin...@gm...> 
> Sent: 11 January 2024 00:49
> To: qui...@li...
> Subject: [Quickfixj-users] Is there any plan to support Simple Binary Encoding (SBE)
> 
> 
> 
> Hi, Chris and Quickfixj experts
> 
> Is there any plan to support SBE in your current or further development?
> Like other commercial FIX engines, such as Chronicle and Onixs.
> 
> Kind Regards
> JH
> 
> _______________________________________________
> Quickfixj-users mailing list
> Qui...@li...
> https://lists.sourceforge.net/lists/listinfo/quickfixj-users

Re: [Quickfixj-users] Is there any plan to support Simple Binary Encoding (SBE)

[Christoph J. <chr...@ma...>]

Hi,

There are currently no such plans. Please keep in mind that we are no commercial FIX engine but an open source one. 😊 So if anyone has added this on top of QFJ they would be welcome to contribute it back to the project.
I could imagine that it is possible to implement it in some kind of MINA filter on top of QFJ but I cannot say how much effort that will be.
However, there is a reference implementation for SBE on github here: https://github.com/real-logic/simple-binary-encoding

Cheers
Chris

--
Christoph John
Software Engineering

D +49 241 557080 28
chr...@ma...



MACD GmbH
Oppenhoffallee 103
52066 Aachen, Germany
www.macd.com

  

Amtsgericht Aachen: HRB 8151
Ust.-Id: DE 813021663
Geschäftsführer: George Macdonald



-----Original Message-----
From: JinJin Huang <jin...@gm...> 
Sent: 11 January 2024 00:49
To: qui...@li...
Subject: [Quickfixj-users] Is there any plan to support Simple Binary Encoding (SBE)



Hi, Chris and Quickfixj experts

Is there any plan to support SBE in your current or further development?
Like other commercial FIX engines, such as Chronicle and Onixs.

Kind Regards
JH

[Quickfixj-users] Is there any plan to support Simple Binary Encoding (SBE)

[JinJin H. <jin...@gm...>]

Hi, Chris and Quickfixj experts

Is there any plan to support SBE in your current or further development?
Like other commercial FIX engines, such as Chronicle and Onixs.

Kind Regards
JH

[Quickfixj-users] Does QFJ still need a distribution ZIP file?

[Christoph J. <chr...@ma...>]

https://github.com/quickfix-j/quickfixj/discussions/734


--
Christoph John
Software Engineering

D +49 241 557080 28
chr...@ma...<mailto:chr...@ma...>

[cid:image001.png@01DA2F51.71E96AB0]

MACD GmbH
Oppenhoffallee 103
52066 Aachen, Germany
www.macd.com<http://www.macd.com/>

[cid:image002.png@01DA2F51.71E96AB0]<https://www.linkedin.com/company/macd/>  [cid:image003.png@01DA2F51.71E96AB0] <https://www.xing.com/pages/macd>

Amtsgericht Aachen: HRB 8151
Ust.-Id: DE 813021663
Geschäftsführer: George Macdonald

Re: [Quickfixj-users] Missing BeginString tag on outbound Execution Reports

[Colin D. <co...@ma...>]

For my part, I wondered about adding headers programatically. Can you elaborate on why you do that?

> On Oct 19, 2023, at 3:14 AM, Christoph John <chr...@ma...> wrote:
> 
> QFJ Documentation: http://www.quickfixj.org/documentation/
> QFJ Support: http://www.quickfixj.org/support/
> 
> 
> Hi,
> 
> unrelated question: do you happen to use Outlook? If so, please send mails as plain text or HTML (NOT as rich text), otherwise they might get mangled by some servers. I must use Outlook and your mail gets split up in several attachments for some reason. Or maybe this is only a problem for me, and I don't know how to use Outlook right (coming from Thunderbird). 😊
> 
> 
> However, w.r.t. your problem: does this only occur for ExecReports but not for other application messages? Always on the same session?
> Why do you instantiate the message class instead of using a MessageFactory?
> How do you send the messages out? Via Session.sendToTarget()?
> 
> Cheers
> Chris
> 
> 
> -----Original Message-----
> From: Vlad Radu <vra...@gm...>
> Sent: 18 October 2023 16:35
> To: qui...@li...
> Subject: [Quickfixj-users] Missing BeginString tag on outbound Execution Reports
> 
> QFJ Documentation: http://www.quickfixj.org/documentation/
> QFJ Support: http://www.quickfixj.org/support/
> 
> In our production environment we've seen this occur sporadically for
> outbound Execution Reports on a particular FIX 4.4 session, using
> QuickFIX/J 2.3.1 .
> 
> As an example:
> 
> Good:
> 8=FIX.4.4|9=532|35=8|34=2766|49=SENDER|52=20231006-20:00:18.322|56=TARGET|116=TRADE|
> .... |10=114|
> 
> Bad:
> 9=558|35=8|34=2767|49=SENDER|52=20231006-20:00:18.329|56=TARGET|116=TRADE|
> .... |10=114|
> 
> I had to remove the body but left the headers. The messages are built using
> quickfix.Message class, adding following headers programmatically:
> MsgType,OnBehalfOfSubID , DeliverToCompID.
> 
> Unlike quickfix.fix44.Message prior class does not update the BeginString
> as normally this is done anyway by the Session before sending a message
> out. As can be seen in the above example all other headers are present
> (besides BeginString) and looking at the code they get initialized in the
> same method.
> 
> I can find no explanation for this occurring and cannot reproduce myself.
> Has anyone seen something like this occur?
> 
> _______________________________________________
> Quickfixj-users mailing list
> Qui...@li...
> https://lists.sourceforge.net/lists/listinfo/quickfixj-users