Re: [Quickfixj-users] Seeking Advice: Handling Message Processing Errors and Pausing FIX Messages in QuickFIX/J

[Christoph J. <chr...@ma...>]

Hi Felipe,

I think the most natural and built in way would be to throw the exception which will not increment the seqnum and hence on the next incoming message a gap will be detected. You could optionally log off there if you want to sort out the issue first.

Directly putting the message onto a queue and/or persisting it outside of QFJ is another option. It would also ensure that you spend as little time as possible in the fromApp callback, minimising blocking.

Cheers
Chris




From: Felipe Windmoller <fel...@gm...>
Sent: Sunday, March 17, 2024 2:24:53 PM
To: qui...@li... <qui...@li...>
Subject: [Quickfixj-users] Seeking Advice: Handling Message Processing Errors and Pausing FIX Messages in QuickFIX/J


Dear QuickFIX/J community,

I am seeking advice on how to manage messages from a drop copy session.
Each business message must be processed by a separate application, and I
need to ensure that every message is processed without fail.

I am considering how to handle scenarios where, during the execution of the
fromApp() method, calling this external application results in an error. Is
it feasible to pause the processing of my FIX messages until the external
call is successful? In such a scenario, the pending messages would
accumulate. Would this approach be advisable? Additionally, I am curious
about the mechanisms available for halting QFJ processing in response to an
error encountered within the fromApp() method. Specifically, how can I
effectively stop the processing of messages by QFJ at this juncture? Would
it involve rejecting the problematic FIX message, thereby necessitating its
resending by the FIX counterparty?

I have contemplated the following strategies:

   1.

   1. Utilizing the QFJ Incoming Log:

   My idea is to configure the QFJ application to solely receive and log
   FIX messages. A separate application would then periodically (every 1
   minute) check this log, guided by a control table that tracks the last
   sequence number processed. It would then process the messages accordingly.
   Point of Attention:

   As Christoph John has elucidated on StackOverflow
   <https://stackoverflow.com/a/78167504/11110455>, *"QFJ logs messages
   upon receipt at AbstractIoHandler.java
   <https://github.com/quickfix-j/quickfixj/blob/d7c9fe52fabdc56f28ace4063f6ac78fe422b511/quickfixj-core/src/main/java/quickfix/mina/AbstractIoHandler.java#L142>.
   Message parsing and subsequent processing occur later; only then is the
   message conveyed to the Session class where it undergoes validation,
   leading to a potential call to fromApp() if validation is successful."*
   Consequently, there's an inherent uncertainty about processing solely valid
   messages.

   2.


   3.

   2. Employing a Message Broker:

   Another approach is to leverage a message broker as an intermediary in
   the fromApp() method to queue the received messages. This would
   essentially serve as a buffer, allowing another application to process
   these messages asynchronously.
   Point of Attention:

   A critical consideration is the strategy to adopt if the message broker
   becomes unavailable during the execution of a fromApp() method call.

   4.


   5.

   3. Storing Pending Messages in a Database Table:

   Similar to using a message broker, this method involves storing the
   messages received in the fromApp() method in a database table. This
   table would then act as a buffer until the messages can be processed.


I greatly appreciate your insights and suggestions on these approaches or
any other potential solutions you might recommend.

Thank you, Felipe Windmoller