quickfix-developers

[Quickfix-developers] What should be done if ResendRequest generated by doTargetTooHigh have never been received or acted upon

[len <le...@fa...>]

Hi,
 
We recently encounter the situation when remote FIX engine seems to ignore
ResendRequest message generated by our QuickFix engine. Unfortunately that
ultimately blocked our running QuickFix engine because remote party was keep
sending messages with higher and higher sequence numbers and QuickFix simply
queue the message and just log ""Already sent ResendRequest FROM:.".
Therefore Application code was not aware about problem and QuickFix did not
try to resolve it.
 
Did anyone have experience with it? What solution should be? I am inclining
to add some time-out check (basically add time when resendRequest was sent
into m_state and resend the message after certain period of time). May be
upcoming QuickFix releases address this problem?
 
Thank you.

Re: [Quickfix-developers] What should be done if ResendRequest generated by doTargetTooHigh have never been received or acted upon

[Joerg T. <Joe...@ma...>]

> We recently encounter the situation when remote FIX engine seems to ign=
ore
> ResendRequest message generated by our QuickFix engine. Unfortunately t=
hat
> ultimately blocked our running QuickFix engine because remote party was=
 keep
> sending messages with higher and higher sequence numbers and QuickFix s=
imply
> queue the message and just log ""Already sent ResendRequest FROM:.".
> Therefore Application code was not aware about problem and QuickFix did=
 not
> try to resolve it.
> =20
> Did anyone have experience with it? What solution should be? I am incli=
ning
> to add some time-out check (basically add time when resendRequest was s=
ent
> into m_state and resend the message after certain period of time). May =
be
> upcoming QuickFix releases address this problem?

Hi Len,

reading the discussion in the FIX Protocol forums, I would like to suppor=
t John Prewetts suggestion
to force a logout (or disconnect) after a time-out. Oren, do you think it=
 is worth to add
configuration option as "ResendTimeout" to support this?

Cheers, J=F6rg

--=20
Joerg Thoennes
                                http://macd.com
Tel.: +49 (0)241 44597-24      Macdonald Associates GmbH
Fax : +49 (0)241 44597-10      Lothringer Str. 52, D-52070 Aachen

Re: [Quickfix-developers] What should be done if ResendRequest generated by doTargetTooHigh have never been received or acted upon

[len <le...@fa...>]

Hi J=F6rg,

Because Oren was not on the Cc list, can you please keep me in the loop
about my question future discussion?=20

Thanks again.

Len.

-----Original Message-----
From: Joerg Thoennes [mailto:Joe...@ma...]=20
Sent: Monday, August 21, 2006 3:42 PM
To: len
Cc: qui...@li...
Subject: Re: [Quickfix-developers] What should be done if ResendRequest
generated by doTargetTooHigh have never been received or acted upon

> We recently encounter the situation when remote FIX engine seems to =
ignore
> ResendRequest message generated by our QuickFix engine. Unfortunately =
that
> ultimately blocked our running QuickFix engine because remote party =
was
keep
> sending messages with higher and higher sequence numbers and QuickFix
simply
> queue the message and just log ""Already sent ResendRequest FROM:.".
> Therefore Application code was not aware about problem and QuickFix =
did
not
> try to resolve it.
> =20
> Did anyone have experience with it? What solution should be? I am
inclining
> to add some time-out check (basically add time when resendRequest was =
sent
> into m_state and resend the message after certain period of time). May =
be
> upcoming QuickFix releases address this problem?

Hi Len,

reading the discussion in the FIX Protocol forums, I would like to =
support
John Prewetts suggestion
to force a logout (or disconnect) after a time-out. Oren, do you think =
it is
worth to add
configuration option as "ResendTimeout" to support this?

Cheers, J=F6rg

--=20
Joerg Thoennes
                                http://macd.com
Tel.: +49 (0)241 44597-24      Macdonald Associates GmbH
Fax : +49 (0)241 44597-10      Lothringer Str. 52, D-52070 Aachen

Re: [Quickfix-developers] What should be done if ResendRequest generated by doTargetTooHigh have never been received or acted upon

[Caleb E. <cal...@gm...>]

On 8/21/06, Joerg Thoennes <Joe...@ma...> wrote:
> > We recently encounter the situation when remote FIX engine seems to ignore
> > ResendRequest message generated by our QuickFix engine. Unfortunately that
> > ultimately blocked our running QuickFix engine because remote party was keep
> > sending messages with higher and higher sequence numbers and QuickFix simply
> > queue the message and just log ""Already sent ResendRequest FROM:.".
> > Therefore Application code was not aware about problem and QuickFix did not
> > try to resolve it.
> >
> > Did anyone have experience with it? What solution should be? I am inclining
> > to add some time-out check (basically add time when resendRequest was sent
> > into m_state and resend the message after certain period of time). May be
> > upcoming QuickFix releases address this problem?
>
> reading the discussion in the FIX Protocol forums, I would like to support John Prewetts suggestion
> to force a logout (or disconnect) after a time-out. Oren, do you think it is worth to add
> configuration option as "ResendTimeout" to support this?

What version of QF are people testing this behavior with?  I think the
very latest release 1.12.2 (and anything built from the SVN sources
after 4/29/06 (r1437)) will timeout the connection when this happens
and no new code or configuration should be necessary to support it.
It would be great if someone could test this.

There *was* a bug where the Session's lastReceived timestamp was being
updated before the sequence number checking was done, which caused
test requests never to go out and the connection to stay alive in this
scenario.  this bug was fixed in April, but there haven't been any
QuickFIX releases since then ((until last week).

See my email about this from July 20.

http://sourceforge.net/mailarchive/message.php?msg_id=32260466

-- 
Caleb Epstein

Re: [Quickfix-developers] What should be done if ResendRequest generated by doTargetTooHigh have never been received or acted upon

[len <le...@fa...>]

Thanks everybody for the very quick response. 

The effect I have described occurs in version 1.11.x. I have already
downloaded latest source 1.12.2 and very quick check reveal addition of
Session level "m_sendRedundantResendRequests" variable which can be set by
Application level code which control reissue of already sent ResendRequest,
which is probably enough for my objectives - what can I say, stupid me, I
have to do it first before arrogantly asking the question.

Interestingly enough Caleb refers for time-out behavior which I did not
discover yet by surfing through the code. I am planning to test it though
but it will take me a week or two because I am on the other project and I
will need to back-port our QuickFix sources changes (compiling it into DLL
and dozen of other little things)

I will post my findings as soon as I can.

Thank you guys.

Len.

-----Original Message-----
From: Caleb Epstein [mailto:cal...@gm...] 
Sent: Monday, August 21, 2006 4:26 PM
To: Joerg Thoennes
Cc: len; qui...@li...
Subject: Re: [Quickfix-developers] What should be done if ResendRequest
generated by doTargetTooHigh have never been received or acted upon

On 8/21/06, Joerg Thoennes <Joe...@ma...> wrote:
> > We recently encounter the situation when remote FIX engine seems to
ignore
> > ResendRequest message generated by our QuickFix engine. Unfortunately
that
> > ultimately blocked our running QuickFix engine because remote party was
keep
> > sending messages with higher and higher sequence numbers and QuickFix
simply
> > queue the message and just log ""Already sent ResendRequest FROM:.".
> > Therefore Application code was not aware about problem and QuickFix did
not
> > try to resolve it.
> >
> > Did anyone have experience with it? What solution should be? I am
inclining
> > to add some time-out check (basically add time when resendRequest was
sent
> > into m_state and resend the message after certain period of time). May
be
> > upcoming QuickFix releases address this problem?
>
> reading the discussion in the FIX Protocol forums, I would like to support
John Prewetts suggestion
> to force a logout (or disconnect) after a time-out. Oren, do you think it
is worth to add
> configuration option as "ResendTimeout" to support this?

What version of QF are people testing this behavior with?  I think the
very latest release 1.12.2 (and anything built from the SVN sources
after 4/29/06 (r1437)) will timeout the connection when this happens
and no new code or configuration should be necessary to support it.
It would be great if someone could test this.

There *was* a bug where the Session's lastReceived timestamp was being
updated before the sequence number checking was done, which caused
test requests never to go out and the connection to stay alive in this
scenario.  this bug was fixed in April, but there haven't been any
QuickFIX releases since then ((until last week).

See my email about this from July 20.

http://sourceforge.net/mailarchive/message.php?msg_id=32260466

-- 
Caleb Epstein

Re: [Quickfix-developers] What should be done if ResendRequest generated by doTargetTooHigh have never been received or acted upon

[Caleb E. <cal...@gm...>]

On 8/21/06, len <le...@fa...> wrote:

> The effect I have described occurs in version 1.11.x. I have already
> downloaded latest source 1.12.2 and very quick check reveal addition of
> Session level "m_sendRedundantResendRequests" variable which can be set by
> Application level code which control reissue of already sent ResendRequest,
> which is probably enough for my objectives - what can I say, stupid me, I
> have to do it first before arrogantly asking the question.

It doesn't seem like issuing more ResendRequests would help with this
counterparty.

> Interestingly enough Caleb refers for time-out behavior which I did not
> discover yet by surfing through the code. I am planning to test it though

This is the standard FIX heartbeating mechanism.  If no messages are
received (in proper sequence) in <HeartbeatInterval> seconds, a
heartbeat message will be sent.  If still no messages (in proper
sequence) are received after 1.2 * <HeartbeatInterval>, a Test request
is sent, and the connection will eventually timeout if no messages are
received for 2.4 * HeartbeatInterval.  Look at the code in
SessionState.h and Session::next()

> but it will take me a week or two because I am on the other project and I
> will need to back-port our QuickFix sources changes (compiling it into DLL
> and dozen of other little things)

Perhaps you could expoound on what changes you've made to the QuickFIX
sources.  If they are beneficial to others, maybe you can have them
incorporated into the upstream codebase to reduce the need for this
sort of integration.

QuickFIX already builds as a DLL doesn't it?

-- 
Caleb Epstein

Re: [Quickfix-developers] What should be done if ResendRequest generated by doTargetTooHigh have never been received or acted upon

[len <le...@fa...>]

Caleb,

Thanks for insightful reply. My response is below

On 8/21/06, Caleb Epstein wrote:

>> The effect I have described occurs in version 1.11.x. I have already
>> downloaded latest source 1.12.2 and very quick check reveal addition of
>> Session level "m_sendRedundantResendRequests" variable which can be set
by
>> Application level code which control reissue of already sent
ResendRequest,
>> which is probably enough for my objectives - what can I say, stupid me, I
>> have to do it first before arrogantly asking the question.

>It doesn't seem like issuing more ResendRequests would help with this
>counterparty.

I believe that ResentRequest message was just simply swallowed somewhere
after network glitch and accordingly second re-send would make the
difference in our particular situation. Generally speaking when you
explicitly elaborated on timeouts when no messages in the PROPER sequence
are arriving - I like it a lot and I am going to test it really soon.

>Perhaps you could expoound on what changes you've made to the QuickFIX
>sources.  If they are beneficial to others, maybe you can have them
>incorporated into the upstream codebase to reduce the need for this
>sort of integration.

>QuickFIX already builds as a DLL doesn't it?
No, QuickFIX is not a DLL in the Windows project. It is a statically linked
library and there nothing wrong with it except that you will have a memory
overhead when you run more than one executable linked to it. It was not in
version 1.11 and I have checked 1.12.2 it is still not the case.

Besides some trivial configuration changes (debug symbols, locations) and
converting it to DLL (small config changes and I had to add MACRO expanding
to __declspec(dllexport) in few key classes to make them exportable) I have
made following changes to the project:

1. Add resource file (xxxx.RC) which includes QuickFIX version - that
allowed under Windows right-click on the quicfix.dll and see its version via
standard property dialog
2. Made bunch of changes which allowed me dynamically config
session/sessions. May be because of my in-expirience with QuickFIX engine
but I've got impression that its life-cycle is too straight-forward for our
needs: Global Application (and its infrastructure such as log, message
store, ThreadedConnection) is loaded and configured only once during
executable host startup. My modification allowed me to do it at any time
dynamically. In addition all our configuration details are stored in DB and
not init file - client code reads config from DB and (re)configure QuickFIX
application. But again there may be a way to achieve it in other way - I
just was not able to find it.
3. Moved Message's validate() method from private to public section -
because it was a beneficial to validate message prior to posting it.
4. Added two more types of logon. One just to check remote sequence number
and disconnect and second to realign local sequence number and continue
connection - this for pure testing purposes.
5. Added a client control methods to simulate loss of incoming or/and
outgoing messages - as well for testing only.

If you want me a share code I also can do it very easily.

Thanks.

Len.

-----Original Message-----
From: Caleb Epstein [mailto:cal...@gm...] 
Sent: Monday, August 21, 2006 4:45 PM
To: len
Cc: Joerg Thoennes; qui...@li...
Subject: Re: [Quickfix-developers] What should be done if ResendRequest
generated by doTargetTooHigh have never been received or acted upon

On 8/21/06, len <le...@fa...> wrote:

> The effect I have described occurs in version 1.11.x. I have already
> downloaded latest source 1.12.2 and very quick check reveal addition of
> Session level "m_sendRedundantResendRequests" variable which can be set by
> Application level code which control reissue of already sent
ResendRequest,
> which is probably enough for my objectives - what can I say, stupid me, I
> have to do it first before arrogantly asking the question.

It doesn't seem like issuing more ResendRequests would help with this
counterparty.

> Interestingly enough Caleb refers for time-out behavior which I did not
> discover yet by surfing through the code. I am planning to test it though

This is the standard FIX heartbeating mechanism.  If no messages are
received (in proper sequence) in <HeartbeatInterval> seconds, a
heartbeat message will be sent.  If still no messages (in proper
sequence) are received after 1.2 * <HeartbeatInterval>, a Test request
is sent, and the connection will eventually timeout if no messages are
received for 2.4 * HeartbeatInterval.  Look at the code in
SessionState.h and Session::next()

> but it will take me a week or two because I am on the other project and I
> will need to back-port our QuickFix sources changes (compiling it into DLL
> and dozen of other little things)

Perhaps you could expoound on what changes you've made to the QuickFIX
sources.  If they are beneficial to others, maybe you can have them
incorporated into the upstream codebase to reduce the need for this
sort of integration.

QuickFIX already builds as a DLL doesn't it?

-- 
Caleb Epstein

Re: [Quickfix-developers] What should be done if ResendRequest generated by doTargetTooHigh have never been received or acted upon

[len <le...@fa...>]

Hi,

I have tested 1.12.2 in regards of the bug when Session was not timing =
out
when no messages in the right order were coming (although messages with =
too
high sequence number kept coming) 1.11.1 version in such condition stuck =
in
the loop insisting that it already sent ResendRequest.

In regards of testing: I have simulated the same conditions and it DID =
work
as advertised. After two-something heartbeats the session was =
automatically
logged of with event "Timed out waiting for heartbeat". Thank you guys =
you
have saved me from putting in na=EFve hack.

By the way I also have posted a question about delayed logon and pending
connections under different name - I have a hard time to reply to this =
news
group - even though I have registered with it.

Len.

-----Original Message-----
From: Caleb Epstein [mailto:cal...@gm...]=20
Sent: Monday, August 21, 2006 4:26 PM
To: Joerg Thoennes
Cc: len; qui...@li...
Subject: Re: [Quickfix-developers] What should be done if ResendRequest
generated by doTargetTooHigh have never been received or acted upon

On 8/21/06, Joerg Thoennes <Joe...@ma...> wrote:
> > We recently encounter the situation when remote FIX engine seems to
ignore
> > ResendRequest message generated by our QuickFix engine. =
Unfortunately
that
> > ultimately blocked our running QuickFix engine because remote party =
was
keep
> > sending messages with higher and higher sequence numbers and =
QuickFix
simply
> > queue the message and just log ""Already sent ResendRequest FROM:.".
> > Therefore Application code was not aware about problem and QuickFix =
did
not
> > try to resolve it.
> >
> > Did anyone have experience with it? What solution should be? I am
inclining
> > to add some time-out check (basically add time when resendRequest =
was
sent
> > into m_state and resend the message after certain period of time). =
May
be
> > upcoming QuickFix releases address this problem?
>
> reading the discussion in the FIX Protocol forums, I would like to =
support
John Prewetts suggestion
> to force a logout (or disconnect) after a time-out. Oren, do you think =
it
is worth to add
> configuration option as "ResendTimeout" to support this?

What version of QF are people testing this behavior with?  I think the
very latest release 1.12.2 (and anything built from the SVN sources
after 4/29/06 (r1437)) will timeout the connection when this happens
and no new code or configuration should be necessary to support it.
It would be great if someone could test this.

There *was* a bug where the Session's lastReceived timestamp was being
updated before the sequence number checking was done, which caused
test requests never to go out and the connection to stay alive in this
scenario.  this bug was fixed in April, but there haven't been any
QuickFIX releases since then ((until last week).

See my email about this from July 20.

http://sourceforge.net/mailarchive/message.php?msg_id=3D32260466

--=20
Caleb Epstein

Re: [Quickfix-developers] What should be done if ResendRequestgenerated by doTargetTooHigh have never been received or acted upon

[Sean K. <sea...@pi...>]

Len,

We have experienced the same issue with 1.11.x versions of the code.  I =
believe there is a change in logic to not update the "last time =
received" if this error condition is encountered in 1.12.x which should =
cause the test request logic to kick in.  I haven't been able to test it =
yet.

--Sean

> -----Original Message-----
> From: qui...@li...
> [mailto:qui...@li...]On Behalf Of
> len
> Sent: Monday, August 21, 2006 1:36 PM
> To: qui...@li...
> Subject: [Quickfix-developers] What should be done if
> ResendRequestgenerated by doTargetTooHigh have never been received or
> acted upon
>=20
>=20
> QuickFIX Documentation:=20
> http://www.quickfixengine.org/quickfix/doc/html/index.html
> QuickFIX Support: http://www.quickfixengine.org/services.html
>=20
>=20


Disclaimer: Any references to Pipeline performance contained herein are =
based on historic performance levels which Pipeline expects to maintain =
or exceed but nevertheless does not guarantee. Congested networks, price =
volatility, or other extraordinary events may impede future trading =
activities and degrade performance statistics.