quickfix-developers

[Quickfix-developers] Quickfix acceptance tests.

[Timothy Y. <ty...@pa...>]

The automated acceptance tests for quickfix are impressive but have some
very significant holes.

For example, the tests surrounding processing of resend requests do not
cover the most common resend scenario, namely loss of synchronization after
an unexpected loss of the TCP connection.   All the acceptance tests assume
that sequence numbers are set to 1 at every logon.  However, it is usually
at logon that the missing messages are detected.

One shouldn't, therefore, shouldn't get too complacent about the robustness
of the quickfix session level code, especially when it is talking to a buggy
peer fix app (e.g. one that often crashes), or over an unreliable network
connection.

Tim Yates
Lead Developer 
Patsystems (US) LLC 
141 West Jackson Boulevard
Chicago 60604, USA 
Tel +1 (312) 542-1336 
www.patsystems.com 


DISCLAIMER: This e-mail is confidential and may also be legally privileged.
If you are not the intended recipient, use of the information contained in
this e-mail (including disclosure, copying or distribution) is prohibited
and may be unlawful.  Please inform the sender and delete the message
immediately from your system.  This e-mail is attributed to the sender and
may not necessarily reflect the views of the Patsystems Group and no member
of the Patsystems Group accepts any liability for any action taken in
reliance on the contents of this e-mail (other than where it has a legal or
regulatory obligation to do so) or for the consequences of any computer
viruses which may have been transmitted by this e-mail. The Patsystems Group
comprises Patsystems plc and its subsidiary group of companies.

Re: [Quickfix-developers] Quickfix acceptance tests.

[Joerg T. <Joe...@ma...>]

Hi Tim,

thanks for the tests already provided. I did some initial reading 
whether the message should be ignored or rejected, but did not come to 
any conclusion yet.

> The automated acceptance tests for quickfix are impressive but have some
> very significant holes.
> 
> For example, the tests surrounding processing of resend requests do not
> cover the most common resend scenario, namely loss of synchronization after
> an unexpected loss of the TCP connection.   All the acceptance tests assume
> that sequence numbers are set to 1 at every logon.  However, it is usually
> at logon that the missing messages are detected.

Agreed. Since we use QuickFIX  in production, we see oft such kind of 
re-syncs. Probably I could grab some tests from production issues. But 
this kind of stuff is always difficult to analyse.

> One shouldn't, therefore, shouldn't get too complacent about the robustness
> of the quickfix session level code, especially when it is talking to a buggy
> peer fix app (e.g. one that often crashes), or over an unreliable network
> connection.

Actually, it is difficult to simulate an unreliable network connection. 
There are TCP/IP level tools to simulate that (nistnet), but I did not 
do any experiments so far. Perhaps it is possible to implement a "lossy"
low level socket read/write interface to be plugged in.

Cheers, Jörg

-- 
Joerg Thoennes
                                http://macd.com
Tel.: +49 (0)241 44597-24      Macdonald Associates GmbH
Fax : +49 (0)241 44597-10      Lothringer Str. 52, D-52070 Aachen