Re: [Quickfix-developers] Using .seqnums file to trigger a full resend
Brought to you by:
orenmnero
Menu
▾
▴
Re: [Quickfix-developers] Using .seqnums file to trigger a full resend
|
From: Mike G. <mg...@co...> - 2017-02-28 16:39:06
|
Dermot, Thanks for the accurate steps to repro. I just tried it, and the problem turns out to be quite simple: you are resetting the target seqnum to zero instead of resetting it to 1 -- something that quickfix itself would never do. When the executor sends a sequence reset message under MsgSeqNum=1, the tradeclient rejects it because it is expecting MsgSeqNum=0, which is an impossible condition. So, long story short, just set your seqnum file (or database) to "0000000004 : 0000000001" instead of "0000000004 : 0000000000" and it should behave like you wanted. -- Mike Gatny Connamara Systems, LLC On Tue, Feb 28, 2017 at 10:11 AM, <daw...@ya...> wrote: > Hi Mike, > > Yes I've reproduced it with 'tradeclient'<->'executor' both with no > 'RefreshOnLogon' entry and with RefreshOnLogon=Y. > > The steps are:- > 1. Log on > 2. Send order > 3. Receive execution > 4. Exit program > 5. Manually change the .seqnums file from "0000000004 : 0000000004" to > "0000000004 : 0000000000" > 6. Log on (resend is triggered) > 7. Exit program > 8. Check the seqnums file again and you get "0000000007 : 0000000001" when > it should be "0000000007 : 0000000006" > > Actually I'm using MySql for the store and logs so it suits me better to > just update using SQL. > > Yes I'm using quickfix 1.14.3 C++ version. > > Thanks, > Dermot > > > ----- Original Message ----- > *From:* Mike Gatny <mg...@co...> > *To:* daw...@ya... > *Cc:* Grant Birchmeier <gbi...@co...>; " > qui...@li..." <quickfix-developers@lists. > sourceforge.net> > *Date:* 2017/2/28, Tue 22:39 > *Subject:* Re: [Quickfix-developers] Using .seqnums file to trigger a > full resend > > Dermot, > > Are you able to reproduce your original seqnums-file-not-updating issue > using e.g. the 'tradeclient' and 'executor' example apps, with > RefreshOnLogon=Y? I'd like to rule out the weirdness of the resends they > are sending you as a factor. > > I don't see a problem with what you are trying to do -- I have used this > technique many times when building systems that must failover to a remote > site. But as I mentioned, I've only ever done it by completely shutting > down the app while editing the file. > > Also, if it fits into your architecture, do consider trying the > HttpAcceptPort setting, and using the http interface to edit your seqnums. > > Finally, are you using the latest version of quickfix (1.14.3)? > > -- > Mike Gatny > Connamara Systems, LLC > > On Tue, Feb 28, 2017 at 8:23 AM, <daw...@ya...> wrote: > > Hi again Mike, I spoke to the broker who gave me the following:- > > 1. They don't have any special procedure for sending historical messages > and recommend using the regular resend (i.e. they don't truncate > messages). > 2. They agree there should not be any admin messages included - this is a > bug and they are looking into it. > > So this means I'll have to go with the contrived resend via the seqnums > file. And this brings me all the way back to my original post regarding the > seqnums file not containing the correct incoming seq no after resend. Do > you have any inclination to take a look at this and perhaps "fix" it? I'm > guessing that's a No given you don't really recommend this approach anyway > so in that case what are my options? I guess I'll just have to manually > update the seqnums file again with the correct number after resend. > Thoughts? > > Thanks, > Dermot > > > > ----- Original Message ----- > *From:* Mike Gatny <mg...@co...> > *To:* daw...@ya... > *Cc:* Grant Birchmeier <gbi...@co...>; "quickfix-developers@lists. > sourceforge.net <qui...@li...>" <quickfix-developers@lists. > sourceforge.net <qui...@li...>> > *Date:* 2017/2/25, Sat 03:05 > *Subject:* Re: [Quickfix-developers] Using .seqnums file to trigger a > full resend > > On Fri, Feb 24, 2017 at 12:21 PM, <daw...@ya...> wrote: > > Ok noted, thanks - don't send my own resend requests. I'm still waiting to > hear back from the broker re: recommendations for getting a full day's > messages resent. Their support is not great to be honest. > > In the meantime I tried editing the seqnums file and that worked but with > some problems:- > > 1. The resend included all heartbeats as well. Is there a way to specify > not to include heartbeats/test requests? > 2. The resend included the broker's response to the originallogout > message. This in turn triggers a new logout on my side. Is this normal > behavior? Is there a way to exclude this message also? Basically I'm only > interested in messages relating to orders/executions. > > > They should not be sending you any admin-type messages > (htbeat/testrequest/logon/ logout) at all during a resend. Instead they > should be replacing those msgs with sequence reset msgs with GapFillFlag > field set to "Y". That's just totally broken resend behavior I'm afraid > -- broken enough that I don't see what we can do on your side to mitigate > it. And it could trip you up even if you aren't doing a contrived resend > via the seqnums file. > > > -- > Mike Gatny > Connamara Systems, LLC > > > > > Thanks, > Dermot > > ----- Original Message ----- > *From:* Mike Gatny <mg...@co...> > *To:* daw...@ya... > *Cc:* Grant Birchmeier <gbi...@co...>; "quickfix-developers@lists. > sourceforge.net <qui...@li...>" <quickfix-developers@lists. > sourceforge.net <qui...@li...>> > *Date:* 2017/2/24, Fri 01:41 > *Subject:* Re: [Quickfix-developers] Using .seqnums file to trigger a > full resend > > Definitely don't send your own resend requests (or any other admin msg) -- > that will mess with the QF state in unsafe ways. > Editing the seqnums file is definitely safe if you do it while your app is > not running. And it should be safe if you are running but logged out, and > RefreshOnLogon=Y. > Also note that you can use QF's http interface to reset seqnums. > > -- > Mike Gatny > Connamara Systems, LLC > > On Thu, Feb 23, 2017 at 11:29 AM, <daw...@ya...> wrote: > > Hi Mike, Grant, > > Ok thanks for your feedback. Let me see what the broker comes back with > first. There's nothing specific in their ROE regarding history-retrieval. > > As a test I did try sending a resend request message > [BeginSeqNo=0,EndSeqNo=0] but my side then immediately logged out for some > reason. > > Dermot > > ----- Original Message ----- > *From:* Mike Gatny <mg...@co...> > *To:* daw...@ya... > *Cc:* Grant Birchmeier <gbi...@co...>; "quickfix-developers@lists. > sourceforge.net <qui...@li...>" <quickfix-developers@lists. > sourceforge.net <qui...@li...>> > *Date:* 2017/2/24, Fri 00:50 > *Subject:* Re: [Quickfix-developers] Using .seqnums file to trigger a > full resend > > Fiddling with the seqnum file should definitely work if you completely > shut down your app, edit the file, then restart your app. > To do it with just a logout/edit/logon, try RefreshOnLogon=Y. > > -- > Mike Gatny > Connamara Systems, LLC > > On Thu, Feb 23, 2017 at 12:24 AM, <daw...@ya...> wrote: > > QuickFIX Documentation: http://www.quickfixengine.org/ quickfix/doc/html/ > <http://www.quickfixengine.org/quickfix/doc/html/> > > > Hi Grant, > > Yes it sounded a bit 'hacky' to me too but this was the approach > recommended to me by Asher Newcomer on this forum last October. Ok I will > speak to the broker and ask what they recommend. > > Thanks, > Dermot > > ----- Original Message ----- > *From:* Grant Birchmeier <gbi...@co...> > *To:* daw...@ya... > *Cc:* "quickfix-developers@lists. sourceforge.net > <qui...@li...>" <quickfix-developers@lists. > sourceforge.net <qui...@li...>> > *Date:* 2017/2/23, Thu 13:07 > *Subject:* Re: [Quickfix-developers] Using .seqnums file to trigger a > full resend > > I would call this a bad practice. (You're not the first to invent it.) > > You're hacking your app's state to basically lie to your counterparty > about missing messages. If you are asking for a *lot* of history, they'll > probably truncate it. > > Have you checked your counterparty's docs to see if they provide another > way to get the historical data that you need? Surely you aren't the first > person to need historical data from this system. > > On Wed, Feb 22, 2017 at 9:29 PM, <daw...@ya...> wrote: > > QuickFIX Documentation: http://www.quickfixengine.org/ quickfix/doc/html/ > <http://www.quickfixengine.org/quickfix/doc/html/> > > > Hi Guys, > > I'm trying to trigger a resend of all the executor's messages for the day. I > manually set the incoming seq no in the .seqnums file to 0, then login > and this triggers a resend request for all messages as soon as my FIX > engine receives the logon response message - all good so far. All the > messages are resent and received correctly. The problem is when I log out I > can see that the .seqnums file does not have the updated incoming seq no > (it's just set to 1) so when I log in again it triggers another resend > request. > > This problem should be very easy to reproduce. The steps are:- > 1. Log on > 2. Send order > 3. Receive execution > 4. Log out > 5. Manually change the .seqnums file from "0000000004 : 0000000004" to > "0000000004 : 0000000000" > 6. Log on (resend is triggered) > 7. Log out. > 8. Check the seqnums file again and you get "0000000007 : 0000000001" when > it should be "0000000007 : 0000000006" > > Any ideas? > > Thanks a lot! > Dermot > > ------------------------------ ------------------------------ > ------------------ > Check out the vibrant tech community on one of the world's most > engaging tech sites, SlashDot.org! http://sdm.link/slashdot > ______________________________ _________________ > Quickfix-developers mailing list > Quickfix-developers@lists. sourceforge.net > <Qui...@li...> > https://lists.sourceforge.net/ lists/listinfo/quickfix- developers > <https://lists.sourceforge.net/lists/listinfo/quickfix-developers> > > > > > -- > Grant Birchmeier > *Connamara Systems, LLC* > *Made-To-Measure Trading Solutions.* > Exactly what you need. No more. No less. > http://connamara.com > > > > ------------------------------ ------------------------------ > ------------------ > Check out the vibrant tech community on one of the world's most > engaging tech sites, SlashDot.org! http://sdm.link/slashdot > ______________________________ _________________ > Quickfix-developers mailing list > Quickfix-developers@lists. sourceforge.net > <Qui...@li...> > https://lists.sourceforge.net/ lists/listinfo/quickfix- developers > <https://lists.sourceforge.net/lists/listinfo/quickfix-developers> > > > > > > > > > > > > > > |
