Re: [Quickfix-developers] Using .seqnums file to trigger a full resend
Brought to you by:
orenmnero
Menu
▾
▴
Re: [Quickfix-developers] Using .seqnums file to trigger a full resend
|
From: <daw...@ya...> - 2017-02-28 15:11:30
|
Hi Mike, Yes I've reproduced it with 'tradeclient'<->'executor' both with no 'RefreshOnLogon' entry and with RefreshOnLogon=Y. The steps are:- 1. Log on 2. Send order 3. Receive execution 4. Exit program 5. Manually change the .seqnums file from "0000000004 : 0000000004" to "0000000004 : 0000000000" 6. Log on (resend is triggered) 7. Exit program 8. Check the seqnums file again and you get "0000000007 : 0000000001" when it should be "0000000007 : 0000000006" Actually I'm using MySql for the store and logs so it suits me better to just update using SQL. Yes I'm using quickfix 1.14.3 C++ version. Thanks, Dermot ----- Original Message ----- >From: Mike Gatny <mg...@co...> >To: daw...@ya... >Cc: Grant Birchmeier <gbi...@co...>; "qui...@li..." <qui...@li...> >Date: 2017/2/28, Tue 22:39 >Subject: Re: [Quickfix-developers] Using .seqnums file to trigger a full resend > > >Dermot, > > >Are you able to reproduce your original seqnums-file-not-updating issue using e.g. the 'tradeclient' and 'executor' example apps, with RefreshOnLogon=Y? I'd like to rule out the weirdness of the resends they are sending you as a factor. > > >I don't see a problem with what you are trying to do -- I have used this technique many times when building systems that must failover to a remote site. But as I mentioned, I've only ever done it by completely shutting down the app while editing the file. > > >Also, if it fits into your architecture, do consider trying the HttpAcceptPort setting, and using the http interface to edit your seqnums. > > >Finally, are you using the latest version of quickfix (1.14.3)? > > >--Mike Gatny >Connamara Systems, LLC > >On Tue, Feb 28, 2017 at 8:23 AM, <daw...@ya...>wrote: > >Hi again Mike, I spoke to the broker who gave me the following:- >> >> >>1. They don't have any special procedure for sending historical messages and recommend using the regular resend (i.e. they don't truncate messages). >>2. They agree there should not be any admin messages included - this is a bug and they are looking into it. >> >> >>So this means I'll have to go with the contrived resend via the seqnums file. And this brings me all the way back to my original post regarding the seqnums file not containing the correct incoming seq no after resend. Do you have any inclination to take a look at this and perhaps "fix" it? I'm guessing that's a No given you don't really recommend this approach anyway so in that case what are my options? I guess I'll just have to manually update the seqnums file again with the correct number after resend. Thoughts? >> >> >>Thanks, >>Dermot >> >> >> >> >> >> >>----- Original Message ----- >>>From:Mike Gatny <mg...@co...> >>>To: daw...@ya... >>>Cc:Grant Birchmeier <gbi...@co...>; "quickfix-developers@lists. sourceforge.net" <quickfix-developers@lists. sourceforge.net> >>>Date:2017/2/25, Sat 03:05 >>>Subject:Re: [Quickfix-developers] Using .seqnums file to trigger a full resend >>> >>> >>>On Fri, Feb 24, 2017 at 12:21 PM, <daw...@ya...>wrote: >>> >>>Ok noted, thanks - don't send my own resend requests. I'm still waiting to hear back from the broker re: recommendations for getting a full day's messages resent. Their support is not great to be honest. >>>> >>>> >>>>In the meantime I tried editing the seqnums file and that worked but with some problems:- >>>> >>>> >>>>1. The resend included all heartbeats as well. Is there a way to specify not to include heartbeats/test requests? >>>>2. The resend included the broker's response to the originallogout message. This in turn triggers a new logout on my side. Is this normal behavior? Is there a way to exclude this message also? Basically I'm only interested in messages relating to orders/executions. >>> >>> >>>They should not be sending you any admin-type messages (htbeat/testrequest/logon/ logout) at all during a resend. Instead they should be replacing those msgs with sequence reset msgs with GapFillFlag field set to "Y". That's just totally broken resend behavior I'm afraid -- broken enough that I don't see what we can do on your side to mitigate it. And it could trip you up even if you aren't doing a contrived resend via the seqnums file. >>> >>> >>>--Mike Gatny >>>Connamara Systems, LLC >>> >>> >>> >>>> >>>>Thanks, >>>>Dermot >>>> >>>> >>>>----- Original Message ----- >>>>>From:Mike Gatny <mg...@co...> >>>>>To: daw...@ya... >>>>> >>>>>Cc:Grant Birchmeier <gbi...@co...>; "quickfix-developers@lists. sourceforge.net" <quickfix-developers@lists. sourceforge.net> >>>>> >>>>>Date:2017/2/24, Fri 01:41 >>>>>Subject:Re: [Quickfix-developers] Using .seqnums file to trigger a full resend >>>>> >>>>> >>>>> >>>>>Definitely don't send your own resend requests (or any other admin msg) -- that will mess with the QF state in unsafe ways. >>>>>Editing the seqnums file is definitely safe if you do it while your app is not running. And it should be safe if you are running but logged out, and RefreshOnLogon=Y. >>>>>Also note that you can use QF's http interface to reset seqnums. >>>>> >>>>> >>>>>--Mike Gatny >>>>>Connamara Systems, LLC >>>>> >>>>>On Thu, Feb 23, 2017 at 11:29 AM, <daw...@ya...>wrote: >>>>> >>>>>Hi Mike, Grant, >>>>>> >>>>>> >>>>>>Ok thanks for your feedback. Let me see what the broker comes back with first. There's nothing specific in their ROE regarding history-retrieval. >>>>>> >>>>>> >>>>>>As a test I did try sending a resend request message [BeginSeqNo=0,EndSeqNo=0] but my side then immediately logged out for some reason. >>>>>> >>>>>> >>>>>>Dermot >>>>>> >>>>>> >>>>>>----- Original Message ----- >>>>>>>From:Mike Gatny <mg...@co...> >>>>>>>To: daw...@ya... >>>>>>> >>>>>>>Cc:Grant Birchmeier <gbi...@co...>; "quickfix-developers@lists. sourceforge.net" <quickfix-developers@lists. sourceforge.net> >>>>>>>Date:2017/2/24, Fri 00:50 >>>>>>>Subject:Re: [Quickfix-developers] Using .seqnums file to trigger a full resend >>>>>>> >>>>>>> >>>>>>>Fiddling with the seqnum file should definitely work if you completely shut down your app, edit the file, then restart your app. >>>>>>>To do it with just a logout/edit/logon, try RefreshOnLogon=Y. >>>>>>> >>>>>>> >>>>>>>--Mike Gatny >>>>>>>Connamara Systems, LLC >>>>>>> >>>>>>>On Thu, Feb 23, 2017 at 12:24 AM, <daw...@ya...>wrote: >>>>>>> >>>>>>>QuickFIX Documentation: http://www.quickfixengine.org/ quickfix/doc/html/ >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>Hi Grant, >>>>>>>> >>>>>>>> >>>>>>>>Yes it sounded a bit 'hacky' to me too but this was the approach recommended to me by Asher Newcomer on this forum last October. Ok I will speak to the broker and ask what they recommend. >>>>>>>> >>>>>>>> >>>>>>>>Thanks, >>>>>>>>Dermot >>>>>>>> >>>>>>>> >>>>>>>>----- Original Message ----- >>>>>>>>>From:Grant Birchmeier <gbi...@co...> >>>>>>>>>To: daw...@ya... >>>>>>>>>Cc:"quickfix-developers@lists. sourceforge.net" <quickfix-developers@lists. sourceforge.net> >>>>>>>>>Date:2017/2/23, Thu 13:07 >>>>>>>>>Subject:Re: [Quickfix-developers] Using .seqnums file to trigger a full resend >>>>>>>>> >>>>>>>>> >>>>>>>>>I would call this a bad practice. (You're not the first to invent it.) >>>>>>>>> >>>>>>>>> >>>>>>>>>You're hacking your app's state to basically lie to your counterparty about missing messages. If you are asking for a *lot* of history, they'll probably truncate it. >>>>>>>>> >>>>>>>>> >>>>>>>>>Have you checked your counterparty's docs to see if they provide another way to get the historical data that you need? Surely you aren't the first person to need historical data from this system. >>>>>>>>> >>>>>>>>> >>>>>>>>>On Wed, Feb 22, 2017 at 9:29 PM, <daw...@ya...>wrote: >>>>>>>>> >>>>>>>>>QuickFIX Documentation: http://www.quickfixengine.org/ quickfix/doc/html/ >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>Hi Guys, >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>I'm trying to trigger a resend of all the executor's messages for the day. I manually set the incoming seq no in the .seqnums file to 0, then login and this triggers a resend request for all messages as soon as my FIX engine receives the logon response message - all good so far. All the messages are resent and received correctly. The problem is when I log out I can see that the .seqnums file does not have the updated incoming seq no (it's just set to 1) so when I log in again it triggers another resend request. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>This problem should be very easy to reproduce. The steps are:- >>>>>>>>>>1. Log on >>>>>>>>>>2. Send order >>>>>>>>>>3. Receive execution >>>>>>>>>>4. Log out >>>>>>>>>>5. Manually change the .seqnums file from "0000000004 : 0000000004" to "0000000004 : 0000000000" >>>>>>>>>>6. Log on (resend is triggered) >>>>>>>>>>7. Log out. >>>>>>>>>>8. Check the seqnums file again and you get "0000000007 : 0000000001" when it should be "0000000007 : 0000000006" >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>Any ideas? >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>Thanks a lot! >>>>>>>>>>Dermot >>>>>>>>>>------------------------------ ------------------------------ ------------------ >>>>>>>>>>Check out the vibrant tech community on one of the world's most >>>>>>>>>>engaging tech sites, SlashDot.org! http://sdm.link/slashdot >>>>>>>>>>______________________________ _________________ >>>>>>>>>>Quickfix-developers mailing list >>>>>>>>>>Quickfix-developers@lists. sourceforge.net >>>>>>>>>>https://lists.sourceforge.net/ lists/listinfo/quickfix- developers >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>-- >>>>>>>>> >>>>>>>>>Grant Birchmeier >>>>>>>>> >>>>>>>>>Connamara Systems, LLC >>>>>>>>> >>>>>>>>>Made-To-Measure Trading Solutions. >>>>>>>>>Exactly what you need. No more. No less. >>>>>>>>> >>>>>>>>>http://connamara.com >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>------------------------------ ------------------------------ ------------------ >>>>>>>>Check out the vibrant tech community on one of the world's most >>>>>>>>engaging tech sites, SlashDot.org! http://sdm.link/slashdot >>>>>>>>______________________________ _________________ >>>>>>>>Quickfix-developers mailing list >>>>>>>>Quickfix-developers@lists. sourceforge.net >>>>>>>>https://lists.sourceforge.net/ lists/listinfo/quickfix- developers >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>> >>>>> >>>>> >>> >>> >>> > > > |
