[Quickfix-developers] Best Way to Keep track of Messages

[John K. <kam...@gm...>]

Hi,
I've managed to connect to my fix app and subscribed to a market data feed
(i.e. trades, market status etc). 
Ny problem is not really a QF issue, but rather an implementation issue. Let
me explain.
Once the feed I'm connecting to starts, messages are sent real-time, i.e.
when a trade occurs, an announcemnt is placed etc. During this time, I
receive each message, do some stuff then save it to a sql db. Since I"m
saving to db and don't want duplicates, I wanted to use the message sequence
no. to track each message so as not to save it twice. 
Then I realised that the app i"m connecting to does not keep track of
sessions, so, during the real-time feed should the connection drop, the
sequence restarts. (this actually caused me some headaches till i realised i
needed to set ResetOnDisconnect=Y in the cfg). 
On a disconnect the server assumes its a new session and re-sends all
previous messages again (even if they are in the thousands), and resumes the
normal feed. Since I don't want duplicates so I'm wondering whats the best
way to keep track of the messages!
I'd thought of logging the last message I received (using the msgseqno) and
waiting till it I get a match during the batch re-send but I'm not sure if
during the re-send the it will have the exact same msgseqno, and so I could
end up with bad data (i.e. with missing or duplicates messgaes).
My second idea was to delete all messages in my database for that day and
restart afresh, but considering that I'm processing around 50-200k messages,
it may take too long.
Has anyone ever had the need to uniquely id a message and save it? 
If so, got any tips on how I should go about this.

Thanks
John
-- 
View this message in context: http://www.nabble.com/Best-Way-to-Keep-track-of-Messages-tf4562443.html#a13021106
Sent from the QuickFIX - Dev mailing list archive at Nabble.com.