Re: [Quickfixj-users] Server crash leaves QuickFIX/J engine (1.5.3) in corrupt state for FileStorePath=filestore

[Christoph J. <chr...@ma...>]

Hi Tom,

IMHO you can always run into this kind of problem (data corruption) when the power fails. You could 
try turning on synchronous writes but this of course will slow down the message processing.
The option to turn this on is FileStoreSync=Y. I just discovered that it is not part of the 
documentation. :-/ Will add it.

Hope that helps a bit.
Cheers,
Chris.



On 06/12/2018 09:33, tom...@up... wrote:
> QuickFIX/J Documentation: http://www.quickfixj.org/documentation/
> QuickFIX/J Support: http://www.quickfixj.org/support/
>
>
>
> Hello dear readers,
>
> This is a situation that I’ve encountered twice now, at two different clients of ours. Their 
> application server with our FIX module running on it based on the QuickFIX/J engine (1.5.3) 
> crashes, and leaves the QuickFIX/J engine in a corrupt state. The first time, the server crash was 
> for a virtualized server (some reason the hosting provider never told me) and the second was due a 
> UPS that did not signal the server to shut down properly when the battery started running out. Our 
> configuration has /FileStorePath=filestore/ in both instances. Our instances are buy side (i.e. 
> initiators).
>
> On inspection of the QuickFIX/J state files, I’m seeing that they are corrupted. I can fix this by 
> removing all state files and restart our module, then all is fine again. The first time happened 
> about a year ago, the second just earlier today. All state files were empty (.body, .header, 
> .senderseqnums, .targetseqnums), except for the .session file (see attachment). You’ll see it’s 
> filled with all ASCII NUL characters. A valid one as I’ve noticed has something like 
> /<NUL><ETX><some number>/ as contents.
>
> The exceptions I’m getting in my module’s log file goes like:
>
> quickfix.ConfigError: error during session initialization
>
>              at 
> quickfix.mina.initiator.AbstractSocketInitiator.createSessions(AbstractSocketInitiator.java:169)
>
>              at 
> quickfix.mina.initiator.AbstractSocketInitiator.createSessionInitiators(AbstractSocketInitiator.java:84)
>
>              at quickfix.SocketInitiator.initialize(SocketInitiator.java:86)
>
>              at quickfix.SocketInitiator.start(SocketInitiator.java:64)
>
>              at <INTERNALS-STRIPPED>
>
>              at <INTERNALS-STRIPPED>
>
>              at <INTERNALS-STRIPPED>
>
>              at java.lang.Thread.run(Unknown Source)
>
> Caused by: java.lang.RuntimeException: java.io.IOException: invalid UTC timestamp value:
>
>              at quickfix.FileStoreFactory.create(FileStoreFactory.java:80)
>
>              at quickfix.Session.<init>(Session.java:467)
>
>              at quickfix.DefaultSessionFactory.create(DefaultSessionFactory.java:185)
>
>              at quickfix.mina.SessionConnector.createSession(SessionConnector.java:140)
>
>              at 
> quickfix.mina.initiator.AbstractSocketInitiator.createSessions(AbstractSocketInitiator.java:163)
>
>              ... 7 more
>
> Caused by: java.io.IOException: invalid UTC timestamp value:
>
>              at quickfix.FileStore.initializeSessionCreateTime(FileStore.java:137)
>
>              at quickfix.FileStore.initializeCache(FileStore.java:123)
>
>              at quickfix.FileStore.initialize(FileStore.java:116)
>
>              at quickfix.FileStore.<init>(FileStore.java:101)
>
>              at quickfix.FileStoreFactory.create(FileStoreFactory.java:78)
>
>              ... 11 more
>
> I remember this is the same exception I got the first time one of our clients’ server crashed.
>
> Of course in an ideal world a server should never crash, but alas it does. It would appear that 
> the QuickFIX/J engine’s state isn’t properly flushed to disk? Or maybe this happens because the 
> engine’s state cannot be written atomically? I do not know, as I’ve never had the time to delve 
> into QuickFIX/J internals and see how this situation can arise.
>
> Perhaps it has been reported before and fixed in a later version, I’ve not had time to delve into 
> that either. Perhaps this is something that simply can’t be fixed due to the way filesystems work? 
> Or perhaps it can?
>
> Besides this, I’m pretty happy with QuickFIX/J 1.5.3 and I wish to thank all contributors to 
> QuickFIX/J!
>
> Kind regards,
>
> --
>
> Tom Tempelaere
>
> Upsilon SA
>
>
>
> _______________________________________________
> Quickfixj-users mailing list
> Qui...@li...
> https://lists.sourceforge.net/lists/listinfo/quickfixj-users

-- 
Christoph John
Software Engineering
T +49 241 557080-28
chr...@ma...

MACD GmbH
Oppenhoffallee 103
52066 Aachen, Germany
www.macd.com

Amtsgericht Aachen: HRB 8151
Ust.-Id: DE 813021663
Geschäftsführer: George Macdonald