I have been using STXXL for some time now and it works as expected. However, i noticed that whenever the application does not close normally due to unhandled exception or quitting debugging sessions midway, cause the source file to be in a invalid state that is instead of being the original size(say 1KB), it is now at 2048KB. I'm no longer able to fetch the inserted data from it, it is corrupted. Please let me know the reason and the ways to handle it.
Source(important lines only for clarity):
stxxl::syscall_file in_file(strFileName, stxxl::file::DIRECT | stxxl::file::RDWR);//in_file is used as a member object and allocated on the heap.
stxxl::vector, blockSize> vValues(&in_file);
Thanks in advance.
Well, this is intended behaviour, as far as intended things can happend when you crash a program.
The stxxl::vector will resize the underlying file to a multiple of the configured block size, since it must be able to write full blocks to disk.
But when the vector destructs itself, the actually used number of bytes is correctly set. This is done in vector::~vector() via file->set_size.
Since the vector caches blocks via the paging mechanism, the data is only written to disk when the cache is flushed. Thus after a crash, data on disk is not "corrupt", but actually just stale.
Neither flushing not truncation happens when your program segfaults. But if you manage to throw exceptions instead, then you can correctly close the vectors during stack unwinding.
Thanks for the quick response Timo. The reason i said it is corrupt is bcos of the situation below:
1.First run - sucessfully insert a data point(say X), exit gracefully.File Size is 1KB
2.Second run - successfully insert a data point(say Y), app crashes. File Size is 2048KB - as explained.
3.Third run - fetch data point Y, not fetched, behaviour is as explained. Fetch data point X, this is also not fetched, this is the issue.This is important otherwise anytime my app crashed for some reason, the old data is lost.
I hope this clarifies why i mentioned it as corrupt.
Can you comment on the corruption behaviour i mentioned above? That is do you agree it is an issue and if so, will there be a plan to fix it?
Thanks & Regards,
I guess I dont quite understand the issue yet.
You should check the state of the file before 3. It should not contain Y, but should contain X at the position you inserted it, then in 3 the point X should be loaded as expected.