I'm a newbie to STXXL and my requirement was to check how efficient it is to retrieve a range of data from a big file.So i created a big data file(STXXL 1.4.0), then opened in Read-Only mode, and associated a stxxl vector to it. Then used stl::lower_bound with that vector. It works fine with smaller files, however with large files(say of size 600MB), it throws exception at the below line:
Line:1627,wait_all(reqs, last_block - page_no * page_size);
Source(important lines only for clarity):
stxxl::syscall_file in_file(strFileName, stxxl::file::DIRECT | stxxl::file::RDONLY);
stxxl::vector<scopeType,noOfBlocksPerPage, stxxl::lru_pager<cachePageSize="">, blockSize> vValues(&in_file);
//This code throws the exception
vector_iterator itUpper =std::lower_bound(vValues.begin(), vValues.end(), point2);
However, opening the file in RDWR mode solves the issue. Please let me know what is the issue and how to handle it.
you have run into the mutable iterator problem of stxxl::vector.
The issue is that you use mutable iterators, and these require the data blocks to be read and written back to disk on each operator*() access, because STXXL cannot know if you changed the data inside.
You can easily solve your problem by using const_iterators together with cbegin() and cend().
Thanks a lot, it worked like a charm:-)
BTW, in order to know under the hood stuff like this,which i should know, can you please recommend any article, book about STXXL? Thanks in advance.
There is no book on STXXL, except for Roman Dementiev's PhD thesis. That contains much of the design, but is from an older year.
The Doxygen Docs are supposed to be the primary source, which is why I initiated them.