Exception with stl lower_bound

Ramki
2014-01-24
2014-01-27
  • Ramki
    Ramki
    2014-01-24

    Hello,
    I'm a newbie to STXXL and my requirement was to check how efficient it is to retrieve a range of data from a big file.So i created a big data file(STXXL 1.4.0), then opened in Read-Only mode, and associated a stxxl vector to it. Then used stl::lower_bound with that vector. It works fine with smaller files, however with large files(say of size 600MB), it throws exception at the below line:

    File:vector.h
    Method:write_page
    Line:1627,wait_all(reqs, last_block - page_no * page_size);

    Source(important lines only for clarity):

    stxxl::syscall_file in_file(strFileName, stxxl::file::DIRECT | stxxl::file::RDONLY);

    stxxl::vector<scopeType,noOfBlocksPerPage, stxxl::lru_pager<cachePageSize="">, blockSize> vValues(&in_file);

    //This code throws the exception
    vector_iterator itUpper =std::lower_bound(vValues.begin(), vValues.end(), point2);

    However, opening the file in RDWR mode solves the issue. Please let me know what is the issue and how to handle it.

    Thanks.

     
    Last edit: Ramki 2014-01-24
  • Timo Bingmann
    Timo Bingmann
    2014-01-24

    Hi Ramki,
    you have run into the mutable iterator problem of stxxl::vector.

    The issue is that you use mutable iterators, and these require the data blocks to be read and written back to disk on each operator*() access, because STXXL cannot know if you changed the data inside.

    You can easily solve your problem by using const_iterators together with cbegin() and cend().

    Timo

     
  • Ramki
    Ramki
    2014-01-27

    Thanks a lot, it worked like a charm:-)
    BTW, in order to know under the hood stuff like this,which i should know, can you please recommend any article, book about STXXL? Thanks in advance.

     
  • Timo Bingmann
    Timo Bingmann
    2014-01-27

    There is no book on STXXL, except for Roman Dementiev's PhD thesis. That contains much of the design, but is from an older year.

    The Doxygen Docs are supposed to be the primary source, which is why I initiated them.