Exception with stl lower_bound

Ramki
2014-01-24
2014-01-27
  • Ramki

    Ramki - 2014-01-24

    Hello,
    I'm a newbie to STXXL and my requirement was to check how efficient it is to retrieve a range of data from a big file.So i created a big data file(STXXL 1.4.0), then opened in Read-Only mode, and associated a stxxl vector to it. Then used stl::lower_bound with that vector. It works fine with smaller files, however with large files(say of size 600MB), it throws exception at the below line:

    File:vector.h
    Method:write_page
    Line:1627,wait_all(reqs, last_block - page_no * page_size);

    Source(important lines only for clarity):

    stxxl::syscall_file in_file(strFileName, stxxl::file::DIRECT | stxxl::file::RDONLY);

    stxxl::vector<scopeType,noOfBlocksPerPage, stxxl::lru_pager<cachePageSize="">, blockSize> vValues(&in_file);

    //This code throws the exception
    vector_iterator itUpper =std::lower_bound(vValues.begin(), vValues.end(), point2);

    However, opening the file in RDWR mode solves the issue. Please let me know what is the issue and how to handle it.

    Thanks.

     
    Last edit: Ramki 2014-01-24
  • Timo Bingmann

    Timo Bingmann - 2014-01-24

    Hi Ramki,
    you have run into the mutable iterator problem of stxxl::vector.

    The issue is that you use mutable iterators, and these require the data blocks to be read and written back to disk on each operator*() access, because STXXL cannot know if you changed the data inside.

    You can easily solve your problem by using const_iterators together with cbegin() and cend().

    Timo

     
  • Ramki

    Ramki - 2014-01-27

    Thanks a lot, it worked like a charm:-)
    BTW, in order to know under the hood stuff like this,which i should know, can you please recommend any article, book about STXXL? Thanks in advance.

     
  • Timo Bingmann

    Timo Bingmann - 2014-01-27

    There is no book on STXXL, except for Roman Dementiev's PhD thesis. That contains much of the design, but is from an older year.

    The Doxygen Docs are supposed to be the primary source, which is why I initiated them.

     

Log in to post a comment.

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:

JavaScript is required for this form.





No, thanks