block allocator segfault

2010-10-18
2013-04-25
  • Dimitris Michail

    Hi,

    I am experiencing a segmentation fault with the following short program:

    #include <vector>
    #include <stxxl/mng>
    int main(int argc, char* argv[]) {
        typedef stxxl::typed_block< 16 * 1024 * 1024, double> typed_block;
        std::vector<typed_block, stxxl::new_alloc<typed_block> > blocks(1);
    }
    

    If I change the block size from 16MB to 2MB, the problem disappears.
    Dynamically creating an array of typed_block with the new operator also works fine.

    I am using stxxl-1.3.0 and g++-4.4 on an amd64 platform compiled with boost support and pmode.

    Am I doing something wrong here?

    Best Regards,
    Dimitris

     
  • Andreas Beckmann

    This is a stack issue. Default stack limit is 8 MB (check with ulimit -s).

    The following code does not use stxxl or a custom allocator and fails the same way:

    #include <vector>
    struct data { char a_lot[16 * 1024 * 1024]; };
    int main() {
            std::vector<data> alotofdata(1);
    }
    

    std::vector::vector(size_type n) needs to put at least one temporary element on the stack to initialize the n elements.

    But on the other hand do you *really* want a

    std::vector<typed_block>
    

    ? This is bad because it involves copying typed_blocks around in memory, e.g. for (unneccessary) initialization?
    Did you want perhaps

      std::vector<typed_block *> blocks;
      for (int i = 0; i < n; ++i)
        blocks.push_back(new typed_block);
    

    or

      typed_block * block = new typed_block[n];
    

    ?

    stxxl mng layer functions only take

    typed_block *
    

    as argument.

    Andreas

     
  • Dimitris Michail

    Thanks for the clarification.

    Dimitris

     
  • Edgar

    Edgar - 2011-01-19

    Hi,

    I like to compile algo/test_sort_all_parameters.cpp with VC9 but can't continue due to a static assertion in the typed_block constructor: STXXL_STATIC_ASSERT(sizeof(typed_block) == raw_size);

    Well, I think there is something wrong with the intended use of filler_struct__<unsigned n>. If there is no internal memory fragmentation with template type T_, which should be true for integral types, than the template argument for filler_struct__ becomes 0. But filler_struct__<0> has a size of 1, which means, if RawSize_ == 2^^17 => sizeof (typed_block) == 2^^17+1 (+3 bytes for alignment on 32-bit x86).

    This problem can be solved by eliminating filler_struct__ and passing an additional padding template parameter to the base class, so that the block elements can be defined as something like so:

    T elem;
    char __padding;

    This needs some extra construction code for elements 1..n-1.

    Edgar

     
  • Johannes Singler

    First of all, are you reporting this bug against STXXL 1.3.0 or against the trunk version.  If the former, please try with the latest trunk.  We have made some changes to this part.

    Johannes

     
  • Edgar

    Edgar - 2011-01-19

    Changes applied.

    Thanks
    Edgar

     
  • Johannes Singler

    Edgar,

    could you please clarify your answer?  Does the current trunk version fix the problem?  This is very important for us since we soon want to release version 1.3.1, which will be based on the trunk.

    Johannes

     
  • Edgar

    Edgar - 2011-01-19

    Yes, the current trunk version fixes the problem.

    Thank you
    Edgar

     

Log in to post a comment.