a simple example and another error

2008-10-28
2013-04-25
  • Hi,
    when I tried to create a "cooked down" example for the unanticipated high memory usage problem (see more disks and less space thread) I stumbled into another problem - I decided to "cook down" that problem first. - Let's start with the easy stuff...
    Enclosed, you find the error message and the complete source code...

    error_test
    [STXXL-MSG] STXXL v1.2.1 (release)
    [STXXL-ERRMSG] Warning: no config file found.
    [STXXL-ERRMSG] Using default disk configuration.
    [STXXL-MSG] 1 disks are allocated, total space: 1000 MB
    error_test: /TL/home-1/lkunert/src/stxxl/stxxl/include/stxxl/bits/algo/sort.h:804: void stxxl::sort(ExtIterator_, ExtIterator_, StrictWeakOrdering_, stxxl::unsigned_type) [with ExtIterator_ = stxxl::vector_iterator<Uint, stxxl::RC, long long unsigned int, long long int, 2097152u, stxxl::lru_pager<8u>, 4u>, StrictWeakOrdering_ = Compare<Uint>]: Assertion `2 * block_type::raw_size * sort_memory_usage_factor() <= M' failed.
    Aborted

    #include "stxxl.h"

    typedef unsigned int uint;

    template <typename T>
    struct Compare : public std::less<T>
    {
        static T
        min_value() { return T( 0 ); };

        static T
        max_value() { return T( std::numeric_limits<uint>::max()); };
    };

    struct Uint
    {
        uint i_;

        Uint( uint i =0 )
          : i_( i )
        {};

        bool
        operator<( const Uint& o ) const
        {
            return i_ < o.i_;
        };
    };

    int main (int argc, char **argv)
    {
        stxxl::vector<Uint> v;
        // stxxl::VECTOR_GENERATOR<Uint>::result v;

        for( uint i =1; i < 100001; i++ )
        {
        v.push_back( Uint( i ));
        };

        stxxl::sort( v.begin(), v.end(), Compare<Uint>(), 10000 );

        return( 0 );
    }

     
    • Hi,

      the memory M you give to the sorter must be enough to fit 2 data blocks of stxxl::vector => reduce the block size to 4K for the vector keeping M=10000 bytes.

      The default block size is 4 MB.

      Roman

       
    • Hi!

      Thanks, that solves the problem.
      1) What do you think about silently setting M to max( 2*block_size, M )?

      2) use of    
      > stxxl::vector< Uint, no_blocks_in_page, no_pages_in_cache, block_size, stxxl::RC, stxxl::lru > v;
      does not work, has the order of the parameters changed?

      >
      >    static const uint no_pages_in_cache  =8;        // number of pages in cache   (default: 8)  
      >    static const uint no_blocks_in_page  =4;        // number of blocks in a page (default: 4; >= D --- no disks)
      >   static const uint block_size         =4*1024;   // block size in B            (default: 4MB; large is better)
      >
      >   stxxl::VECTOR_GENERATOR< Uint, no_blocks_in_page, no_pages_in_cache, block_size, stxxl::RC, stxxl::lru >::result v;
      >
      >    for( uint i =1; i < 100001; i++ )
      >    {
      >    v.push_back( Uint( i ));
      >    };
      >
      >    uint M = 10000;
      >    stxxl::sort( v.begin(), v.end(), Compare<Uint>(), std::max( M, 2*block_size ));
      >
      >    return( 0 );

       
      • The parameter lists for stxxl::vector<> and stxxl::VECTOR_GENERATOR<> are different, you can't substitute them directly. Recommended is using VECTOR_GENERATOR<...>::result

        Andreas