segmentation fault on sort

2011-06-17
2013-04-25
  • Hi there,

    I have used stxxl::vector and stl::sort which gave me no segmentation fault. However, if I use stxxl::vector and stxxl::sort, I am getting a segmentation fault. I have followed the tutorial. Any hint is appreciated. The error is below:

    Program received signal EXC_BAD_ACCESS, Could not access memory.
    Reason: KERN_INVALID_ADDRESS at address: 0x000000000000000f
    0x0000000100047ac4 in stxxl::disk_queued_file::awrite () at stream_iterator.h:94
    94 return *this;
    (gdb) where
    #0  0x0000000100047ac4 in stxxl::disk_queued_file::awrite () at stream_iterator.h:94


    Regards

     
  • Hi,

    I just solved the problem by explicitly clearing the vector and the segmentation fault disappeared. It seems like a write was occuring after the destructor being called. But the problem is solved explicitly calling clear.

     
  • Nice. Which STXXL version are you using? Is the problem reproducible in 1.3.1 and/or SVN trunk?
    Do you have some code example to reproduce the problem (in case it is still happening in the most current version I'd like to fix it).
    What platform do you use? OS, Compiler, Kernel, …. Version numbers. Compiler flags.

    Andreas

     
  • Hi Andreas,

    Thanks for looking into this. stxxl is great!

    I am using stxxl-1.3.1 (and I could test it on SVN trunk version). The code is really large but I could come up with a sample code with the similar problem. Please find the details of the platform below:
    OS: Mac(10.6.7)
    compiler: g++ 4.2.1
    compilation flags :
    CXX              = $(STXXL_CXX) -DSTXXL_VERBOSE_LEVEL=1
    CPPFLAGS        +=  -g
    CPPFLAGS        += -W -Wall
    CPPFLAGS        += $(STXXL_CPPFLAGS)

    $(STXXL_LDLIBS)

    I was testing various block sizes for the sort function and I found a post in the forum that mentions that the minimum amount of memory is 4096 (for linux). Assuming that I have a file as large as 4 MB, the minimum block size seems to be 4 MB as well. Otherwise, I would get an abort function that requests me to add more memory.

    If I may ask, is the minimum amount of memory for sort equivalent to the length of file ? Thanks very much.

    Regards,
    Jasbir

     
  • Or maybe, the post 4096 refers to 4096 KB as the example in the tutorial refers it in KB ? Thanks.

     
  • I believe, it is not the length of the file but the minimum memory required is 4 MB. Since my file is 4 MB, the minimum seemed 4 MB as well. Thanks.

     
  • You need to keep a constant number of blocks in internal memory, at least 3 IIRC. If your file fits entirely into memory less blocks may suffice (just need enough blocks to keep the whole file).

     
  • getting back to the original problem …

    * Could you test a newer gcc than 4.2? 4.4 would be nice.
    * A minimized test case would be nice (but I could reduce it myself in case its open source)

    Andreas

     
  • Hi Andreas,

    I am currently running some tests where I need the environment to be similar. However, once done (by tomorrow), I could update the gcc and reproduce the problem.

    Regards,
    Jasbir

     
  • Hi Andreas,

    I tested the trunk version and it was producible with gcc4.4 on  ubuntu 10.04. The problem occurs when I read a large file, sort it and then end the program. My experiments are still running. So, I can't update the gcc version on mac yet.

    Is there any debugging flags that I could use to produce more debugging info that I can send you ? The code is very large, so, I am trying to reproduce the problem using a small file. Once I am able to reproduce the fault on the code segment, I would send the code segment. Currently, it is not being produced with small files.

    Regards,
    Jasbir

     
  • This is the code segment

    struct DOC {

        int a,
            b;

            DOC(){};
            DOC(int _a,int _b):a(_a),b(_b){}
        ~DOC(){};
    };
    stxxl::vector<DOC> doc;

    M = 10000000;
    for(i=0;i<M;i++)
                    doc.push_back(DOC(i,i));

            for(i=0;i<M;i++)
           {
             if(doc_.a == x)
                    ;
           }

    ---_

     
  • Do you still experience this problem?
    Can you extend the code sample to a working (and crashing) program?
    Adding header includes and a main() may be sufficient. Add
     

    #define STXXL_VERBOSE_LEVEL 1
    

    before the #includes in that test program to get more verbose output

    Andreas