SourceForge has been redesigned. Learn more.
Close

Any significant changes in STXXL?

2010-10-08
2013-04-25
  • Mohammed Mediani

    I had a piece of software which was working correctly. Unfortunately recently I ran into lots of "Bus Error"s  (after updating from trunk and recompiling).
    I mostly get this error when I am I/Oing to an STXXL vector (sometimes while feeling the vector, sometimes whil sorting).
    Are there any significant changes which cause this to happen?
    ps. The program is multithreaded. However, different threads access different vectors. The execution is performed on a cluster using Slurm manager.

     
  • Mohammed Mediani

    *sorry, there was a typo: filling and not feeling ;-)

     
  • Andreas Beckmann

    I had a piece of software which was working correctly. Unfortunately recently I ran into lots of "Bus Error"s (after updating from trunk and recompiling).

    Platform? OS? Compiler? Compiler options?
    What version did you update from? What exact SVN revision does fail? Does it work in 1.3.0? Did you try to bisect to find the first failing revision?
    Did you try to compile with debug information included and try to run the program in a debugger or run the core file through a debugger to get a stack trace?

    I mostly get this error when I am I/Oing to an STXXL vector (sometimes while filling the vector, sometimes whil sorting).

    Is this a vector bound to a file or in the scratch space defined in .stxxl? What filetypes do you use?

    Are there any significant changes which cause this to happen?

    None that I could think of.

    ps. The program is multithreaded. However, different threads access different vectors. The execution is performed on a cluster using Slurm manager.

    Can you reproduce it on a single node? Using only thread? Create a minimal test case?

    Andreas

     
  • Mohammed Mediani

    Thanks a lot!

    Platform? OS? Compiler? Compiler options?

    x86_64, OpenSuse, g++-4.3, -std=c++0x -fopenmp -D_GLIBCXX_PARALLEL -c -g -O3

    What version did you update from? What exact SVN revision does fail? Does it work in 1.3.0? Did you try to bisect to find the first failing revision?

    It was the same version, but different revisions. The one I have most problems with is the current (2671). I didn't try bisecting!

    Did you try to compile with debug information included and try to run the program in a debugger or run the core file through a debugger to get a stack trace?

    None! I would do that next, if no simple and direct answer!

    Is this a vector bound to a file or in the scratch space defined in .stxxl? What filetypes do you use?

    It is not bound to a file. It is filled from processed data read from a file. all threads use the same scratch space file. I have only one huge file of the following type:
    disk=/export/data1/data/i13hpc3_4245.stxxl,200000,syscall

    Can you reproduce it on a single node? Using only thread? Create a minimal test case?

    I will  ASAP.
    Regards,
    Mohammed

     
  • Andreas Beckmann

    Can you try g++-4.4? The PARALLEL MODE is only fully functional in 4.4, STXXL disables the use of several parallel algorithms when compiled with 4.3. Also try the program without PARALLEL MODE (drop -fopenmp -D_GLIBCXX_PARALLEL).

    There is a separate scratch file per node? And only a single process (with possibly multiple threads) is accessing this file? Do not share the stxxl scratch file between concurrent processes.

    Did you look for hardware errors from the disks in the system logs?

    You could try the io/benchmark_disks, io/benchmark_configured_disks, io/verify_disks programs to test the I/O subsystem.

    Andreas

     
  • Johannes Singler

    Well, bus error sounds a lot like wrong alignment.  And we had changes to the alignment/filler stuff recently, did we not?  Please check specifically revisions 2593 and 2594.

    Johannes

     
  • Andreas Beckmann

    Hi Mohammed, did you make any progress investigating this problem?

    Andreas

     
  • Mohammed Mediani

    Hello,
    Thanks for getting back to me!
    At the moment I am in vacation, but some time before starting it. I updated through svn and recompiled ,, and the Bus Error problem went away.
    Still some crushings happen, which I intend to investigate after getting back from vacation.
    Thanks

     

Log in to post a comment.