I'm writing/running a program that handles a large number of protein
structures, each requiring a significant amount of memory. I don't want
to read these structures into memory each time I need them, so I cache
them in a hash table. Whenever the process pass just undre 800 MB
(according to top), sbcl crash with the following error message.
mmap: Cannot allocate memory
fatal error encountered in SBCL pid 32008(tid 182901392576):
remap_free_pages: page moved, 0x0001d2d1 ==> 0x00000000
The system is too badly corrupted or confused to continue at the Lisp
level. If the system had been compiled with the SB-LDB feature, we'd drop
into the LDB low-level debugger now. But there's no LDB in this build, so
we can't really do anything but just exit, sorry.
Apparently it is running out of memory, but that don't make sense, since
the machine is an x86-64 with 8GB of memory, and I'm using the 64-bit
version of sbcl, that maps 8GB of memory (and top agree here), so
effectively I should only be using about 10% of the available memory.
To continue debuging I'm told (according to the error message) to use
SB-LDB, but according to the sbcl-internals wiki
(http://sbcl-internals.cliki.net/ldb), I should add :sb-ldb to the file
customize-target-features.lisp, but I can't find this file in the sbcl
source tree. Furthermore, when trying with GDB, it stops time after time
with "segmentation fault", so gdb alone is not useful for debuging.
While I think to remember it mentioned somewhere that the "segfaults" is
a part of the normal operation of the garbage collector (or similar), it
don't help either.
Does anyone have a clue what is going on, and where I can go from here?