Menu
▾
▴
#16 Could not realloc memory (764188368 bytes)
A new attempt at building a assembly using the Illumina 181bp data has crashed with this error:
STRING_NUM_BITS 31
OFFSET_BITS 31
STRING_NUM_MASK 2147483647
OFFSET_MASK 2147483647
MAX_STRING_NUM 2147483647
Hash_Mask_Bits 23
Max_Hash_Strings 792879
Max_Hash_Data_Len 141243306
Max_Hash_Load 0.750000
Kmer Length 14
Min Overlap Length 40
MAX_ERRORS 1967
ERRORS_FOR_FREE 1
Num_PThreads 2
Max_Reads_Per_Batch 100000
Max_Reads_Per_Thread 12500
HASH_TABLE_SIZE 8388608
sizeof(Hash_Bucket_t) 216
hash table size: 1728 MB
check 32 MB
info 3 MB
start 6 MB
Initialize_Work_Area: MAX_ERRORS=1967 allocated 16MB
Initialize_Work_Area: MAX_ERRORS=1967 allocated 16MB
Build_Hash_Index from 4756694 to 5549572
String_Ct: 100000/ 792879 totalLen: 17815301/ 141243306 Hash_Entries: 12074826/ 132120576 Load: 6.85%
String_Ct: 200000/ 792879 totalLen: 35627743/ 141243306 Hash_Entries: 21330158/ 132120576 Load: 12.11%
String_Ct: 300000/ 792879 totalLen: 53441163/ 141243306 Hash_Entries: 29153989/ 132120576 Load: 16.55%
String_Ct: 400000/ 792879 totalLen: 71259074/ 141243306 Hash_Entries: 36030114/ 132120576 Load: 20.45%
String_Ct: 500000/ 792879 totalLen: 89072542/ 141243306 Hash_Entries: 42133095/ 132120576 Load: 23.92%
String_Ct: 600000/ 792879 totalLen: 106882113/ 141243306 Hash_Entries: 47625106/ 132120576 Load: 27.04%
String_Ct: 700000/ 792879 totalLen: 124694355/ 141243306 Hash_Entries: 52561971/ 132120576 Load: 29.84%
HASH LOADING STOPPED: strings 792879 out of 792879 max.
HASH LOADING STOPPED: length 141243306 out of 141243306 max.
HASH LOADING STOPPED: entries 56788677 out of 132120576 max (load 32.24).
realloc Extra_Ref_Space max_extra_ref_ct = 95523546
Could not realloc memory (764188368 bytes)
overlapInCore: AS_UTL_alloc.C:81: void safe_realloc(void, size_t): Assertion `p != __null' failed.
Failed with 'Aborted'
Backtrace (mangled):
/scratch/wyim/dnaseq/ca_grid/wgs-8.1/Linux-amd64/bin/overlapInCore(_Z17AS_UTL_catchCrashiP7siginfoPv+0x27)[0x416197]
/lib64/libpthread.so.0(+0xf500)[0x7fb8ccd06500]
/lib64/libc.so.6(gsignal+0x35)[0x7fb8cc9968a5]
/lib64/libc.so.6(abort+0x175)[0x7fb8cc998085]
/lib64/libc.so.6(+0x2ba1e)[0x7fb8cc98fa1e]
/lib64/libc.so.6(__assert_perror_fail+0x0)[0x7fb8cc98fae0]
/scratch/wyim/dnaseq/ca_grid/wgs-8.1/Linux-amd64/bin/overlapInCore[0x415367]
/scratch/wyim/dnaseq/ca_grid/wgs-8.1/Linux-amd64/bin/overlapInCore(_Z16Build_Hash_IndexP8gkStreamiP10gkFragment+0xa66)[0x408906]
/scratch/wyim/dnaseq/ca_grid/wgs-8.1/Linux-amd64/bin/overlapInCore(_Z13OverlapDriverv+0x2e9)[0x4114a9]
/scratch/wyim/dnaseq/ca_grid/wgs-8.1/Linux-amd64/bin/overlapInCore(main+0x1016)[0x412e96]
/lib64/libc.so.6(__libc_start_main+0xfd)[0x7fb8cc982cdd]
/scratch/wyim/dnaseq/ca_grid/wgs-8.1/Linux-amd64/bin/overlapInCore[0x406509]
Backtrace (demangled):
[0] /scratch/wyim/dnaseq/ca_grid/wgs-8.1/Linux-amd64/bin/overlapInCore::AS_UTL_catchCrash(int, siginfo, void) + 0x27 [0x416197]
[1] /lib64/libpthread.so.0::(null) + 0xf500 [0x7fb8ccd06500]
[2] /lib64/libc.so.6::(null) + 0x35 [0x7fb8cc9968a5]
[3] /lib64/libc.so.6::(null) + 0x175 [0x7fb8cc998085]
[4] /lib64/libc.so.6::(null) + 0x2ba1e [0x7fb8cc98fa1e]
[5] /lib64/libc.so.6::(null) + 0 [0x7fb8cc98fae0]
[6] /scratch/wyim/dnaseq/ca_grid/wgs-8.1/Linux-amd64/bin/overlapInCore() [0x415367]
[7] /scratch/wyim/dnaseq/ca_grid/wgs-8.1/Linux-amd64/bin/overlapInCore::Build_Hash_Index(gkStream, int, gkFragment) + 0xa66 [0x408906]
[8] /scratch/wyim/dnaseq/ca_grid/wgs-8.1/Linux-amd64/bin/overlapInCore::OverlapDriver() + 0x2e9 [0x4114a9]
I don't see why it is having trouble allocating only this small amount of memory, the server has over 8GB free so this must be a less obvious problem than indicated by the error.
[9] /scratch/wyim/dnaseq/ca_grid/wgs-8.1/Linux-amd64/bin/overlapInCore::(null) + 0x1016 [0x412e96]
[10] /lib64/libc.so.6::(null) + 0xfd [0x7fb8cc982cdd]
[11] /scratch/wyim/dnaseq/ca_grid/wgs-8.1/Linux-amd64/bin/overlapInCore() [0x406509]
GDB:
/home/wyim/scratch/dnaseq/ca_grid/1-overlapper/overlap.sh: line 60: 9618 Aborted (core dumped) $bin/overlapInCore --hashbits 23 --hashload 0.75 -t 2 $opt -k 14 -k /home/wyim/scratch/dnaseq/ca_grid/0-mercounts/illu.nmers.ovl.fasta -o /home/wyim/scratch/dnaseq/ca_grid/1-overlapper/$bat/$job.ovb.WORKING.gz /home/wyim/scratch/dnaseq/ca_grid/illu.gkpStor
By chance, are your processes limited to only 2gb memory? It is failing on an allocation that, I think, pushes it above 2gb.
The HASH LOADING STOPPED lines claim that the table is mostly empty. You can reduce ovlHashBits by one and save a bunch of space.
http://wgs-assembler.sourceforge.net/wiki/index.php/RunCA#OVL_Overlapper
Do you have good reason to use 14-mers instead of the default? It won't help with memory usage, and will increase run times. It might actually be using more memory, because more 14-mers will look like useless repetitive kmers, and that's what it's trying to allocate space for.