Menu
▾
▴
Re: [wgs-assembler-users] last fragment id < 0 ... runCA can't match the output
|
From: Langhorst, B. <Lan...@ne...> - 2015-03-24 00:55:02
|
Hi Brian:
Looks like this overflow may cause more trouble.
Tried setting the num reads… meryl starts - but dies quickly .
I’ll try to start with merged reads (from PEAR) only… should drop the read count significantly.
Let me know if you want the core file.
brad
/home/NEB/langhorst/wgs-8.3rc1/Linux-amd64/bin/meryl -B -C -v -m 22 -memory 128000 -threads 16 -c 0 -L 2 -s /mnt/galaxy/data/langhorst/deer_unitigs_ca83//deer.gkpStore:chain -o /mnt/galaxy/data/langhorst/deer_unitigs_ca83//0-mercounts/deer-C-ms22-cm0 > /mnt/galaxy/data/langhorst/deer_unitigs_ca83//0-mercounts/meryl.err 2>&1
----------------------------------------END Mon Mar 23 20:43:51 2015 (1364 seconds)
ERROR: Failed with signal ABRT (6)
================================================================================
runCA failed.
----------------------------------------
Stack trace:
at /home/NEB/langhorst/wgs-8.3rc1/Linux-amd64/bin/runCA line 1653.
main::caFailure('meryl failed', '/mnt/galaxy/data/langhorst/deer_unitigs_ca83//0-mercounts/mer...') called at /home/NEB/langhorst/wgs-8.3rc1/Linux-amd64/bin/runCA line 2523
main::runMeryl(22, 0, '-C', 'auto', undef, undef, 'mbt', 0) called at /home/NEB/langhorst/wgs-8.3rc1/Linux-amd64/bin/runCA line 3088
main::merTrim() called at /home/NEB/langhorst/wgs-8.3rc1/Linux-amd64/bin/runCA line 6556
----------------------------------------
Last few lines of the relevant log file (/mnt/galaxy/data/langhorst/deer_unitigs_ca83//0-mercounts/meryl.err):
meryl: AS_MER_gkpStoreChain.C:69: gkpStoreChain::gkpStoreChain(const char*, uint32): Assertion `_numberOfSequences < _maxChains' failed.
Failed with 'Aborted'
Backtrace (mangled):
/home/NEB/langhorst/wgs-8.3rc1/Linux-amd64/bin/meryl(_Z17AS_UTL_catchCrashiP7siginfoPv+0x2a)[0x40d48a]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x10340)[0x7fb3ff878340]
/lib/x86_64-linux-gnu/libc.so.6(gsignal+0x39)[0x7fb3feaaacc9]
/lib/x86_64-linux-gnu/libc.so.6(abort+0x148)[0x7fb3feaae0d8]
/lib/x86_64-linux-gnu/libc.so.6(+0x2fb86)[0x7fb3feaa3b86]
/lib/x86_64-linux-gnu/libc.so.6(+0x2fc32)[0x7fb3feaa3c32]
/home/NEB/langhorst/wgs-8.3rc1/Linux-amd64/bin/meryl(_ZN13gkpStoreChainC2EPKcj+0x4d5)[0x40bd35]
/home/NEB/langhorst/wgs-8.3rc1/Linux-amd64/bin/meryl(_ZN13gkpStoreChain8openFileEPKc+0x62)[0x40be02]
/home/NEB/langhorst/wgs-8.3rc1/Linux-amd64/bin/meryl(_ZN10seqFactory8openFileEPKc+0x3c)[0x42d61c]
/home/NEB/langhorst/wgs-8.3rc1/Linux-amd64/bin/meryl(_ZN9seqStreamC2EPKc+0x2c)[0x42d9cc]
/home/NEB/langhorst/wgs-8.3rc1/Linux-amd64/bin/meryl(_Z12prepareBatchP9merylArgs+0xc9)[0x41e479]
/home/NEB/langhorst/wgs-8.3rc1/Linux-amd64/bin/meryl(_Z5buildP9merylArgs+0x685)[0x421775]
/home/NEB/langhorst/wgs-8.3rc1/Linux-amd64/bin/meryl(main+0xe7)[0x409cb7]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5)[0x7fb3fea95ec5]
/home/NEB/langhorst/wgs-8.3rc1/Linux-amd64/bin/meryl(__gxx_personality_v0+0x141)[0x409a99]
Backtrace (demangled):
[0] /home/NEB/langhorst/wgs-8.3rc1/Linux-amd64/bin/meryl::AS_UTL_catchCrash(int, siginfo*, void*) + 0x2a [0x40d48a]
[1] /lib/x86_64-linux-gnu/libpthread.so.0::(null) + 0x10340 [0x7fb3ff878340]
[2] /lib/x86_64-linux-gnu/libc.so.6::(null) + 0x39 [0x7fb3feaaacc9]
[3] /lib/x86_64-linux-gnu/libc.so.6::(null) + 0x148 [0x7fb3feaae0d8]
[4] /lib/x86_64-linux-gnu/libc.so.6::(null) + 0x2fb86 [0x7fb3feaa3b86]
[5] /lib/x86_64-linux-gnu/libc.so.6::(null) + 0x2fc32 [0x7fb3feaa3c32]
[6] /home/NEB/langhorst/wgs-8.3rc1/Linux-amd64/bin/meryl::gkpStoreChain::gkpStoreChain(char const*, unsigned int) + 0x4d5 [0x40bd35]
[7] /home/NEB/langhorst/wgs-8.3rc1/Linux-amd64/bin/meryl::gkpStoreChain::openFile(char const*) + 0x62 [0x40be02]
[8] /home/NEB/langhorst/wgs-8.3rc1/Linux-amd64/bin/meryl::seqFactory::openFile(char const*) + 0x3c [0x42d61c]
[9] /home/NEB/langhorst/wgs-8.3rc1/Linux-amd64/bin/meryl::seqStream::seqStream(char const*) + 0x2c [0x42d9cc]
[10] /home/NEB/langhorst/wgs-8.3rc1/Linux-amd64/bin/meryl::prepareBatch(merylArgs*) + 0xc9 [0x41e479]
[11] /home/NEB/langhorst/wgs-8.3rc1/Linux-amd64/bin/meryl::build(merylArgs*) + 0x685 [0x421775]
[12] /home/NEB/langhorst/wgs-8.3rc1/Linux-amd64/bin/meryl::(null) + 0xe7 [0x409cb7]
[13] /lib/x86_64-linux-gnu/libc.so.6::(null) + 0xf5 [0x7fb3fea95ec5]
[14] /home/NEB/langhorst/wgs-8.3rc1/Linux-amd64/bin/meryl::(null) + 0x141 [0x409a99]
--
Brad Langhorst, Ph.D.
Applications and Product Development Scientist
On Mar 23, 2015, at 4:32 PM, Walenz, Brian <wa...@nb...<mailto:wa...@nb...>> wrote:
Congratulations! You're the first to try (or announce trying) more than 2 billion reads.
We've run assemblies with up to 2 billion reads, but never more. I forget why we didn't allow up to 4 billion reads. Aside from silly printing errors like this, the are probably places where -1 is used to indicate an invalid ID, or where exactly 31 bits were available to store an ID.
I think I can spend a little time running a mock assembly with ~ 4 billion reads. This should catch most of the obvious issues.
The fix for this issue is simple. The easiest for you would be to change that perl function to always return 3902080154 (the number of reads you actually have). You can try continuing with that hack, or wait until I can run the mock assembly. The only risk is lost compute; every step of the assembler can be undone.
b
________________________________
From: Langhorst, Brad [Lan...@ne...<mailto:Lan...@ne...>]
Sent: Sunday, March 22, 2015 5:09 PM
To: wgs...@li...<mailto:wgs...@li...>
Subject: [wgs-assembler-users] last fragment id < 0 ... runCA can't match the output
Hi:
My frag store has finally been built, but runCA can’t continue because it can’t figure out the number of frags in the store.
Is it normal for the endIID to be < 0?
gatekeeper -lastfragiid deer.gkpStore
Last frag in store is iid = -392887142
runCA can’t handle that.
in getNumberOfFragsInStore, this regex fails to match due to the “-"
$numFrags = $1 if (m/^Last frag in store is iid = (\d+)$/);
I could fix that, but it doesn’t seem very reasonable to report a negative value from a function that is supposed to be counting the number of frags in a store.
Should I fix the getNumberOfFragsInStore function to just parse this file and return the “active” column? or is this an indication of some deeper problem? Where should I look next?
Thanks,
Brad
Here is the .info file:
libIID bgnIID endIID active deleted mated totLen clrLen libName
0 1 -392887142 3902080154 0 3902080154 589214103254 589214103254 GLOBAL
0 0 0 0 0 0 0 0 LegacyUnmatedReads
1 1 1321836050 1321836050 0 1321836050 199597243550 199597243550 run15
2 1321836051 2564548948 1242712898 0 1242712898 187649647598 187649647598 run16
3 2564548949 3902080154 1337531206 0 1337531206 201967212106 201967212106 run17
--
Brad Langhorst, Ph.D.
Applications and Product Development Scientist
|
