#326 failed to unitig

[Jason Tsai]

Hi,

I tried to assemble the Pacbio reads using the PbCR pipeline. It was all good until the unitigger stage. The last few lines are attached below. It would be great if some suggestions are given. My suspicion is that unitagger detected all the cores in the servers and tried to use them all (?) but obviously is restricted by how much RAM I requested. Best, Jason

OverlapCache()-- Loading overlap information
OverlapCache()-- Loading overlap information: overlaps processed 1550 (000.01%) loaded 591 (000.01%) (at read iid 2)
OverlapCache()-- Loading overlap information: overlaps processed 10824978 (100.00%) loaded 8256519 (076.27%)
setLogFile()-- Now logging to '/home/ishengtsai/Burx/Pacbio/KaCL4/4-unitigger/asm.002.bestOverlapGraph'

libgomp: Thread creation failed: Resource temporarily unavailable
WARNING: open file '/home/ishengtsai/Burx/Pacbio/KaCL4/4-unitigger/asm.002.bestOverlapGraph.thr000'

---

Failure message:

failed to unitig

----------------------------------------END Sun Oct 4 10:12:11 2015 (14268 seconds)
Failed to execute /lustre2/home/ishengtsai/bin/wgs-8.3rc1/Linux-amd64/bin/runCA -s /home/ishengtsai/Burx/Pacbio//tempKaCL4/KaCL4.spec -p asm -d KaCL4 ovlRefBlockLength=100000000000 ovlRefBlockSize=0 useGrid=0 scriptOnGrid=0 unitigger=bogart ovlErrorRate=0.10 utgErrorRate=0.06 cgwErrorRate=0.10 cnsErrorRate=0.10 utgGraphErrorLimit=0 utgGraphErrorRate=0.05 utgMergeErrorLimit=0 utgMergeErrorRate=0.05 frgCorrBatchSize=100000 doOverlapBasedTrimming=1 obtErrorRate=0.08 obtErrorLimit=4.5 frgMinLen=3000 ovlMinLen=100 "batOptions=-RS -NS -CS" consensus=pbutgcns merSize=22 cnsMaxCoverage=1 cnsReuseUnitigs=1 gridEnginePropagateHold="pBcR_asm" KaCL4.longest25.frg

[Brian Walenz]

That's a new one! Sounds like bogart is trying to use more threads than your limits allow.

1) Near the top of 4-unitigger/unitigger.err, the number of threads it is trying to use is reported. Post this file if possible - it also reports memory usage.
2) What's in the various /home/ishengtsai/Burx/Pacbio/KaCL4/4-unitigger/asm.002.bestOverlapGraph* files? These should show how far it got before failing.

Two things to try, both try to limit the number of threads it uses:

1) add batThreads=<some-number> to the runCA command, where <some-number> is the limit on the number of threads you can run.
2) set environment variable OMP_NUM_THREADS to the same number before running runCA.</some-number></some-number>

Unfortunately, it'll still take 4 hours to get to the point where it could fail.