Menu
▾
▴
wgs-assembler-users — Discussion about Celera Assembler
You can subscribe to this list here.
| 2012 |
Jan
(1) |
Feb
(2) |
Mar
|
Apr
(29) |
May
(8) |
Jun
(5) |
Jul
(46) |
Aug
(16) |
Sep
(5) |
Oct
(6) |
Nov
(17) |
Dec
(7) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2013 |
Jan
(5) |
Feb
(2) |
Mar
(10) |
Apr
(13) |
May
(20) |
Jun
(7) |
Jul
(6) |
Aug
(14) |
Sep
(9) |
Oct
(19) |
Nov
(17) |
Dec
(3) |
| 2014 |
Jan
(3) |
Feb
|
Mar
(7) |
Apr
(1) |
May
(1) |
Jun
(30) |
Jul
(10) |
Aug
(2) |
Sep
(18) |
Oct
(3) |
Nov
(4) |
Dec
(13) |
| 2015 |
Jan
(27) |
Feb
|
Mar
(19) |
Apr
(12) |
May
(10) |
Jun
(18) |
Jul
(4) |
Aug
(2) |
Sep
(2) |
Oct
|
Nov
(1) |
Dec
(9) |
| 2016 |
Jan
(6) |
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
(1) |
Aug
(1) |
Sep
(1) |
Oct
|
Nov
|
Dec
|
|
From: Richard B. <r....@qm...> - 2013-04-26 08:55:25
|
Hi all,
I am running CABOG for the first time on 4.3X 454 coverage of an 880Mb plant genome. It has stopped working at the end of the 79th iteration of "olap-from-seeds", due to a "segmentation fault". Can anyone advise me how to resume the assembly, please? I have tried restarting it but the problem recurs.
The end of my standard output was:
/home/btw434/work-yearly/Celera/wgs-7.0/Linux-amd64/bin/AshCelera1/0-overlaptrim-overlap/olap-from-seeds.sh 77 > /home/btw434/work-yearly/Celera/wgs-7.0/Linux-amd64/bin/AshCelera1/0-overlaptrim-overlap/olaps/0077.err 2>&1
/home/btw434/work-yearly/Celera/wgs-7.0/Linux-amd64/bin/AshCelera1/0-overlaptrim-overlap/olap-from-seeds.sh 78 > /home/btw434/work-yearly/Celera/wgs-7.0/Linux-amd64/bin/AshCelera1/0-overlaptrim-overlap/olaps/0078.err 2>&1
/home/btw434/work-yearly/Celera/wgs-7.0/Linux-amd64/bin/AshCelera1/0-overlaptrim-overlap/olap-from-seeds.sh 79 > /home/btw434/work-yearly/Celera/wgs-7.0/Linux-amd64/bin/AshCelera1/0-overlaptrim-overlap/olaps/0079.err 2>&1
----------------------------------------END CONCURRENT Wed Apr 24 12:32:33 2013 (1582382 seconds)
================================================================================
runCA failed.
----------------------------------------
Stack trace:
at ./runCA line 1237.
main::caFailure('olap-from-seeds failed. See *.err in /home/btw434/work-yearl...', undef) called at ./runCA line 3200
main::merOverlapper('trim') called at ./runCA line 3258
main::createOverlapJobs('trim') called at ./runCA line 3647
main::overlapTrim() called at ./runCA line 5876
----------------------------------------
Failure message:
olap-from-seeds failed. See *.err in /home/btw434/work-yearly/Celera/wgs-7.0/Linux-amd64/bin/AshCelera1/0-overlaptrim-overlap.
The tail of /home/btw434/work-yearly/Celera/wgs-7.0/Linux-amd64/bin/AshCelera1/0-overlaptrim-overlap/olaps/0079.err read:
Thread 1 processed 1170163 olaps at Wed Apr 24 12:21:07 2013
Thread 0 processed 1174288 olaps at Wed Apr 24 12:21:11 2013
Extracted 12711 of 12711 fragments in iid range 5900001 .. 5912794
Thread 1 processed 1374622 olaps at Wed Apr 24 12:29:16 2013
Thread 0 processed 1383950 olaps at Wed Apr 24 12:29:20 2013
Thread 1 processed 224905 olaps at Wed Apr 24 12:30:49 2013
Thread 0 processed 229187 olaps at Wed Apr 24 12:30:51 2013
Failed overlaps = 106922508
Start analyzing multi-alignments Num_Frags = 62794
/home/btw434/work-yearly/Celera/wgs-7.0/Linux-amd64/bin/AshCelera1/0-overlaptrim-overlap/olap-from-seeds.sh: line 63: 40962 Segmentation fault $bin/olap-from-seeds -a -b -t 2 -S /home/btw434/work-yearly/Celera/wgs-7.0/Linux-amd64/bin/AshCelera1/0-overlaptrim-overlap/Ash1.merStore -G -o /home/btw434/work-yearly/Celera/wgs-7.0/Linux-amd64/bin/AshCelera1/0-overlaptrim-overlap/olaps/$jobid.ovb.WORKING.gz /home/btw434/work-yearly/Celera/wgs-7.0/Linux-amd64/bin/AshCelera1/Ash1.gkpStore $minid $maxid
Do you know how I might solve this problem, please?
many thanks
Richard
_______
Dr Richard Buggs | Senior Lecturer | School of Biological and Chemical Sciences, Queen Mary University of London, E1 4NS, United Kingdom | email: r....@qm... | website: http://www.sbcs.qmul.ac.uk/staff/richardbuggs.html | office: +44(0)207 882 3058 | mobile: +44(0)772 992 0401 | twitter: @RJABuggs
|
|
From: Sergey K. <se...@um...> - 2013-04-25 00:45:40
|
Hi, That certainly doesn't look normal, especially the over 100% error rates. Could you post one of your asm.*.lay and asm.*.log files. Either one that failed or one that had over 100% error. Sergey On Apr 23, 2013, at 1:32 PM, Ole Kristian Tørresen <o.k...@ib...> wrote: > Hi, > I'm correcting some XLXL PacBio reads, and hit upon some failed assertions in make-consensus. I hope it's possible to solve. > > This is the first error (last 10 lines of the out file from the runPartition.sh): > with 100 errors (99.01% error) > In contig 814 forced alignment of 65 bases of string 48036185 subscript 344 to 2 bases of consensus > with 64 errors (98.46% error) > In contig 814 forced alignment of 398 bases of string 52752554 subscript 345 to 1 bases of consensus > with 397 errors (99.75% error) > In contig 814 forced alignment of 227 bases of string 56213438 subscript 346 to 18 bases of consensus > with 214 errors (94.27% error) > In contig 814 forced alignment of 112 bases of string 75693310 subscript 347 to 2 bases of consensus > with 110 errors (98.21% error) > make-consensus: align.cc:4903: void Complete_Align(const char*, int, int, const char*, int, int, int, int, int, int, int, Align_Score_Entry_t*, Align_Score_Entry_t&, Alignment_t&): Assertion `t_lo < t_slip' failed. > > 98.21 % error is a bit weird. Got any idea what that assertion is? > > > This is not an error, but even weirder: > with 62 errors (151.22% error) > In Reset_From_Votes in contig 1369 > Forced alignment of string subscript 685 to consensus > with 169 errors (174.23% error) > In Reset_From_Votes in contig 1369 > Forced alignment of string subscript 686 to consensus > with 154 errors (179.07% error) > In Reset_From_Votes in contig 1369 > Forced alignment of string subscript 687 to consensus > with 78 errors (185.71% error) > > 185.71 % error? That's a lot. > > This is another error: > In contig 1135 forced alignment of 82 bases of string 52574042 subscript 809 to 11 bases of consensus > with 74 errors (90.24% error) > In contig 1135 forced alignment of 225 bases of string 41625223 subscript 810 to 3 bases of consensus > with 222 errors (98.67% error) > In contig 1135 forced alignment of 146 bases of string 55029895 subscript 811 to 2 bases of consensus > with 144 errors (98.63% error) > In contig 1135 forced alignment of 140 bases of string 40943538 subscript 812 to 3 bases of consensus > with 137 errors (97.86% error) > ERROR: Bad from = 3 in Global_Align > in file align.cc at line 6872 errno = 0 > > How can I get this kind of errors in the alignments? > > The genome is about 830 Mbp (it's cod). Correcting with 454 reads, about 27x total. This is about 3.5x coverage in PacBio reads: > Running with 2536048074 bp for pblr_XLXL_cod. > Correcting with 22252248087 bp. > > Could something have gone wrong in the correctPacBio step? > > Thank you. > > Ole > > > ------------------------------------------------------------------------------ > Try New Relic Now & We'll Send You this Cool Shirt > New Relic is the only SaaS-based application performance monitoring service > that delivers powerful full stack analytics. Optimize and monitor your > browser, app, & servers with just a few lines of code. Try New Relic > and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_apr_______________________________________________ > wgs-assembler-users mailing list > wgs...@li... > https://lists.sourceforge.net/lists/listinfo/wgs-assembler-users |
|
From: Ole K. T. <o.k...@ib...> - 2013-04-23 17:53:23
|
Hi, I'm correcting some XLXL PacBio reads, and hit upon some failed assertions in make-consensus. I hope it's possible to solve. This is the first error (last 10 lines of the out file from the runPartition.sh): with 100 errors (99.01% error) In contig 814 forced alignment of 65 bases of string 48036185 subscript 344 to 2 bases of consensus with 64 errors (98.46% error) In contig 814 forced alignment of 398 bases of string 52752554 subscript 345 to 1 bases of consensus with 397 errors (99.75% error) In contig 814 forced alignment of 227 bases of string 56213438 subscript 346 to 18 bases of consensus with 214 errors (94.27% error) In contig 814 forced alignment of 112 bases of string 75693310 subscript 347 to 2 bases of consensus with 110 errors (98.21% error) make-consensus: align.cc:4903: void Complete_Align(const char*, int, int, const char*, int, int, int, int, int, int, int, Align_Score_Entry_t*, Align_Score_Entry_t&, Alignment_t&): Assertion `t_lo < t_slip' failed. 98.21 % error is a bit weird. Got any idea what that assertion is? This is not an error, but even weirder: with 62 errors (151.22% error) In Reset_From_Votes in contig 1369 Forced alignment of string subscript 685 to consensus with 169 errors (174.23% error) In Reset_From_Votes in contig 1369 Forced alignment of string subscript 686 to consensus with 154 errors (179.07% error) In Reset_From_Votes in contig 1369 Forced alignment of string subscript 687 to consensus with 78 errors (185.71% error) 185.71 % error? That's a lot. This is another error: In contig 1135 forced alignment of 82 bases of string 52574042 subscript 809 to 11 bases of consensus with 74 errors (90.24% error) In contig 1135 forced alignment of 225 bases of string 41625223 subscript 810 to 3 bases of consensus with 222 errors (98.67% error) In contig 1135 forced alignment of 146 bases of string 55029895 subscript 811 to 2 bases of consensus with 144 errors (98.63% error) In contig 1135 forced alignment of 140 bases of string 40943538 subscript 812 to 3 bases of consensus with 137 errors (97.86% error) ERROR: Bad from = 3 in Global_Align in file align.cc at line 6872 errno = 0 How can I get this kind of errors in the alignments? The genome is about 830 Mbp (it's cod). Correcting with 454 reads, about 27x total. This is about 3.5x coverage in PacBio reads: Running with 2536048074 bp for pblr_XLXL_cod. Correcting with 22252248087 bp. Could something have gone wrong in the correctPacBio step? Thank you. Ole |
|
From: Walenz, B. <bw...@jc...> - 2013-04-17 20:09:33
|
Due to changes at sourceforge, the code repository address was going to change. We took this opportunity to switch from CVS to Subversion. You can check out from subversion with: svn checkout svn://svn.code.sf.net/p/wgs-assembler/svn/trunk/src src Full directions are at: http://sourceforge.net/apps/mediawiki/wgs-assembler/index.php?title=Check_out_and_Compile The CVS repository is still active, but will disappear in the near future. bri -- Brian Walenz Senior Software Engineer J. Craig Venter Institute |
|
From: Hiroaki S. <hir...@af...> - 2013-04-11 13:52:42
|
Hi Ole,
Thank you so much for your immediate response. I have just restarted the
runCA and it seems the assembly is moving forward. Thanks! Let's see what
happens!
Best,
Hiro.
From: Ole Kristian Tørresen [mailto:o.k...@ib...]
Sent: Thursday, April 11, 2013 10:26 PM
To: Hiroaki Sakai
Cc: wgs...@li...
Subject: Re: [wgs-assembler-users] Unitig splitter failed
Hi Hiroaki.
You can skip the unitig splitter. I'm not sure what effects that will have
for your assembly, but I've done it a couple of times and it looks good.
You have to create the files splitUnitigs.out and consensus-fix.out in
the 5-consensus-split folder ("touch splitUnitigs.out; touch
consensus-fix.out"), and restart CA. Then CA will skip the splitting.
Ole
On 11 April 2013 13:31, Hiroaki Sakai <hir...@af...> wrote:
Hi,
I'm working of the de-novo genome assembly of a plant species and trying to
do a hybrid assembly combining PacBio reads (1.9Gbp) corrected by pacBioToCA
and 454 reads (11Gbp) by runCA. I'm using the CSV:
% ./runCA -version
CA version CVS TIP ($Id: AS_GKP_main.c,v 1.105 2012/09/13 17:41:13 skoren
Exp $).
CA version CVS TIP ($Id: AS_CGB_unitigger.c,v 1.45 2011/09/06 02:15:18
mkotelbajcvi Exp $).
CA version CVS TIP ($Id: BuildUnitigs.cc,v 1.88 2012/01/15 23:49:34
brianwalenz Exp $).
Using up to 12 OpenMP threads.
CA version CVS TIP ($Id: AS_CGW_main.c,v 1.116 2012/11/15 05:04:54
brianwalenz Exp $).
CA version CVS TIP ($Id: terminator.C,v 1.17 2012/09/10 08:58:11 brianwalenz
Exp $).
When I ran the runCA, it stopped during 5-consensus-split stage. The error
messages I got were:
----------------------------------------START Thu Apr 11 17:11:18 2013
/tool/wgs/Linux-amd64/bin/utgcnsfix \
-g /my_genome/454_pacbio/result/vigan_454.gkpStore \
-t /my_genome/454_pacbio/result/vigan_454.tigStore 2 \
> /my_genome/454_pacbio/result/5-consensus/consensus-fix.out 2>&1
----------------------------------------END Thu Apr 11 18:10:54 2013 (3576
seconds)
----------------------------------------START Thu Apr 11 18:10:54 2013
/tool/wgs/Linux-amd64/bin/tigStore \
-g /my_genome/454_pacbio/result/vigan_454.gkpStore \
-t /my_genome/454_pacbio/result/5-consensus-insert-sizes/vigan_454.tigStore
3 \
-d matepair -U \
> /my_genome/454_pacbio/result/5-consensus-insert-sizes/estimates.out 2>&1
----------------------------------------END Thu Apr 11 18:11:23 2013 (29
seconds)
----------------------------------------START Thu Apr 11 18:11:23 2013
/tool/wgs/Linux-amd64/bin/gatekeeper \
--edit
/my_genome/454_pacbio/result/5-consensus-insert-sizes/vigan_454.tigStore.dis
tupdate \
/my_genome/454_pacbio/result/vigan_454.gkpStore \
> /my_genome/454_pacbio/result/5-consensus-insert-sizes/updates.err 2>&1
----------------------------------------END Thu Apr 11 18:12:10 2013 (47
seconds)
----------------------------------------START Thu Apr 11 18:12:10 2013
/tool/wgs/Linux-amd64/bin/splitUnitigs \
-g /my_genome/454_pacbio/result/vigan_454.gkpStore \
-t /my_genome/454_pacbio/result/vigan_454.tigStore 3 \
> /my_genome/454_pacbio/result/5-consensus-split/splitUnitigs.out 2>&1
sh: line 3: 63004 Aborted (core dumped)
/tool/wgs/Linux-amd64/bin/splitUnitigs -g
/my_genome/454_pacbio/result/vigan_454.gkpStore -t
/my_genome/454_pacbio/result/vigan_454.tigStore 3 >
/my_genome/454_pacbio/result/5-consensus-split/splitUnitigs.out 2>&1
----------------------------------------END Thu Apr 11 18:58:30 2013 (2780
seconds)
ERROR: Failed with signal ABRT (6)
============================================================================
====
runCA failed.
----------------------------------------
Stack trace:
at /tool/wgs/Linux-amd64/bin/runCA line 1390
main::caFailure('Unitig splitter failed', '/my_genome...') called at
/tool/wgs/Linux-amd64/bin/runCA line 4824
main::postUnitiggerConsensus() called at
/tool/wgs/Linux-amd64/bin/runCA line 6144
----------------------------------------
Last few lines of the relevant log file
(/my_genome/454_pacbio/result/5-consensus-split/splitUnitigs.out.FAILED):
/lib64/libc.so.6(__libc_start_main+0xfd)[0x309dc1ecdd]
/tool/wgs/Linux-amd64/bin/splitUnitigs[0x408b69]
Backtrace (demangled):
[0] /tool/wgs/Linux-amd64/bin/splitUnitigs::AS_UTL_catchCrash(int, siginfo*,
void*) + 0x27 [0x40f857]
[1] /lib64/libpthread.so.0() [0x309e40f4a0]
[2] /lib64/libc.so.6::(null) + 0x35 [0x309dc32885]
[3] /lib64/libc.so.6::(null) + 0x175 [0x309dc34065]
[4] /lib64/libc.so.6() [0x309dc2b9fe]
[5] /lib64/libc.so.6::(null) + 0 [0x309dc2bac0]
[6] /tool/wgs/Linux-amd64/bin/splitUnitigs::unitigConsensus::alignFragment()
+ 0x3ee [0x42c28e]
[7] /tool/wgs/Linux-amd64/bin/splitUnitigs::MultiAlignUnitig(MultiAlignT*,
gkStore*, CNS_Options*, int*) + 0x578 [0x42db88]
[8] /tool/wgs/Linux-amd64/bin/splitUnitigs::(null) + 0x2c7b [0x40b8ab]
[9] /lib64/libc.so.6::(null) + 0xfd [0x309dc1ecdd]
[10] /tool/wgs/Linux-amd64/bin/splitUnitigs() [0x408b69]
GDB:
----------------------------------------
Failure message:
Unitig splitter failed
----------------------------------------
I would appreciate it if any of you would kindly give me any suggestions.
Best regards,
-------------
Hiroaki Sakai, PhD
National Institute of Agrobiological Sciences, Japan
hir...@af...
----------------------------------------------------------------------------
--
Precog is a next-generation analytics platform capable of advanced
analytics on semi-structured data. The platform includes APIs for building
apps and a phenomenal toolset for data science. Developers can use
our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter
_______________________________________________
wgs-assembler-users mailing list
wgs...@li...
https://lists.sourceforge.net/lists/listinfo/wgs-assembler-users
|
|
From: Ole K. T. <o.k...@ib...> - 2013-04-11 13:25:55
|
Hi Hiroaki.
You can skip the unitig splitter. I'm not sure what effects that will have
for your assembly, but I've done it a couple of times and it looks good.
You have to create the files splitUnitigs.out and consensus-fix.out in
the 5-consensus-split folder ("touch splitUnitigs.out; touch
consensus-fix.out"), and restart CA. Then CA will skip the splitting.
Ole
On 11 April 2013 13:31, Hiroaki Sakai <hir...@af...> wrote:
> Hi,
>
> I'm working of the de-novo genome assembly of a plant species and trying to
> do a hybrid assembly combining PacBio reads (1.9Gbp) corrected by
> pacBioToCA
> and 454 reads (11Gbp) by runCA. I'm using the CSV:
>
> % ./runCA -version
> CA version CVS TIP ($Id: AS_GKP_main.c,v 1.105 2012/09/13 17:41:13 skoren
> Exp $).
> CA version CVS TIP ($Id: AS_CGB_unitigger.c,v 1.45 2011/09/06 02:15:18
> mkotelbajcvi Exp $).
> CA version CVS TIP ($Id: BuildUnitigs.cc,v 1.88 2012/01/15 23:49:34
> brianwalenz Exp $).
> Using up to 12 OpenMP threads.
> CA version CVS TIP ($Id: AS_CGW_main.c,v 1.116 2012/11/15 05:04:54
> brianwalenz Exp $).
> CA version CVS TIP ($Id: terminator.C,v 1.17 2012/09/10 08:58:11
> brianwalenz
> Exp $).
>
> When I ran the runCA, it stopped during 5-consensus-split stage. The error
> messages I got were:
>
> ----------------------------------------START Thu Apr 11 17:11:18 2013
> /tool/wgs/Linux-amd64/bin/utgcnsfix \
> -g /my_genome/454_pacbio/result/vigan_454.gkpStore \
> -t /my_genome/454_pacbio/result/vigan_454.tigStore 2 \
> > /my_genome/454_pacbio/result/5-consensus/consensus-fix.out 2>&1
> ----------------------------------------END Thu Apr 11 18:10:54 2013 (3576
> seconds)
> ----------------------------------------START Thu Apr 11 18:10:54 2013
> /tool/wgs/Linux-amd64/bin/tigStore \
> -g /my_genome/454_pacbio/result/vigan_454.gkpStore \
> -t
> /my_genome/454_pacbio/result/5-consensus-insert-sizes/vigan_454.tigStore
> 3 \
> -d matepair -U \
> > /my_genome/454_pacbio/result/5-consensus-insert-sizes/estimates.out 2>&1
> ----------------------------------------END Thu Apr 11 18:11:23 2013 (29
> seconds)
> ----------------------------------------START Thu Apr 11 18:11:23 2013
> /tool/wgs/Linux-amd64/bin/gatekeeper \
> --edit
>
> /my_genome/454_pacbio/result/5-consensus-insert-sizes/vigan_454.tigStore.dis
> tupdate \
> /my_genome/454_pacbio/result/vigan_454.gkpStore \
> > /my_genome/454_pacbio/result/5-consensus-insert-sizes/updates.err 2>&1
> ----------------------------------------END Thu Apr 11 18:12:10 2013 (47
> seconds)
> ----------------------------------------START Thu Apr 11 18:12:10 2013
> /tool/wgs/Linux-amd64/bin/splitUnitigs \
> -g /my_genome/454_pacbio/result/vigan_454.gkpStore \
> -t /my_genome/454_pacbio/result/vigan_454.tigStore 3 \
> > /my_genome/454_pacbio/result/5-consensus-split/splitUnitigs.out 2>&1
> sh: line 3: 63004 Aborted (core dumped)
> /tool/wgs/Linux-amd64/bin/splitUnitigs -g
> /my_genome/454_pacbio/result/vigan_454.gkpStore -t
> /my_genome/454_pacbio/result/vigan_454.tigStore 3 >
> /my_genome/454_pacbio/result/5-consensus-split/splitUnitigs.out 2>&1
> ----------------------------------------END Thu Apr 11 18:58:30 2013 (2780
> seconds)
> ERROR: Failed with signal ABRT (6)
>
> ============================================================================
> ====
>
> runCA failed.
>
> ----------------------------------------
> Stack trace:
>
> at /tool/wgs/Linux-amd64/bin/runCA line 1390
> main::caFailure('Unitig splitter failed', '/my_genome...') called
> at
> /tool/wgs/Linux-amd64/bin/runCA line 4824
> main::postUnitiggerConsensus() called at
> /tool/wgs/Linux-amd64/bin/runCA line 6144
> ----------------------------------------
> Last few lines of the relevant log file
> (/my_genome/454_pacbio/result/5-consensus-split/splitUnitigs.out.FAILED):
>
> /lib64/libc.so.6(__libc_start_main+0xfd)[0x309dc1ecdd]
> /tool/wgs/Linux-amd64/bin/splitUnitigs[0x408b69]
>
> Backtrace (demangled):
>
> [0] /tool/wgs/Linux-amd64/bin/splitUnitigs::AS_UTL_catchCrash(int,
> siginfo*,
> void*) + 0x27 [0x40f857]
> [1] /lib64/libpthread.so.0() [0x309e40f4a0]
> [2] /lib64/libc.so.6::(null) + 0x35 [0x309dc32885]
> [3] /lib64/libc.so.6::(null) + 0x175 [0x309dc34065]
> [4] /lib64/libc.so.6() [0x309dc2b9fe]
> [5] /lib64/libc.so.6::(null) + 0 [0x309dc2bac0]
> [6]
> /tool/wgs/Linux-amd64/bin/splitUnitigs::unitigConsensus::alignFragment()
> + 0x3ee [0x42c28e]
> [7] /tool/wgs/Linux-amd64/bin/splitUnitigs::MultiAlignUnitig(MultiAlignT*,
> gkStore*, CNS_Options*, int*) + 0x578 [0x42db88]
> [8] /tool/wgs/Linux-amd64/bin/splitUnitigs::(null) + 0x2c7b [0x40b8ab]
> [9] /lib64/libc.so.6::(null) + 0xfd [0x309dc1ecdd]
> [10] /tool/wgs/Linux-amd64/bin/splitUnitigs() [0x408b69]
>
> GDB:
>
>
>
> ----------------------------------------
> Failure message:
>
> Unitig splitter failed
> ----------------------------------------
>
> I would appreciate it if any of you would kindly give me any suggestions.
>
> Best regards,
>
> -------------
> Hiroaki Sakai, PhD
> National Institute of Agrobiological Sciences, Japan
> hir...@af...
>
>
>
>
> ------------------------------------------------------------------------------
> Precog is a next-generation analytics platform capable of advanced
> analytics on semi-structured data. The platform includes APIs for building
> apps and a phenomenal toolset for data science. Developers can use
> our toolset for easy data analysis & visualization. Get a free account!
> http://www2.precog.com/precogplatform/slashdotnewsletter
> _______________________________________________
> wgs-assembler-users mailing list
> wgs...@li...
> https://lists.sourceforge.net/lists/listinfo/wgs-assembler-users
>
|
|
From: Hiroaki S. <hir...@af...> - 2013-04-11 12:06:48
|
Hi,
I'm working of the de-novo genome assembly of a plant species and trying to
do a hybrid assembly combining PacBio reads (1.9Gbp) corrected by pacBioToCA
and 454 reads (11Gbp) by runCA. I'm using the CSV:
% ./runCA -version
CA version CVS TIP ($Id: AS_GKP_main.c,v 1.105 2012/09/13 17:41:13 skoren
Exp $).
CA version CVS TIP ($Id: AS_CGB_unitigger.c,v 1.45 2011/09/06 02:15:18
mkotelbajcvi Exp $).
CA version CVS TIP ($Id: BuildUnitigs.cc,v 1.88 2012/01/15 23:49:34
brianwalenz Exp $).
Using up to 12 OpenMP threads.
CA version CVS TIP ($Id: AS_CGW_main.c,v 1.116 2012/11/15 05:04:54
brianwalenz Exp $).
CA version CVS TIP ($Id: terminator.C,v 1.17 2012/09/10 08:58:11 brianwalenz
Exp $).
When I ran the runCA, it stopped during 5-consensus-split stage. The error
messages I got were:
----------------------------------------START Thu Apr 11 17:11:18 2013
/tool/wgs/Linux-amd64/bin/utgcnsfix \
-g /my_genome/454_pacbio/result/vigan_454.gkpStore \
-t /my_genome/454_pacbio/result/vigan_454.tigStore 2 \
> /my_genome/454_pacbio/result/5-consensus/consensus-fix.out 2>&1
----------------------------------------END Thu Apr 11 18:10:54 2013 (3576
seconds)
----------------------------------------START Thu Apr 11 18:10:54 2013
/tool/wgs/Linux-amd64/bin/tigStore \
-g /my_genome/454_pacbio/result/vigan_454.gkpStore \
-t /my_genome/454_pacbio/result/5-consensus-insert-sizes/vigan_454.tigStore
3 \
-d matepair -U \
> /my_genome/454_pacbio/result/5-consensus-insert-sizes/estimates.out 2>&1
----------------------------------------END Thu Apr 11 18:11:23 2013 (29
seconds)
----------------------------------------START Thu Apr 11 18:11:23 2013
/tool/wgs/Linux-amd64/bin/gatekeeper \
--edit
/my_genome/454_pacbio/result/5-consensus-insert-sizes/vigan_454.tigStore.dis
tupdate \
/my_genome/454_pacbio/result/vigan_454.gkpStore \
> /my_genome/454_pacbio/result/5-consensus-insert-sizes/updates.err 2>&1
----------------------------------------END Thu Apr 11 18:12:10 2013 (47
seconds)
----------------------------------------START Thu Apr 11 18:12:10 2013
/tool/wgs/Linux-amd64/bin/splitUnitigs \
-g /my_genome/454_pacbio/result/vigan_454.gkpStore \
-t /my_genome/454_pacbio/result/vigan_454.tigStore 3 \
> /my_genome/454_pacbio/result/5-consensus-split/splitUnitigs.out 2>&1
sh: line 3: 63004 Aborted (core dumped)
/tool/wgs/Linux-amd64/bin/splitUnitigs -g
/my_genome/454_pacbio/result/vigan_454.gkpStore -t
/my_genome/454_pacbio/result/vigan_454.tigStore 3 >
/my_genome/454_pacbio/result/5-consensus-split/splitUnitigs.out 2>&1
----------------------------------------END Thu Apr 11 18:58:30 2013 (2780
seconds)
ERROR: Failed with signal ABRT (6)
============================================================================
====
runCA failed.
----------------------------------------
Stack trace:
at /tool/wgs/Linux-amd64/bin/runCA line 1390
main::caFailure('Unitig splitter failed', '/my_genome...') called at
/tool/wgs/Linux-amd64/bin/runCA line 4824
main::postUnitiggerConsensus() called at
/tool/wgs/Linux-amd64/bin/runCA line 6144
----------------------------------------
Last few lines of the relevant log file
(/my_genome/454_pacbio/result/5-consensus-split/splitUnitigs.out.FAILED):
/lib64/libc.so.6(__libc_start_main+0xfd)[0x309dc1ecdd]
/tool/wgs/Linux-amd64/bin/splitUnitigs[0x408b69]
Backtrace (demangled):
[0] /tool/wgs/Linux-amd64/bin/splitUnitigs::AS_UTL_catchCrash(int, siginfo*,
void*) + 0x27 [0x40f857]
[1] /lib64/libpthread.so.0() [0x309e40f4a0]
[2] /lib64/libc.so.6::(null) + 0x35 [0x309dc32885]
[3] /lib64/libc.so.6::(null) + 0x175 [0x309dc34065]
[4] /lib64/libc.so.6() [0x309dc2b9fe]
[5] /lib64/libc.so.6::(null) + 0 [0x309dc2bac0]
[6] /tool/wgs/Linux-amd64/bin/splitUnitigs::unitigConsensus::alignFragment()
+ 0x3ee [0x42c28e]
[7] /tool/wgs/Linux-amd64/bin/splitUnitigs::MultiAlignUnitig(MultiAlignT*,
gkStore*, CNS_Options*, int*) + 0x578 [0x42db88]
[8] /tool/wgs/Linux-amd64/bin/splitUnitigs::(null) + 0x2c7b [0x40b8ab]
[9] /lib64/libc.so.6::(null) + 0xfd [0x309dc1ecdd]
[10] /tool/wgs/Linux-amd64/bin/splitUnitigs() [0x408b69]
GDB:
----------------------------------------
Failure message:
Unitig splitter failed
----------------------------------------
I would appreciate it if any of you would kindly give me any suggestions.
Best regards,
-------------
Hiroaki Sakai, PhD
National Institute of Agrobiological Sciences, Japan
hir...@af...
|
|
From: wcyer <wcy...@16...> - 2013-03-28 08:46:51
|
Hi Brain,
I ran CABOG successfully until reach utgcnsfix.
Utgcnsfix failed, and in the 5-consensus there is a consensus-fix.out.FAILED file:
Checking unitig consensus for b=0 to e=14704513
Could not calloc memory (32 * 1 bytes = 32)
Unexpected error.
The server has 512Gb memory.
Did I miss something?
Thanks in advance,
Gorliver
Here is the last a few lines of output of runCA:
----------------------------------------START Thu Mar 28 16:17:30 2013
/share/home/caulai/software/wgs/wgs-7.0/Linux-amd64/bin/utgcnsfix \
-g /share/home/caulai/real_data/./assembly2/ib_ass2.gkpStore \
-t /share/home/caulai/real_data/./assembly2/ib_ass2.tigStore 2 \
> /share/home/caulai/real_data/./assembly2/5-consensus/consensus-fix.out 2>&1
sh: line 3: 17176 Aborted (core dumped) /share/home/caulai/software/wgs/wgs-7.0/Linux-amd64/bin/utgcnsfix -g /share/home/caulai/real_data/./assembly2/ib_ass2.gkpStore -t /
share/home/caulai/real_data/./assembly2/ib_ass2.tigStore 2 > /share/home/caulai/real_data/./assembly2/5-consensus/consensus-fix.out 2>&1
----------------------------------------END Thu Mar 28 16:20:06 2013 (156 seconds)
ERROR: Failed with signal ABRT (6)
================================================================================
runCA failed.
----------------------------------------
Stack trace:
at /share/home/caulai/software/wgs/wgs-7.0/Linux-amd64/bin/runCA line 1237
main::caFailure('Unitig consensus fixer failed', '/share/home/caulai/real_data/./...') called at /share/home/caulai/software/wgs/wgs-7.0/Linux-amd64/bin/runCA line 4532
main::postUnitiggerConsensus() called at /share/home/caulai/software/wgs/wgs-7.0/Linux-amd64/bin/runCA line 5883
----------------------------------------
Failure message:
Unitig consensus fixer failed
|
|
From: <cto...@ge...> - 2013-03-27 20:07:26
|
Hi, I have used Celera version 7.0 to perform de novo assemblies using Pacbio long reads error corrected with HGAP, but I was wondering if you can also use this version of Celera to perform a hybrid assembly using both Pacbio long reads and Illumina data? Thanks, Chad Tomlinson |
|
From: Walenz, B. <bw...@jc...> - 2013-03-25 09:26:33
|
Hi, Heiner, Ole- The transitive edge reduction was easy to parallelize (I actually did it as an escape from a harder problem). The rest of CGW suffers from _lots_ of thread unsafe code and inherently sequential algorithms. We'd like to parallelize, but its a huge project. On most genomes, the parallel pieces are fast, a few hours at the most. On some genomes, they get 'stuck' and will burn all CPUs for days. If you see this, drop me a line and I'll tell you how to get around it. I don't know what causes it, and the 'fix' damages other assemblies so isn't good to apply in general. Glad to hear the overlap store builder is working for you! It works fantastic if your filer can support the I/O. On my dev cluster, the original ovlStore is faster. b On 3/24/13 7:09 AM, "Ole Kristian Tørresen" <o.k...@ib...> wrote: > Hi Heiner, > only some parts of cgw is multithreaded now. Among them is transistive > edge reduction as you can see from the cvs log here: > http://wgs-assembler.cvs.sourceforge.net/viewvc/wgs-assembler/src/AS_CGW/Trans > itiveReduction_CGW.c?view=log > (April 5th). I'm not sure what plans Brian have, but I think it's > pretty hard to parallelize merge scaffolds aggressively, which is the > most time-consuming step for me now. > > Ole > > On 24 March 2013 11:30, kuhl <ku...@mo...> wrote: >> Dear Brian, >> >> I was just wondering if cgw can be run in a parallel mode. I found >> something saying "48 openmpi threads" in the cgw.out file, but it is still >> running on a single core. Can I somehow configure cgw to use more cores? I >> have installed the latest CA from CVS. I have just used the parallel >> ovlstore creation which is great, thanks for that. >> >> Best wishes, Heiner >> >> ----------------------------------------------------------------------------->> - >> Everyone hates slow websites. So do we. >> Make your web apps faster with AppDynamics >> Download AppDynamics Lite for free today: >> http://p.sf.net/sfu/appdyn_d2d_mar >> _______________________________________________ >> wgs-assembler-users mailing list >> wgs...@li... >> https://lists.sourceforge.net/lists/listinfo/wgs-assembler-users > > ------------------------------------------------------------------------------ > Everyone hates slow websites. So do we. > Make your web apps faster with AppDynamics > Download AppDynamics Lite for free today: > http://p.sf.net/sfu/appdyn_d2d_mar > _______________________________________________ > wgs-assembler-users mailing list > wgs...@li... > https://lists.sourceforge.net/lists/listinfo/wgs-assembler-users |
|
From: Ole K. T. <o.k...@ib...> - 2013-03-24 11:26:15
|
Hi Heiner, only some parts of cgw is multithreaded now. Among them is transistive edge reduction as you can see from the cvs log here: http://wgs-assembler.cvs.sourceforge.net/viewvc/wgs-assembler/src/AS_CGW/TransitiveReduction_CGW.c?view=log (April 5th). I'm not sure what plans Brian have, but I think it's pretty hard to parallelize merge scaffolds aggressively, which is the most time-consuming step for me now. Ole On 24 March 2013 11:30, kuhl <ku...@mo...> wrote: > Dear Brian, > > I was just wondering if cgw can be run in a parallel mode. I found > something saying "48 openmpi threads" in the cgw.out file, but it is still > running on a single core. Can I somehow configure cgw to use more cores? I > have installed the latest CA from CVS. I have just used the parallel > ovlstore creation which is great, thanks for that. > > Best wishes, Heiner > > ------------------------------------------------------------------------------ > Everyone hates slow websites. So do we. > Make your web apps faster with AppDynamics > Download AppDynamics Lite for free today: > http://p.sf.net/sfu/appdyn_d2d_mar > _______________________________________________ > wgs-assembler-users mailing list > wgs...@li... > https://lists.sourceforge.net/lists/listinfo/wgs-assembler-users |
|
From: kuhl <ku...@mo...> - 2013-03-24 10:46:32
|
Dear Brian, I was just wondering if cgw can be run in a parallel mode. I found something saying "48 openmpi threads" in the cgw.out file, but it is still running on a single core. Can I somehow configure cgw to use more cores? I have installed the latest CA from CVS. I have just used the parallel ovlstore creation which is great, thanks for that. Best wishes, Heiner |
|
From: wcyer <wcy...@16...> - 2013-03-16 07:57:46
|
Hi Brian, I assembled the mers with frequency>1600 as you suggested , these sequences are TEs, rRNA, and most of them are highly repetitive. The auto determined threshold 5633 could represent 72% of total mers. Is that saying, 72% of the genome is expected to be assembled? Should I set a large threshold capture most of the genome or set a smaller threshold to capture most of non-repeat sequences of the genome? Kindly regards, Gorliver. At 2013-03-15 01:40:00,"Walenz, Brian" <bw...@jc...> wrote: [seems the reply from Gorliver never made it to the list] The histogram (plot attached) is showing no hump at expected coverage. This could either be from high error in the reads, polymorphism in the sample, or low coverage causing noise and real sequence to overlap in the histogram. I’d go with a threshold of 1000, just because it was the historical default. The histogram is showing signs of lots of repeats. A threshold of 1519 is representing 99.9% of the distinct kmers, but only 59% of all kmers. Said another way, 0.1% of the kmers are representing 40% of the bases. You might want to run this through any of the greedy kmer assemblers to see what this is. CA has one: src/AS_MER/gkrpt.pl 200 < mers.fasta where 200 is the minimum size to report. mers.fasta can be any of the fasta files in 0-mercounts, or dumped: “meryl –Dt –n CUTOFF –s FILE” will report all kmers with count >= CUTOFF. For this task, a cutoff of 1519 (from above) or whatever you use for overlaps is appropriate. You’re just trying to see what sequence will not have overlaps because it is screen by the kmer masking. b On 3/14/13 9:45 AM, "Brian Walenz" <bw...@jc...> wrote: Hi- The threshold is plausible, but on the high side. In 0-mercounts, you can generate a histogram of mer counts using: meryl –Dh –s FILE, where FILE is the prefix of FILE.mcdat and FILE.mcidx. >From this histogram, you want to pick a value that captures most of the hump at your expected coverage, but isn’t so high that repeats will dominate. The assembler tries to do this using the last two columns in the histogram output (via a method that I struggle to explain with a white board and hand waving, so won’t attempt to do it here). The method does break down if there is no hump at expected coverage. Columns: frequency count fraction distinct fraction total An alternate way to pick the threshold is to argue that if you have 8x coverage, setting the threshold to 800 will capture overlaps for a 100 copy repeat. Hope this helped. If you want to send the histogram output, I can take a look at it. b On 3/14/13 9:26 AM, "wcyer" <wcy...@16...> wrote: Hi, everbody, I am using CABOG to assemble a genome about 2.5Gb . Before run real data, I did some simulated assembly (about 8X data) and the threshold of OBT and OVL is from 170 to 460. But when I run CABOG on the real data, the threshold is 5633: Reset OBT mer threshold from auto to 5633. Reset OVL mer threshold from auto to 5633. I see the default setting of the treshold is 'auto'. How CABOG determine this threshold? Is this number normal? Thanks in advance. Gorliver |
|
From: Walenz, B. <bw...@jc...> - 2013-03-14 13:45:59
|
Hi- The threshold is plausible, but on the high side. In 0-mercounts, you can generate a histogram of mer counts using: meryl –Dh –s FILE, where FILE is the prefix of FILE.mcdat and FILE.mcidx. From this histogram, you want to pick a value that captures most of the hump at your expected coverage, but isn’t so high that repeats will dominate. The assembler tries to do this using the last two columns in the histogram output (via a method that I struggle to explain with a white board and hand waving, so won’t attempt to do it here). The method does break down if there is no hump at expected coverage. Columns: frequency count fraction distinct fraction total An alternate way to pick the threshold is to argue that if you have 8x coverage, setting the threshold to 800 will capture overlaps for a 100 copy repeat. Hope this helped. If you want to send the histogram output, I can take a look at it. b On 3/14/13 9:26 AM, "wcyer" <wcy...@16...> wrote: Hi, everbody, I am using CABOG to assemble a genome about 2.5Gb . Before run real data, I did some simulated assembly (about 8X data) and the threshold of OBT and OVL is from 170 to 460. But when I run CABOG on the real data, the threshold is 5633: Reset OBT mer threshold from auto to 5633. Reset OVL mer threshold from auto to 5633. I see the default setting of the treshold is 'auto'. How CABOG determine this threshold? Is this number normal? Thanks in advance. Gorliver |
|
From: wcyer <wcy...@16...> - 2013-03-14 13:27:00
|
Hi, everbody, I am using CABOG to assemble a genome about 2.5Gb . Before run real data, I did some simulated assembly (about 8X data) and the threshold of OBT and OVL is from 170 to 460. But when I run CABOG on the real data, the threshold is 5633: Reset OBT mer threshold from auto to 5633. Reset OVL mer threshold from auto to 5633. I see the default setting of the treshold is 'auto'. How CABOG determine this threshold? Is this number normal? Thanks in advance. Gorliver |
|
From: Ole K. T. <o.k...@ib...> - 2013-02-19 07:09:27
|
Hi Sebastian. I'll try to answer some of your questions, see below. On 18 February 2013 16:52, Sebastian Juenemann <jue...@ce...> wrote: > Hi everybody, > > i want to use CELERA to assemble data from different platforms (PGM, > MiSeq, GS Junior) independently and i'm a bit confused about how to set > the parameters right. > > First off, does anyone already have experience using CELERA and Ion > Torrent PGM data? I would suggest to treat PGM data as 454 data in terms > of parameters but some shared experience could help. Also, as PGM > provides sff files i would like to know if CELERA is capable to handle > PGM based sff files or if i better should fastq exports. I don't think CA has explicit support for PGM data, so I think you should just treat it like 454 data. You could either use sffToCA (http://sourceforge.net/apps/mediawiki/wgs-assembler/index.php?title=SffToCA) or convert to fastq and use fastqToCA (http://sourceforge.net/apps/mediawiki/wgs-assembler/index.php?title=FastqToCA). I'm not sure which options you should use with sffToCA, which clear range etc. > > Next, which options should be set to which values in order to run CELERA > in a non-grid parallel environment. Say i do have a single machine > having 12 CPUs (sockets) each consisting of 4 cores and i want that each > consecutive step of CELERA is using all 48 cores. Is there one single > option for that or do i have to set all *Threads options individually > (e.g. 'ovlThreads=48', 'mbtThreads=48' etc.). Also, are all those steps > for which a thread number can be given really executed consecutive? > Regarding the memory consumption, are all *Memeory settings (as e.g. > 'merylMemory') process based? If so it's crucial to know, because if > relying heavily on multithreading the memory should be raised > accordingly. Would be something like memory=number_of_threads*400MB a > good rule of thumb estimate for those settings? For some steps you can set both the number of processes and number of threads. The number of threads should not affect memory much. I often use this for ovl: ovlConcurrency=4 ovlThreads=8 This will 4 concurrent processes, each using 8 threads. Memory consumption is sometimes harder to control. It's easy for meryl: merylMemory=32678 will let meryl use 32 GB RAM. For overlapper, I use ovlHashBits = 24 ovlHashBlockLength = 800000000 which will consume about 13 GB RAM per process, so with 4 processes (which I set above) this will consume a total of about 52 GB. > > There are also a lot of other options for which i could not find any > useful recommendation or documentation. Do i have to set all those >150 > options in order to get good results or does CELERA have good default > values based on the input data. This confusion already starts at what > overlapper and unitigger should i use for PGM and MiSeq (ovl, mer, umd > and utg, bog, bogart)? Where are the main differences? Does CELERA > determine the input type based on the filename or on the type given when > converting to FRG format (-type). Use ovl for all the different types, PGM, Illumina, 454, PacBio etc. For longer reads with not a huge amount of coverage, use bog, while for shorter reads with a lot of coverage, use bogart. Someone else needs to explain it better, but I usually use bogart for Illumina reads or Illumina reads in combination with other reads, while I use bog for 454 and PacBio reads. You have to set -type for all quality scores that's not sanger (but I think sanger is default on 454 and Illumina today, and probably PGM and PacBio too). You have to set -technology for everything that's not Illumina (MiSeq is Illumina). > > I know these are many questions but there also a lot of settings that > can be altered tweak, so i hope forgive me. Set the options you have a good guess about (all the concurrency and thread options for example) and then just try the default and see how it works. There's some pretty good explanations on the website (http://sourceforge.net/apps/mediawiki/wgs-assembler/index.php?title=RunCA), but you have to tweak it to your organism and server. You have to experiment a bit, and if it doesn't work, ask again on this mailing list describing what you've done, and someone will usually be able to come up with some good advice. Ole > > Tanks, > > Sebastian > > -- > Sebastian Jünemann > CeBiTec - Center for Biotechnology > Bielefeld University, D-33594 Bielefeld, Germany > Office: V6-147 -- Phone: +49-(0)521-106-4827 > eMail: jue...@Ce... > > ------------------------------------------------------------------------------ > The Go Parallel Website, sponsored by Intel - in partnership with Geeknet, > is your hub for all things parallel software development, from weekly thought > leadership blogs to news, videos, case studies, tutorials, tech docs, > whitepapers, evaluation guides, and opinion stories. Check out the most > recent posts - join the conversation now. http://goparallel.sourceforge.net/ > _______________________________________________ > wgs-assembler-users mailing list > wgs...@li... > https://lists.sourceforge.net/lists/listinfo/wgs-assembler-users |
|
From: Sebastian J. <jue...@Ce...> - 2013-02-18 16:17:35
|
Hi everybody, i want to use CELERA to assemble data from different platforms (PGM, MiSeq, GS Junior) independently and i'm a bit confused about how to set the parameters right. First off, does anyone already have experience using CELERA and Ion Torrent PGM data? I would suggest to treat PGM data as 454 data in terms of parameters but some shared experience could help. Also, as PGM provides sff files i would like to know if CELERA is capable to handle PGM based sff files or if i better should fastq exports. Next, which options should be set to which values in order to run CELERA in a non-grid parallel environment. Say i do have a single machine having 12 CPUs (sockets) each consisting of 4 cores and i want that each consecutive step of CELERA is using all 48 cores. Is there one single option for that or do i have to set all *Threads options individually (e.g. 'ovlThreads=48', 'mbtThreads=48' etc.). Also, are all those steps for which a thread number can be given really executed consecutive? Regarding the memory consumption, are all *Memeory settings (as e.g. 'merylMemory') process based? If so it's crucial to know, because if relying heavily on multithreading the memory should be raised accordingly. Would be something like memory=number_of_threads*400MB a good rule of thumb estimate for those settings? There are also a lot of other options for which i could not find any useful recommendation or documentation. Do i have to set all those >150 options in order to get good results or does CELERA have good default values based on the input data. This confusion already starts at what overlapper and unitigger should i use for PGM and MiSeq (ovl, mer, umd and utg, bog, bogart)? Where are the main differences? Does CELERA determine the input type based on the filename or on the type given when converting to FRG format (-type). I know these are many questions but there also a lot of settings that can be altered tweak, so i hope forgive me. Tanks, Sebastian -- Sebastian Jünemann CeBiTec - Center for Biotechnology Bielefeld University, D-33594 Bielefeld, Germany Office: V6-147 -- Phone: +49-(0)521-106-4827 eMail: jue...@Ce... |
|
From: Powers, J. <jp...@ex...> - 2013-01-23 15:24:00
|
Oops, our opt/bin was pointing to runCA from 7.0 while the pathMap was pointing to the latest download ("live"). I give the full runCA path for the "live" version and all is right in the world.
Thanks,
Jason
From: Sergey Koren [mailto:se...@um...]
Sent: Tuesday, January 22, 2013 9:44 AM
To: Powers, Jason
Cc: wgs...@li...
Subject: Re: [wgs-assembler-users] Again with submission issues
I am not sure why your estimate-mer-threshold is using the -g parameter. I just checked the latest CVS version of CA and it does not have the -g option being passed to estimate mer threshold:
2361 if ($merThresh eq "auto") {
2362 if (! -e "$ofile.estMerThresh.out") {
2363 $cmd = "$bin/estimate-mer-threshold ";
2364 $cmd .= " -m $ofile ";
2365 $cmd .= " > $ofile.estMerThresh.out ";
2366 $cmd .= "2> $ofile.estMerThresh.err";
2367
2368 stopBefore("meryl", $cmd);
2369
2370 if (runCommand("$wrk/0-mercounts", $cmd)) {
2371 rename "$ofile.estMerThresh.out", "$ofile.estMerThresh.out.FAILED";
2372 caFailure("estimate-mer-threshold failed", "$ofile.estMerThresh.err");
2373 }
It used to have a -g option in CA version 7.0 so maybe your AS_MER directory got updated but not your AS_RUN? It looks like the change happened a while ago (March 2012). I would double-check that you are at the latest version of AS_RUN directory and don't have any modifications that would have prevented an update. I would also update the kmer package you are using to build CA (kmer in wgs-assembler) and see if that fixes the issue.
Sergey
On Jan 19, 2013, at 10:15 AM, Powers, Jason wrote:
Okay, interesting. Thanks for the info. I had noticed that it seemed to continue with several steps even though it quit. However, it didn't seem to get all the way done, and got stuck at a few steps. I will have to better categorize why it didn't proceed and get back to you. However just manually running the program repeatedly got it to finally finish.
However, now I have a runCA error cropping up that I don't understand, it's related to the estimate-mer-threshold command. It doesn't recognize the -g parameter. I don't see anywhere in the spec file or elsewhere that I dictate this parameter, so I think it is something internal to the program?
/opt/bin/wgs-bin/estimate-mer-threshold -g /[PATH]/specfile_9.spec/specfile_9.spec.gkpStore:chain -m /[PATH]/specfile_9.spec/0-mercounts/specfile_9.spec-C-ms22-cm0 > /[PATH]/specfile_9.spec/0-mercounts/specfile_9.spec-C-ms22-cm0.estMerThresh.out 2> /[PATH]/specfile_9.spec/0-mercounts/specfile_9.spec-C-ms22-cm0.estMerThresh.err
----------------------------------------END Fri Jan 18 17:45:23 2013 (0 seconds)
ERROR: Failed with signal HUP (1)
================================================================================
runCA failed.
----------------------------------------
Stack trace:
at /opt/bin/runCA line 1237.
main::caFailure('estimate-mer-threshold failed', '/[PATH]/BL21-DE3_long...') called at /opt/bin/runCA line 2218
main::runMeryl(22, 0, '-C', 'auto', 'ovl', 1) called at /opt/bin/runCA line 2350
main::meryl() called at /opt/bin/runCA line 3280
main::createOverlapJobs('trim') called at /opt/bin/runCA line 3647
main::overlapTrim() called at /opt/bin/runCA line 5876
----------------------------------------
Last few lines of the relevant log file (/[PATH]/specfile_9.spec/0-mercounts/specfile_9.spec-C-ms22-cm0.estMerThresh.err):
unknown option '-g'
unknown option '/[PATH]/specfile_9.spec/specfile_9.spec.gkpStore:chain'
usage: /opt/bin/wgs-bin/estimate-mer-threshold -m mercounts
-m mercounts file of mercounts
----------------------------------------
Failure message:
estimate-mer-threshold failed
From: Sergey Koren [mailto:se...@um...]
Sent: Friday, January 18, 2013 5:30 PM
To: Powers, Jason
Cc: wgs...@li...<mailto:wgs...@li...>
Subject: Re: [wgs-assembler-users] Again with submission issues
Hi,
The latest version of the pacBioToCA pipeline does not use (or require -sync y). Instead, it functions like runCA, submitting itself to the grid immediately after launch and then using the qhold option to control dependencies. It is the expected behavior for the job to quit as soon as it has submitted itself to the grid. The final output before quitting should be:
Your job 1544850 ("pBcR_asm_ccsman25") has been submitted
You can use this ID to see the job running on the grid. The file runPBcR.sge.out.00 will also record the output of the submitted job so you can track progress.
Sergey
On Jan 17, 2013, at 5:23 PM, Powers, Jason wrote:
Hi all,
A few months ago I wrote in about having problems getting distributed to computing to work with our job submission system. I am happy to say we got all the kinks ironed out between qsub and our system. Unfortunately we recently upgraded to the latest checkout of pacBioToCA (as of December 11), and this seems to have broken things a bit.
I run pacBioToCA and it submits the runPBcR.sge.out.00.sh job and just quits. I looked at the log and it is calling it via qsub as follows:
qsub -A assembly -pe threads 4 -cwd -N "pBcR_asm" -j y -o /mnt/Celera/BL21-DE3/MiSeq//tempBL21-DE3_long.clipped.corrected/runPBcR.sge.out.00 /mnt/Celera/BL21-DE3/MiSeq//tempBL21-DE3_long.clipped.corrected/runPBcR.sge.out.00.sh
I noticed that there wasn't a sync command or a hold_jid in this qsub call...is that correct? I looked at an old "out" file, and it did indeed have the sync
qsub -A assembly -sync y -pe threads 4 -cwd -N "rCA_asm" -j y -o /mnt/50X_Illum/20X_PB/temp94-3025.LongReads.20X.corrected/runCA.sge.out.00 /mnt/50X_Illum/20X_PB/temp94-3025.LongReads.20X.corrected/runCA.sge.out.00.sh
So I edited my spec file so that instead of this
sge = -A assembly
it says
sge = -A assembly -sync y
It now "looks" correct
qsub -A assembly -sync y -pe threads 4 -cwd -N "pBcR_asm" -j y -o /mnt/Celera/BL21-DE3/Ion//tempBL21-DE3_long.clipped.corrected/runPBcR.sge.out.00 /mnt/Celera/BL21-DE3/Ion//tempBL21-DE3_long.clipped.corrected/runPBcR.sge.out.00.sh
Although now it seems to just hang.
So any ideas?
Thanks,
Jason
------------------------------------------------------------------------------
Master Visual Studio, SharePoint, SQL, ASP.NET<http://ASP.NET>, C# 2012, HTML5, CSS,
MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current
with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft
MVPs and experts. ON SALE this month only -- learn more at:
http://p.sf.net/sfu/learnmore_122712_______________________________________________
wgs-assembler-users mailing list
wgs...@li...<mailto:wgs...@li...>
https://lists.sourceforge.net/lists/listinfo/wgs-assembler-users
------------------------------------------------------------------------------
Master Visual Studio, SharePoint, SQL, ASP.NET<http://ASP.NET>, C# 2012, HTML5, CSS,
MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current
with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft
MVPs and experts. SALE $99.99 this month only -- learn more at:
http://p.sf.net/sfu/learnmore_122912_______________________________________________
wgs-assembler-users mailing list
wgs...@li...<mailto:wgs...@li...>
https://lists.sourceforge.net/lists/listinfo/wgs-assembler-users
|
|
From: Sergey K. <se...@um...> - 2013-01-22 15:17:51
|
I am not sure why your estimate-mer-threshold is using the -g parameter. I just checked the latest CVS version of CA and it does not have the -g option being passed to estimate mer threshold:
2361 if ($merThresh eq "auto") {
2362 if (! -e "$ofile.estMerThresh.out") {
2363 $cmd = "$bin/estimate-mer-threshold ";
2364 $cmd .= " -m $ofile ";
2365 $cmd .= " > $ofile.estMerThresh.out ";
2366 $cmd .= "2> $ofile.estMerThresh.err";
2367
2368 stopBefore("meryl", $cmd);
2369
2370 if (runCommand("$wrk/0-mercounts", $cmd)) {
2371 rename "$ofile.estMerThresh.out", "$ofile.estMerThresh.out.FAILED";
2372 caFailure("estimate-mer-threshold failed", "$ofile.estMerThresh.err");
2373 }
It used to have a -g option in CA version 7.0 so maybe your AS_MER directory got updated but not your AS_RUN? It looks like the change happened a while ago (March 2012). I would double-check that you are at the latest version of AS_RUN directory and don't have any modifications that would have prevented an update. I would also update the kmer package you are using to build CA (kmer in wgs-assembler) and see if that fixes the issue.
Sergey
On Jan 19, 2013, at 10:15 AM, Powers, Jason wrote:
Okay, interesting. Thanks for the info. I had noticed that it seemed to continue with several steps even though it quit. However, it didn’t seem to get all the way done, and got stuck at a few steps. I will have to better categorize why it didn’t proceed and get back to you. However just manually running the program repeatedly got it to finally finish.
However, now I have a runCA error cropping up that I don’t understand, it's related to the estimate-mer-threshold command. It doesn't recognize the -g parameter. I don't see anywhere in the spec file or elsewhere that I dictate this parameter, so I think it is something internal to the program?
/opt/bin/wgs-bin/estimate-mer-threshold -g /[PATH]/specfile_9.spec/specfile_9.spec.gkpStore:chain -m /[PATH]/specfile_9.spec/0-mercounts/specfile_9.spec-C-ms22-cm0 > /[PATH]/specfile_9.spec/0-mercounts/specfile_9.spec-C-ms22-cm0.estMerThresh.out 2> /[PATH]/specfile_9.spec/0-mercounts/specfile_9.spec-C-ms22-cm0.estMerThresh.err
----------------------------------------END Fri Jan 18 17:45:23 2013 (0 seconds)
ERROR: Failed with signal HUP (1)
================================================================================
runCA failed.
----------------------------------------
Stack trace:
at /opt/bin/runCA line 1237.
main::caFailure('estimate-mer-threshold failed', '/[PATH]/BL21-DE3_long...') called at /opt/bin/runCA line 2218
main::runMeryl(22, 0, '-C', 'auto', 'ovl', 1) called at /opt/bin/runCA line 2350
main::meryl() called at /opt/bin/runCA line 3280
main::createOverlapJobs('trim') called at /opt/bin/runCA line 3647
main::overlapTrim() called at /opt/bin/runCA line 5876
----------------------------------------
Last few lines of the relevant log file (/[PATH]/specfile_9.spec/0-mercounts/specfile_9.spec-C-ms22-cm0.estMerThresh.err):
unknown option '-g'
unknown option '/[PATH]/specfile_9.spec/specfile_9.spec.gkpStore:chain'
usage: /opt/bin/wgs-bin/estimate-mer-threshold -m mercounts
-m mercounts file of mercounts
----------------------------------------
Failure message:
estimate-mer-threshold failed
From: Sergey Koren [mailto:se...@um...]
Sent: Friday, January 18, 2013 5:30 PM
To: Powers, Jason
Cc: wgs...@li...<mailto:wgs...@li...>
Subject: Re: [wgs-assembler-users] Again with submission issues
Hi,
The latest version of the pacBioToCA pipeline does not use (or require -sync y). Instead, it functions like runCA, submitting itself to the grid immediately after launch and then using the qhold option to control dependencies. It is the expected behavior for the job to quit as soon as it has submitted itself to the grid. The final output before quitting should be:
Your job 1544850 ("pBcR_asm_ccsman25") has been submitted
You can use this ID to see the job running on the grid. The file runPBcR.sge.out.00 will also record the output of the submitted job so you can track progress.
Sergey
On Jan 17, 2013, at 5:23 PM, Powers, Jason wrote:
Hi all,
A few months ago I wrote in about having problems getting distributed to computing to work with our job submission system. I am happy to say we got all the kinks ironed out between qsub and our system. Unfortunately we recently upgraded to the latest checkout of pacBioToCA (as of December 11), and this seems to have broken things a bit.
I run pacBioToCA and it submits the runPBcR.sge.out.00.sh job and just quits. I looked at the log and it is calling it via qsub as follows:
qsub -A assembly -pe threads 4 -cwd -N "pBcR_asm" -j y -o /mnt/Celera/BL21-DE3/MiSeq//tempBL21-DE3_long.clipped.corrected/runPBcR.sge.out.00 /mnt/Celera/BL21-DE3/MiSeq//tempBL21-DE3_long.clipped.corrected/runPBcR.sge.out.00.sh
I noticed that there wasn’t a sync command or a hold_jid in this qsub call…is that correct? I looked at an old “out” file, and it did indeed have the sync
qsub -A assembly -sync y -pe threads 4 -cwd -N "rCA_asm" -j y -o /mnt/50X_Illum/20X_PB/temp94-3025.LongReads.20X.corrected/runCA.sge.out.00 /mnt/50X_Illum/20X_PB/temp94-3025.LongReads.20X.corrected/runCA.sge.out.00.sh
So I edited my spec file so that instead of this
sge = -A assembly
it says
sge = -A assembly –sync y
It now “looks” correct
qsub -A assembly -sync y -pe threads 4 -cwd -N "pBcR_asm" -j y -o /mnt/Celera/BL21-DE3/Ion//tempBL21-DE3_long.clipped.corrected/runPBcR.sge.out.00 /mnt/Celera/BL21-DE3/Ion//tempBL21-DE3_long.clipped.corrected/runPBcR.sge.out.00.sh
Although now it seems to just hang.
So any ideas?
Thanks,
Jason
------------------------------------------------------------------------------
Master Visual Studio, SharePoint, SQL, ASP.NET<http://ASP.NET>, C# 2012, HTML5, CSS,
MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current
with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft
MVPs and experts. ON SALE this month only -- learn more at:
http://p.sf.net/sfu/learnmore_122712_______________________________________________
wgs-assembler-users mailing list
wgs...@li...<mailto:wgs...@li...>
https://lists.sourceforge.net/lists/listinfo/wgs-assembler-users
------------------------------------------------------------------------------
Master Visual Studio, SharePoint, SQL, ASP.NET<http://ASP.NET>, C# 2012, HTML5, CSS,
MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current
with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft
MVPs and experts. SALE $99.99 this month only -- learn more at:
http://p.sf.net/sfu/learnmore_122912_______________________________________________
wgs-assembler-users mailing list
wgs...@li...<mailto:wgs...@li...>
https://lists.sourceforge.net/lists/listinfo/wgs-assembler-users
|
|
From: Powers, J. <jp...@ex...> - 2013-01-19 15:15:14
|
Okay, interesting. Thanks for the info. I had noticed that it seemed to continue with several steps even though it quit. However, it didn’t seem to get all the way done, and got stuck at a few steps. I will have to better categorize why it didn’t proceed and get back to you. However just manually running the program repeatedly got it to finally finish.
However, now I have a runCA error cropping up that I don’t understand, it's related to the estimate-mer-threshold command. It doesn't recognize the -g parameter. I don't see anywhere in the spec file or elsewhere that I dictate this parameter, so I think it is something internal to the program?
/opt/bin/wgs-bin/estimate-mer-threshold -g /[PATH]/specfile_9.spec/specfile_9.spec.gkpStore:chain -m /[PATH]/specfile_9.spec/0-mercounts/specfile_9.spec-C-ms22-cm0 > /[PATH]/specfile_9.spec/0-mercounts/specfile_9.spec-C-ms22-cm0.estMerThresh.out 2> /[PATH]/specfile_9.spec/0-mercounts/specfile_9.spec-C-ms22-cm0.estMerThresh.err
----------------------------------------END Fri Jan 18 17:45:23 2013 (0 seconds)
ERROR: Failed with signal HUP (1)
================================================================================
runCA failed.
----------------------------------------
Stack trace:
at /opt/bin/runCA line 1237.
main::caFailure('estimate-mer-threshold failed', '/[PATH]/BL21-DE3_long...') called at /opt/bin/runCA line 2218
main::runMeryl(22, 0, '-C', 'auto', 'ovl', 1) called at /opt/bin/runCA line 2350
main::meryl() called at /opt/bin/runCA line 3280
main::createOverlapJobs('trim') called at /opt/bin/runCA line 3647
main::overlapTrim() called at /opt/bin/runCA line 5876
----------------------------------------
Last few lines of the relevant log file (/[PATH]/specfile_9.spec/0-mercounts/specfile_9.spec-C-ms22-cm0.estMerThresh.err):
unknown option '-g'
unknown option '/[PATH]/specfile_9.spec/specfile_9.spec.gkpStore:chain'
usage: /opt/bin/wgs-bin/estimate-mer-threshold -m mercounts
-m mercounts file of mercounts
----------------------------------------
Failure message:
estimate-mer-threshold failed
From: Sergey Koren [mailto:se...@um...]
Sent: Friday, January 18, 2013 5:30 PM
To: Powers, Jason
Cc: wgs...@li...
Subject: Re: [wgs-assembler-users] Again with submission issues
Hi,
The latest version of the pacBioToCA pipeline does not use (or require -sync y). Instead, it functions like runCA, submitting itself to the grid immediately after launch and then using the qhold option to control dependencies. It is the expected behavior for the job to quit as soon as it has submitted itself to the grid. The final output before quitting should be:
Your job 1544850 ("pBcR_asm_ccsman25") has been submitted
You can use this ID to see the job running on the grid. The file runPBcR.sge.out.00 will also record the output of the submitted job so you can track progress.
Sergey
On Jan 17, 2013, at 5:23 PM, Powers, Jason wrote:
Hi all,
A few months ago I wrote in about having problems getting distributed to computing to work with our job submission system. I am happy to say we got all the kinks ironed out between qsub and our system. Unfortunately we recently upgraded to the latest checkout of pacBioToCA (as of December 11), and this seems to have broken things a bit.
I run pacBioToCA and it submits the runPBcR.sge.out.00.sh job and just quits. I looked at the log and it is calling it via qsub as follows:
qsub -A assembly -pe threads 4 -cwd -N "pBcR_asm" -j y -o /mnt/Celera/BL21-DE3/MiSeq//tempBL21-DE3_long.clipped.corrected/runPBcR.sge.out.00 /mnt/Celera/BL21-DE3/MiSeq//tempBL21-DE3_long.clipped.corrected/runPBcR.sge.out.00.sh
I noticed that there wasn’t a sync command or a hold_jid in this qsub call…is that correct? I looked at an old “out” file, and it did indeed have the sync
qsub -A assembly -sync y -pe threads 4 -cwd -N "rCA_asm" -j y -o /mnt/50X_Illum/20X_PB/temp94-3025.LongReads.20X.corrected/runCA.sge.out.00 /mnt/50X_Illum/20X_PB/temp94-3025.LongReads.20X.corrected/runCA.sge.out.00.sh
So I edited my spec file so that instead of this
sge = -A assembly
it says
sge = -A assembly –sync y
It now “looks” correct
qsub -A assembly -sync y -pe threads 4 -cwd -N "pBcR_asm" -j y -o /mnt/Celera/BL21-DE3/Ion//tempBL21-DE3_long.clipped.corrected/runPBcR.sge.out.00 /mnt/Celera/BL21-DE3/Ion//tempBL21-DE3_long.clipped.corrected/runPBcR.sge.out.00.sh
Although now it seems to just hang.
So any ideas?
Thanks,
Jason
------------------------------------------------------------------------------
Master Visual Studio, SharePoint, SQL, ASP.NET<http://ASP.NET>, C# 2012, HTML5, CSS,
MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current
with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft
MVPs and experts. ON SALE this month only -- learn more at:
http://p.sf.net/sfu/learnmore_122712_______________________________________________
wgs-assembler-users mailing list
wgs...@li...<mailto:wgs...@li...>
https://lists.sourceforge.net/lists/listinfo/wgs-assembler-users
|
|
From: Sergey K. <se...@um...> - 2013-01-18 23:04:27
|
Hi,
The latest version of the pacBioToCA pipeline does not use (or require -sync y). Instead, it functions like runCA, submitting itself to the grid immediately after launch and then using the qhold option to control dependencies. It is the expected behavior for the job to quit as soon as it has submitted itself to the grid. The final output before quitting should be:
Your job 1544850 ("pBcR_asm_ccsman25") has been submitted
You can use this ID to see the job running on the grid. The file runPBcR.sge.out.00 will also record the output of the submitted job so you can track progress.
Sergey
On Jan 17, 2013, at 5:23 PM, Powers, Jason wrote:
Hi all,
A few months ago I wrote in about having problems getting distributed to computing to work with our job submission system. I am happy to say we got all the kinks ironed out between qsub and our system. Unfortunately we recently upgraded to the latest checkout of pacBioToCA (as of December 11), and this seems to have broken things a bit.
I run pacBioToCA and it submits the runPBcR.sge.out.00.sh job and just quits. I looked at the log and it is calling it via qsub as follows:
qsub -A assembly -pe threads 4 -cwd -N "pBcR_asm" -j y -o /mnt/Celera/BL21-DE3/MiSeq//tempBL21-DE3_long.clipped.corrected/runPBcR.sge.out.00 /mnt/Celera/BL21-DE3/MiSeq//tempBL21-DE3_long.clipped.corrected/runPBcR.sge.out.00.sh
I noticed that there wasn’t a sync command or a hold_jid in this qsub call…is that correct? I looked at an old “out” file, and it did indeed have the sync
qsub -A assembly -sync y -pe threads 4 -cwd -N "rCA_asm" -j y -o /mnt/50X_Illum/20X_PB/temp94-3025.LongReads.20X.corrected/runCA.sge.out.00 /mnt/50X_Illum/20X_PB/temp94-3025.LongReads.20X.corrected/runCA.sge.out.00.sh
So I edited my spec file so that instead of this
sge = -A assembly
it says
sge = -A assembly –sync y
It now “looks” correct
qsub -A assembly -sync y -pe threads 4 -cwd -N "pBcR_asm" -j y -o /mnt/Celera/BL21-DE3/Ion//tempBL21-DE3_long.clipped.corrected/runPBcR.sge.out.00 /mnt/Celera/BL21-DE3/Ion//tempBL21-DE3_long.clipped.corrected/runPBcR.sge.out.00.sh
Although now it seems to just hang.
So any ideas?
Thanks,
Jason
------------------------------------------------------------------------------
Master Visual Studio, SharePoint, SQL, ASP.NET<http://ASP.NET>, C# 2012, HTML5, CSS,
MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current
with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft
MVPs and experts. ON SALE this month only -- learn more at:
http://p.sf.net/sfu/learnmore_122712_______________________________________________
wgs-assembler-users mailing list
wgs...@li...<mailto:wgs...@li...>
https://lists.sourceforge.net/lists/listinfo/wgs-assembler-users
|
|
From: Powers, J. <jp...@ex...> - 2013-01-17 22:36:08
|
Hi all, A few months ago I wrote in about having problems getting distributed to computing to work with our job submission system. I am happy to say we got all the kinks ironed out between qsub and our system. Unfortunately we recently upgraded to the latest checkout of pacBioToCA (as of December 11), and this seems to have broken things a bit. I run pacBioToCA and it submits the runPBcR.sge.out.00.sh job and just quits. I looked at the log and it is calling it via qsub as follows: qsub -A assembly -pe threads 4 -cwd -N "pBcR_asm" -j y -o /mnt/Celera/BL21-DE3/MiSeq//tempBL21-DE3_long.clipped.corrected/runPBcR.sge.out.00 /mnt/Celera/BL21-DE3/MiSeq//tempBL21-DE3_long.clipped.corrected/runPBcR.sge.out.00.sh I noticed that there wasn't a sync command or a hold_jid in this qsub call...is that correct? I looked at an old "out" file, and it did indeed have the sync qsub -A assembly -sync y -pe threads 4 -cwd -N "rCA_asm" -j y -o /mnt/50X_Illum/20X_PB/temp94-3025.LongReads.20X.corrected/runCA.sge.out.00 /mnt/50X_Illum/20X_PB/temp94-3025.LongReads.20X.corrected/runCA.sge.out.00.sh So I edited my spec file so that instead of this sge = -A assembly it says sge = -A assembly -sync y It now "looks" correct qsub -A assembly -sync y -pe threads 4 -cwd -N "pBcR_asm" -j y -o /mnt/Celera/BL21-DE3/Ion//tempBL21-DE3_long.clipped.corrected/runPBcR.sge.out.00 /mnt/Celera/BL21-DE3/Ion//tempBL21-DE3_long.clipped.corrected/runPBcR.sge.out.00.sh Although now it seems to just hang. So any ideas? Thanks, Jason |
|
From: Walenz, B. <bw...@jc...> - 2012-12-11 22:45:08
|
Hi Audrey- Very strange. It computed a non-positive overlap length! This could be a corrupt overlap store, but lets hope not. I added some debug code to both the bog and bogart unitiggers, but you’ll need to upgrade to the CVS version to get it. I also made it accept that the overlap is problematic, and just not use it, instead of crashing. It will output warning messages (to unitigger.err) when this happens. To restart, just delete the 4-unitigger directory and run runCA from the newer code base. If there are only a few warnings, you’re probably OK. Losing a few overlaps won’t kill the assembly. If there are lots, overlaps will need to be recomputed to rebuild the store. As for the spec, the only change I’d make is to use bogart instead of bog. I don’t see any dnc* options -- did you clean up the PE/MP confusion in the long-insert Illumina? (http://sourceforge.net/apps/mediawiki/wgs-assembler/index.php?title=Pair_classification_within_Illumina_mate_pair_data) b -- Brian Walenz Senior Software Engineer J. Craig Venter Institute On 12/11/12 11:10 AM, "Audrey Nisole" <aud...@ho...> wrote: Hi, I used runCA7.0 on a large hybrid assembly with 454 and Illumina data. 454 libraries were single and 6kb paired end, and Illumina libraries were short and long-insert Paired End. There's a total of 20 Gb for up to 100 000 000 fragments. Everything went well at the beginning but after completing the different overlap process, the program stopped during the 4-unitigger step with the following error message : #----------------------------------------START Sun Dec 9 13:39:45 2012 #/prg/wgs/7.0/Linux-amd64/bin/overlapStore -u /budworm_wgs_20121120/budworm_20121120.ovlStore /budworm_wgs_20121120/3-overlapcorrection/budworm_20121120.erates> /budworm_wgs_20121120/3-#overlapcorrection/overlapStore-update-erates.err 2>&1 #----------------------------------------END Sun Dec 9 13:58:20 2012 (1115 seconds) #----------------------------------------START Sun Dec 9 13:58:20 2012 #/prg/wgs/7.0/Linux-amd64/bin/buildUnitigs -O /budworm_wgs_20121120/budworm_20121120.ovlStore -G /budworm_wgs_20121120/budworm_20121120.gkpStore -T #/budworm_wgs_20121120/budworm_20121120.tigStore -B 782495 -e 0.03 -E 2.5 -b -m 7 -U -o /budworm_wgs_20121120/4-unitigger/budworm_20121120 > /budworm_wgs_20121120/4-unitigger/unitigger.err #2>&1 #sh: line 1: 41499 Abandon /prg/wgs/7.0/Linux-amd64/bin/buildUnitigs -O /budworm_wgs_20121120/budworm_20121120.ovlStore -G /budworm_wgs_20121120/budworm_20121120.gkpStore -T #/budworm_wgs_20121120/budworm_20121120.tigStore -B 782495 -e 0.03 -E 2.5 -b -m 7 -U -o /budworm_wgs_20121120/4-unitigger/budworm_20121120 > /budworm_wgs_20121120/4-unitigger/unitigger.err 2>&1 #----------------------------------------END Sun Dec 9 18:50:41 2012 (17541 seconds) #ERROR: Failed with signal ABRT (6) #================================================================================ #runCA failed. #---------------------------------------- #Stack trace: #Goto undefined subroutine &Carp::shortmess_real at /usr/lib/perl5/5.10.0/Carp.pm line 41.# I started it again to try to make it recovering from a premature termination but it stopped once again with this message : #at /prg/bin/runCA line 1237. # main::caFailure('failed to unitig', '/budworm_wgs_20121120/4-uniti...') called at /prg/bin/runCA line 4382 # main::unitigger() called at /prg/bin/runCA line 5882 #---------------------------------------- #Last few lines of the relevant log file (/budworm_wgs_20121120/4-unitigger/unitigger.err): #Bad mate threshold = -7 #Error threshold = 0.030 (3.000%) #Error limit = 2.500 errors #sizeof(ufPath) = 24 #FragmentInfo()-- Loading fragment information deleted: 0 active: 10000000 #FragmentInfo()-- Loading fragment information deleted: 0 active: 20000000 #FragmentInfo()-- Loading fragment information deleted: 0 active: 30000000 #FragmentInfo()-- Loading fragment information deleted: 0 active: 40000000 #FragmentInfo()-- Loading fragment information deleted: 0 active: 50000000 #FragmentInfo()-- Loading fragment information deleted: 0 active: 60000000 #FragmentInfo()-- Loading fragment information deleted: 0 active: 70000000 #FragmentInfo()-- Loading fragment information deleted: 0 active: 80000000 #FragmentInfo()-- Loading fragment information deleted: 0 active: 90000000 #FragmentInfo()-- Loading fragment information deleted: 0 active:100000000 #FragmentInfo()-- Loaded 100159473 alive fragments, skipped 0 dead fragments. #FragmentInfo()-- Saving fragment information to cache '/budworm_wgs_20121120/4-unitigger/budworm.fragmentInfo' #setLogFile()-- Now logging to '/budworm_wgs_20121120/4-unitigger/budworm.001.bestoverlapgraph-containments.log' #buildUnitigs: AS_BOG_BestOverlapGraph.hh:221: uint32 BestOverlapGraph::olapLength(uint32, uint32, int32, int32): Assertion `bovl > 0' failed. #---------------------------------------- #Failure message: #failed to unitig# I'd like to understand what happen. Is there any particular reason for a failure in the unitig process ? Should I correct some specific settings to avoid that premature termination ? I attached the specfile I used. Thanks for any help about that ! Audrey N. |
|
From: Audrey N. <aud...@ho...> - 2012-12-11 16:10:49
|
utgErrorRate = 0.03 utgErrorLimit = 2.5 ovlErrorRate=0.06 cnsErrorRate=0.06 cgwErrorRate=0.10 # frgMinLen = 64 ovlMinLen = 40 # overlapper = ovl obtOverlapper = ovl ovlOverlapper = ovl # ovlStoreMemory = 100000 saveOverlaps = 1 # merSize = 22 # ovlThreads = 4 ovlConcurrency = 5 # ovlHashBits = 21 ovlHashBlockLength = 30000000 ovlRefBlockSize = 200000000 # merylMemory = 200000 merylThreads = 4 # frgCorrBatchSize = 200000 frgCorrThreads = 4 frgCorrConcurrency = 5 # unitigger = bog # computeInsertSize = 0 # cnsConcurrency = 2 # closureOverlaps = 0 closurePlacement = 2 createACE = 0 # #454 Shotguns /project/Budworm_shotgun2.frg # #Paired-ends /project/Budworm_Pairend.frg /project/S7_subsetPE.frg /project/ill700_subsetPE.frg # |
|
From: Walenz, B. <bw...@jc...> - 2012-12-10 20:57:42
|
Hi, Paul- I was working on something to parse the various posmap files to generate such a report. Per library, it counts the number of reads that end up as: trimmed/deleted in various types of unitig (unique, rock, surrogate, singleton) in small/big contigs It does a little bit of mate pair analysis, but this isn’t finished yet. What classifications are you looking for? b On 12/7/12 9:21 AM, "Paul Cantalupo" <pca...@gm...> wrote: Hi all, Is there a way in CVS version of runCA (or even version 7.0) to get a report about what happened to each and every sequence that went into the assembly? Thank you, Paul Paul Cantalupo University of Pittsburgh |
9 messages has been excluded from this view by a project administrator.
