Menu
▾
▴
Re: [wgs-assembler-users] Again with submission issues
|
From: Sergey K. <se...@um...> - 2013-01-22 15:17:51
|
I am not sure why your estimate-mer-threshold is using the -g parameter. I just checked the latest CVS version of CA and it does not have the -g option being passed to estimate mer threshold:
2361 if ($merThresh eq "auto") {
2362 if (! -e "$ofile.estMerThresh.out") {
2363 $cmd = "$bin/estimate-mer-threshold ";
2364 $cmd .= " -m $ofile ";
2365 $cmd .= " > $ofile.estMerThresh.out ";
2366 $cmd .= "2> $ofile.estMerThresh.err";
2367
2368 stopBefore("meryl", $cmd);
2369
2370 if (runCommand("$wrk/0-mercounts", $cmd)) {
2371 rename "$ofile.estMerThresh.out", "$ofile.estMerThresh.out.FAILED";
2372 caFailure("estimate-mer-threshold failed", "$ofile.estMerThresh.err");
2373 }
It used to have a -g option in CA version 7.0 so maybe your AS_MER directory got updated but not your AS_RUN? It looks like the change happened a while ago (March 2012). I would double-check that you are at the latest version of AS_RUN directory and don't have any modifications that would have prevented an update. I would also update the kmer package you are using to build CA (kmer in wgs-assembler) and see if that fixes the issue.
Sergey
On Jan 19, 2013, at 10:15 AM, Powers, Jason wrote:
Okay, interesting. Thanks for the info. I had noticed that it seemed to continue with several steps even though it quit. However, it didn’t seem to get all the way done, and got stuck at a few steps. I will have to better categorize why it didn’t proceed and get back to you. However just manually running the program repeatedly got it to finally finish.
However, now I have a runCA error cropping up that I don’t understand, it's related to the estimate-mer-threshold command. It doesn't recognize the -g parameter. I don't see anywhere in the spec file or elsewhere that I dictate this parameter, so I think it is something internal to the program?
/opt/bin/wgs-bin/estimate-mer-threshold -g /[PATH]/specfile_9.spec/specfile_9.spec.gkpStore:chain -m /[PATH]/specfile_9.spec/0-mercounts/specfile_9.spec-C-ms22-cm0 > /[PATH]/specfile_9.spec/0-mercounts/specfile_9.spec-C-ms22-cm0.estMerThresh.out 2> /[PATH]/specfile_9.spec/0-mercounts/specfile_9.spec-C-ms22-cm0.estMerThresh.err
----------------------------------------END Fri Jan 18 17:45:23 2013 (0 seconds)
ERROR: Failed with signal HUP (1)
================================================================================
runCA failed.
----------------------------------------
Stack trace:
at /opt/bin/runCA line 1237.
main::caFailure('estimate-mer-threshold failed', '/[PATH]/BL21-DE3_long...') called at /opt/bin/runCA line 2218
main::runMeryl(22, 0, '-C', 'auto', 'ovl', 1) called at /opt/bin/runCA line 2350
main::meryl() called at /opt/bin/runCA line 3280
main::createOverlapJobs('trim') called at /opt/bin/runCA line 3647
main::overlapTrim() called at /opt/bin/runCA line 5876
----------------------------------------
Last few lines of the relevant log file (/[PATH]/specfile_9.spec/0-mercounts/specfile_9.spec-C-ms22-cm0.estMerThresh.err):
unknown option '-g'
unknown option '/[PATH]/specfile_9.spec/specfile_9.spec.gkpStore:chain'
usage: /opt/bin/wgs-bin/estimate-mer-threshold -m mercounts
-m mercounts file of mercounts
----------------------------------------
Failure message:
estimate-mer-threshold failed
From: Sergey Koren [mailto:se...@um...]
Sent: Friday, January 18, 2013 5:30 PM
To: Powers, Jason
Cc: wgs...@li...<mailto:wgs...@li...>
Subject: Re: [wgs-assembler-users] Again with submission issues
Hi,
The latest version of the pacBioToCA pipeline does not use (or require -sync y). Instead, it functions like runCA, submitting itself to the grid immediately after launch and then using the qhold option to control dependencies. It is the expected behavior for the job to quit as soon as it has submitted itself to the grid. The final output before quitting should be:
Your job 1544850 ("pBcR_asm_ccsman25") has been submitted
You can use this ID to see the job running on the grid. The file runPBcR.sge.out.00 will also record the output of the submitted job so you can track progress.
Sergey
On Jan 17, 2013, at 5:23 PM, Powers, Jason wrote:
Hi all,
A few months ago I wrote in about having problems getting distributed to computing to work with our job submission system. I am happy to say we got all the kinks ironed out between qsub and our system. Unfortunately we recently upgraded to the latest checkout of pacBioToCA (as of December 11), and this seems to have broken things a bit.
I run pacBioToCA and it submits the runPBcR.sge.out.00.sh job and just quits. I looked at the log and it is calling it via qsub as follows:
qsub -A assembly -pe threads 4 -cwd -N "pBcR_asm" -j y -o /mnt/Celera/BL21-DE3/MiSeq//tempBL21-DE3_long.clipped.corrected/runPBcR.sge.out.00 /mnt/Celera/BL21-DE3/MiSeq//tempBL21-DE3_long.clipped.corrected/runPBcR.sge.out.00.sh
I noticed that there wasn’t a sync command or a hold_jid in this qsub call…is that correct? I looked at an old “out” file, and it did indeed have the sync
qsub -A assembly -sync y -pe threads 4 -cwd -N "rCA_asm" -j y -o /mnt/50X_Illum/20X_PB/temp94-3025.LongReads.20X.corrected/runCA.sge.out.00 /mnt/50X_Illum/20X_PB/temp94-3025.LongReads.20X.corrected/runCA.sge.out.00.sh
So I edited my spec file so that instead of this
sge = -A assembly
it says
sge = -A assembly –sync y
It now “looks” correct
qsub -A assembly -sync y -pe threads 4 -cwd -N "pBcR_asm" -j y -o /mnt/Celera/BL21-DE3/Ion//tempBL21-DE3_long.clipped.corrected/runPBcR.sge.out.00 /mnt/Celera/BL21-DE3/Ion//tempBL21-DE3_long.clipped.corrected/runPBcR.sge.out.00.sh
Although now it seems to just hang.
So any ideas?
Thanks,
Jason
------------------------------------------------------------------------------
Master Visual Studio, SharePoint, SQL, ASP.NET<http://ASP.NET>, C# 2012, HTML5, CSS,
MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current
with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft
MVPs and experts. ON SALE this month only -- learn more at:
http://p.sf.net/sfu/learnmore_122712_______________________________________________
wgs-assembler-users mailing list
wgs...@li...<mailto:wgs...@li...>
https://lists.sourceforge.net/lists/listinfo/wgs-assembler-users
------------------------------------------------------------------------------
Master Visual Studio, SharePoint, SQL, ASP.NET<http://ASP.NET>, C# 2012, HTML5, CSS,
MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current
with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft
MVPs and experts. SALE $99.99 this month only -- learn more at:
http://p.sf.net/sfu/learnmore_122912_______________________________________________
wgs-assembler-users mailing list
wgs...@li...<mailto:wgs...@li...>
https://lists.sourceforge.net/lists/listinfo/wgs-assembler-users
|
