Re: [svtoolkit-help] Error without description

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 422-6466

You shouldn't follow installtest as the example to run.  Remove:

-genomeMaskFile, -copyNumberMaskFile, -genderMapFile, 
-disableGATKTraversal, -useMultiStep, -reduceInsertSizeDistributions, 
-computeGCProfiles, -computeReadCounts and -P 
chimerism.use.correction:false.

Let these all default as appropriate (the mask files default based on -R).

Change -configFile to "-configFile ${SV_DIR}/conf/genstrip_parameters.txt".

The problem you reported may still remain, however, as it is likely 
something in your LSF environment such that Queue thinks some of the 
individual batch jobs have failed when in fact they have succeeded. This 
may be because the job is returning a non-zero exit status for some 
reason, or some quirk of the DRMAA api in your environment.
These problems can be hard to debug (sometimes this seems to happen more 
often with jobs that complete very quickly). Often, a sufficient 
workaround is to simply run the command again (it will retry just the 
jobs that previously failed). The jobs may run fine the second time around.

-Bob

On 9/25/17 1:42 AM, Elbay Aliyev wrote:
>
> Dear GenomeStrip team,
>
> My name is Elbay Aliyev. I am working as a Research Specialist at 
> Sidra Medical and Research Center. We have huge project of 3000 Qatari 
> Genome Project and we want to use genomestrip in our studies. But we 
> are facing unexplainable error without any stacktrace during 
> Preprocess step without description.
>
> **Preprocess script:
> **
> java -cp ${classpath} ${mx} \
> org.broadinstitute.gatk.queue.QCommandLine \
> -S ${SV_DIR}/qscript/SVPreprocess.q \
> -S ${SV_DIR}/qscript/SVQScript.q \
> -gatk ${SV_DIR}/lib/gatk/GenomeAnalysisTK.jar \
> --disableJobReport \
> -cp ${classpath} \
> -configFile conf/genstrip_installtest_parameters.txt \
> -tempDir ${SV_TMPDIR} \
> -R data/Homo_sapiens_assembly19.fasta \
> -genomeMaskFile data/Homo_sapiens_assembly19.svmask.fasta \
> -copyNumberMaskFile data/Homo_sapiens_assembly19.gcmask.fasta \
> -genderMapFile data/installtest_gender.map \
> -runDirectory ${runDir} \
> -md ${runDir}/metadata \
> -disableGATKTraversal \
> -useMultiStep \
> -reduceInsertSizeDistributions false \
> -computeGCProfiles true \
> -computeReadCounts true \
> -jobLogDir ${runDir}/logs \
> -I ${inputFile} \
> -P chimerism.use.correction:false \
> -run \
> || exit 1
>
> and a on a couple of steps we have errors like that.
>
>
> ERROR 16:09:20,504 FunctionEdge - Error: 'java' '-Xmx2048m' 
> '-XX:+UseParallelOldGC' '-XX:ParallelGCThreads=4' '-XX:GCTimeLimit=50' 
> '-XX:GCHeapFreeLimit=10' 
> '-Djava.io.tmpdir=/gpfs/projects/tmedicine/ealiyev/genomestrip/PMC01/tmpdir' 
> '-cp' 
> '/gpfs/projects/tmedicine/ealiyev/genomestrip/lib/SVToolkit.jar:/gpfs/projects/tmedicine/ealiyev/genomestrip/lib/gatk/GenomeAnalysisTK.jar:/gpfs/projects/tmedicine/ealiyev/genomestrip/lib/gatk/Queue.jar' 
> '-cp' 
> '/gpfs/projects/tmedicine/ealiyev/genomestrip/lib/SVToolkit.jar:/gpfs/projects/tmedicine/ealiyev/genomestrip/lib/gatk/GenomeAnalysisTK.jar:/gpfs/projects/tmedicine/ealiyev/genomestrip/lib/gatk/Queue.jar' 
> 'org.broadinstitute.sv.apps.ComputeDepthProfiles' '-O' 
> '/gpfs/projects/tmedicine/ealiyev/genomestrip/PMC01/PMC01/metadata/profiles_100Kb/profile_seq_GL000232.1_100000.dat.gz' 
> '-I' 'PMC01/metadata/headers.bam' '-configFile' 
> 'conf/genstrip_installtest_parameters.txt' '-P' 
> 'chimerism.use.correction:false' '-R' 
> 'data/Homo_sapiens_assembly19.fasta' '-L' 'GL000232.1:0-0' 
> '-genomeMaskFile' 'data/Homo_sapiens_assembly19.svmask.fasta' '-md' 
> 'PMC01/metadata' '-profileBinSize' '100000' 
> '-maximumReferenceGapLength' '10000'
> ERROR 16:09:20,509 FunctionEdge - Contents of 
> /gpfs/projects/tmedicine/ealiyev/genomestrip/PMC01/PMC01/logs/SVPreprocess-111.out:
> INFO 16:04:10,442 HelpFormatter - 
> -------------------------------------------------------------
> INFO 16:04:10,445 HelpFormatter - Program Name: 
> org.broadinstitute.sv.apps.ComputeDepthProfiles
> INFO 16:04:10,449 HelpFormatter - Program Args: -O 
> /gpfs/projects/tmedicine/ealiyev/genomestrip/PMC01/PMC01/metadata/profiles_100Kb/profile_seq_GL000232.1_100000.dat.gz 
> -I PMC01/metadata/headers.bam -configFile 
> conf/genstrip_installtest_parameters.txt -P 
> chimerism.use.correction:false -R data/Homo_sapiens_assembly19.fasta 
> -L GL000232.1:0-0 -genomeMaskFile 
> data/Homo_sapiens_assembly19.svmask.fasta -md PMC01/metadata 
> -profileBinSize 100000 -maximumReferenceGapLength 10000
> INFO 16:04:10,453 HelpFormatter - Executing as 
> ea...@hp...dra.local on Linux 
> 3.10.0-229.el7.x86_64 amd64; Java HotSpot(TM) 64-Bit Server VM 
> 1.8.0_121-b13.
> INFO 16:04:10,454 HelpFormatter - Date/Time: 2017/09/24 16:04:10
> INFO 16:04:10,454 HelpFormatter - 
> -------------------------------------------------------------
> INFO 16:04:10,454 HelpFormatter - 
> -------------------------------------------------------------
> INFO 16:04:10,461 ComputeDepthProfiles - Opening reference sequence ...
> INFO 16:04:10,462 ComputeDepthProfiles - Opened reference sequence.
> INFO 16:04:10,462 ComputeDepthProfiles - Opening genome mask ...
> INFO 16:04:10,463 ComputeDepthProfiles - Opened genome mask.
> INFO 16:04:10,465 MetaData - Opening metadata ...
> INFO 16:04:10,466 MetaData - Adding metadata location PMC01/metadata ...
> INFO 16:04:10,476 MetaData - Opened metadata.
> INFO 16:04:10,476 ComputeDepthProfiles - Opened metadata.
> INFO 16:04:10,476 ComputeDepthProfiles - Initializing input data set ...
> INFO 16:04:10,513 ComputeDepthProfiles - Initialized data set: 1 file, 
> 1 read group, 1 sample.
> INFO 16:04:10,518 MetaData - Loading insert size histograms ...
> INFO 16:04:11,940 ReadCountCache - Initializing read count cache with 
> 1 file.
>
> INFO 16:04:12,010 CommandLineProgram - Program completed.
>
> /Done. There were no warn messages./
>
> /Looks like script did his job perfect but still gives error. Same 
> issue with SVPreprocess 6 and 11 :(/
>
> echo "./discovery.sh" | bsub -n 16 -e 
> /gpfs/projects/tmedicine/ealiyev/genomestrip/PMC01/test.err -o 
> /gpfs/projects/tmedicine/ealiyev/genomestrip/PMC01/test.out -P PMC01.test
>
> We are submitting to our LSF system like that. our LSF version is 
> 10.1. Java version is 1.8.21. GATK we are using internal one provided 
> with GATK.
>
> Thanks in advance.
>
> *Elbay Aliyev*
>
> Software Engineer
>
> *Sidra Medical and Research Center***
>
> PO Box 26999 *| *Doha, Qatar
>
> Direct: 40037393
>
> ea...@si... <mailto:ea...@si...>*|* www.sidra.org 
> <http://www.sidra.org/>
>
> cid:image001.jpg@01CFB54A.69FD9EB0
>
> *P****Please consider the environment before printing this email.*
>
> Disclaimer: This email and its attachments may be confidential and are 
> intended solely for the use of the individual to whom it is addressed. 
> If you are not the intended recipient, any reading, printing, storage, 
> disclosure, copying or any other action taken in respect of this 
> e-mail is prohibited and may be unlawful. If you are not the intended 
> recipient, please notify the sender immediately by using the reply 
> function and then permanently delete what you have received. Any views 
> or opinions expressed are solely those of the author and do not 
> necessarily represent those of Sidra Medical and Research Center.
>
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>
>
> _______________________________________________
> svtoolkit-help mailing list
> svt...@li...
> https://lists.sourceforge.net/lists/listinfo/svtoolkit-help