Re: [Samtools-help] BinaryCodec in readmode error

SourceForge Headquarters 1320 Columbia Street Suite 310 San Diego, CA 92101 +1 (858) 422-6466

Interestingly enough I'm getting this error on an Amazon c1.xlarge node as
well.

On Sat, Dec 22, 2012 at 2:38 PM, Ryan Golhar <ngs...@gm...>wrote:

> Hi Tom - The machine is a 64-bit linux cluster.  Each node has 64 cores,
> 128GB of RAM.  I'm running this on a single core.  The disk is an Isilon
> storage system.
>
> I have had weird Java problems with this particular cluster in the past
> and have had to specify both the SGE and the Java VM how much memory I will
> need, so my job submission scripts looks like this:
>
> #!/bin/bash
> #$ -q all.q
> #$ -cwd
> #$ -j y
> #$ -S /bin/bash
> #$ -l h_vmem=8G
> #$ -l mem_free=8G
>
> echo Adding Read Group to $SM
>
> /usr/java/jdk1.6.0_37/bin/java -Xms4096m -Xmx4096m -jar
> AddOrReplaceReadGroups.jar ...
>
> If I don't specify the memory requirements, I get an error:
>
> Error occurred during initialization of VM
> Could not reserve enough space for object heap
> Could not create the Java virtual machine.
>
> so, I wouldn't be surprised if this is a cluster specific problem and not
> a Picard problem.  I can run the same command on a different cluster on the
> same BAM files without problem.
>
>
> On Sat, Dec 22, 2012 at 9:03 AM, Thomas W. Blackwell <tb...@um...>wrote:
>
>> Ryan  -
>>
>> Please tell us more about the machine this is running on.
>>
>> The line  "Runtime.totalMemory()=**4291035136"  is suspiciously close to
>>  2^32=4,294,967,296.  And the line "Caused by: java.io.IOException:
>> Unexpected compressed block length: 1" suggests a failure to read the next
>> block from the file.  No ideas beyond this.
>>
>>                                                 -  tom blackwell  -
>>
>>
>> On Fri, 21 Dec 2012, Ryan Golhar wrote:
>>
>>  Hi - I have a few BAM files generated by Tophat, then sorted with
>>> samtools,
>>> followed by chromosome reordering with ReorderSam.  I collect statistics
>>> of
>>> mapped reads along the way and run other various pipelines with the
>>> resulting BAM file.
>>>
>>> I now need to attach a read group for further analysis and instead of
>>> re-aligning with tophat and providing read group information, I want to
>>> use
>>> addReadGroup.  When I do this, I get a weird error for all my BAM files.
>>> The command I'm using is:
>>>
>>> /usr/java/jdk1.6.0_37/bin/java -Xms4096m -Xmx4096m -jar
>>> picard-tools-1.81/**AddOrReplaceReadGroups.jar \
>>> I=${SM}_hg19_TophatPE_refGene.**hg19GTF.kayrotype.bam \
>>> O=${SM}_hg19_TophatPE_refGene.**hg19GTF.kayrotype.rg.bam \
>>> SO=coordinate \
>>> ID=${SM} \
>>> LB=${SM} \
>>> PL=illumina \
>>> PU=hiseq \
>>> SM=${SM}
>>>
>>> where $SM is the sample name as an environment variable.  The output I
>>> get
>>> is:
>>>
>>> [Fri Dec 21 14:58:18 EST 2012] net.sf.picard.sam.**
>>> AddOrReplaceReadGroups
>>> INPUT=SM12345_hg19_TophatPE_**refGene.hg19GTF.kayrotype.bam
>>> OUTPUT=SM12345_hg19_TophatPE_**refGene.hg19GTF.kayroty
>>> pe.rg.bam SORT_ORDER=coordinate RGID=SM12345 RGLB=SM12345 RGPL=illumina
>>> RGPU=hiseq RGSM=SM12345    VERBOSITY=INFO QUIET=false
>>> VALIDATION_STRINGENCY=STRICT COMPRESSION_LEVEL=5 M
>>> AX_RECORDS_IN_RAM=500000 CREATE_INDEX=false CREATE_MD5_FILE=false
>>> [Fri Dec 21 14:58:18 EST 2012] Executing as golharr@c115 on Linux
>>> 2.6.32-220.13.1.el6.x86_64 amd64; Java HotSpot(TM) 64-Bit Server VM
>>> 1.6.0_37-b06; Picard version: 1.81(1299)
>>> WARNING: BAM index file
>>> SM12345_hg19_TophatPE_refGene.**hg19GTF.kayrotype.bam.bai is older than
>>> BAM SM12345_hg19_TophatPE_refGene.**hg19GTF.kayrotype.bam
>>> INFO 2012-12-21 14:58:18 AddOrReplaceReadGroups Created read group
>>> ID=SM12345 PL=illumina LB=SM12345 SM=SM12345
>>>
>>> INFO 2012-12-21 14:58:47 AddOrReplaceReadGroups Processed     1,000,000
>>> records.  Elapsed time: 00:00:28s.  Time for last 1,000,000:   28s.  Last
>>> read position: chr1:25,169,610
>>> INFO 2012-12-21 14:59:14 AddOrReplaceReadGroups Processed     2,000,000
>>> records.  Elapsed time: 00:00:56s.  Time for last 1,000,000:   27s.  Last
>>> read position: chr1:52,257,758
>>> INFO 2012-12-21 15:00:00 AddOrReplaceReadGroups Processed     3,000,000
>>> records.  Elapsed time: 00:01:41s.  Time for last 1,000,000:   45s.  Last
>>> read position: chr1:115,275,434
>>> INFO 2012-12-21 15:00:26 AddOrReplaceReadGroups Processed     4,000,000
>>> records.  Elapsed time: 00:02:08s.  Time for last 1,000,000:   26s.  Last
>>> read position: chr1:160,654,833
>>> INFO 2012-12-21 15:00:55 AddOrReplaceReadGroups Processed     5,000,000
>>> records.  Elapsed time: 00:02:36s.  Time for last 1,000,000:   28s.  Last
>>> read position: chr1:229,599,366
>>> INFO 2012-12-21 15:01:22 AddOrReplaceReadGroups Processed     6,000,000
>>> records.  Elapsed time: 00:03:03s.  Time for last 1,000,000:   27s.  Last
>>> read position: chr2:62,107,436
>>> [Fri Dec 21 15:01:41 EST 2012] net.sf.picard.sam.**
>>> AddOrReplaceReadGroups
>>> done. Elapsed time: 3.40 minutes.
>>> Runtime.totalMemory()=**4291035136
>>> FAQ:  http://sourceforge.net/apps/**mediawiki/picard/index.php?**
>>> title=Main_Page<http://sourceforge.net/apps/mediawiki/picard/index.php?title=Main_Page>
>>> Exception in thread "main" net.sf.samtools.util.**RuntimeIOException:
>>> Read
>>> error; BinaryCodec in readmode;
>>> file: SM12345_hg19_TophatPE_refGene.**hg19GTF.kayrotype.bam
>>> at net.sf.samtools.util.**BinaryCodec.readBytesOrFewer(**
>>> BinaryCodec.java:396)
>>> at net.sf.samtools.util.**BinaryCodec.readBytes(**BinaryCodec.java:371)
>>> at net.sf.samtools.util.**BinaryCodec.readBytes(**BinaryCodec.java:357)
>>> at net.sf.samtools.**BAMRecordCodec.decode(**BAMRecordCodec.java:200)
>>> at
>>> net.sf.samtools.BAMFileReader$**BAMFileIterator.getNextRecord(**
>>> BAMFileReader.java:557)
>>> at
>>> net.sf.samtools.BAMFileReader$**BAMFileIterator.advance(**
>>> BAMFileReader.java:531)
>>> at
>>> net.sf.samtools.BAMFileReader$**BAMFileIterator.next(**
>>> BAMFileReader.java:521)
>>> at
>>> net.sf.samtools.BAMFileReader$**BAMFileIterator.next(**
>>> BAMFileReader.java:480)
>>> at
>>> net.sf.samtools.SAMFileReader$**AssertableIterator.next(**
>>> SAMFileReader.java:669)
>>> at
>>> net.sf.samtools.SAMFileReader$**AssertableIterator.next(**
>>> SAMFileReader.java:647)
>>> at
>>> net.sf.picard.sam.**AddOrReplaceReadGroups.doWork(**
>>> AddOrReplaceReadGroups.java:**98)
>>> at
>>> net.sf.picard.cmdline.**CommandLineProgram.**instanceMain(**
>>> CommandLineProgram.java:177)
>>> at
>>> net.sf.picard.cmdline.**CommandLineProgram.**instanceMainWithExit(**
>>> CommandLineProgram.java:119)
>>> at
>>> net.sf.picard.sam.**AddOrReplaceReadGroups.main(**
>>> AddOrReplaceReadGroups.java:**66)
>>> Caused by: java.io.IOException: Unexpected compressed block length: 1
>>> at
>>> net.sf.samtools.util.**BlockCompressedInputStream.**readBlock(**
>>> BlockCompressedInputStream.**java:354)
>>> at
>>> net.sf.samtools.util.**BlockCompressedInputStream.**available(**
>>> BlockCompressedInputStream.**java:109)
>>> at
>>> net.sf.samtools.util.**BlockCompressedInputStream.**read(**
>>> BlockCompressedInputStream.**java:234)
>>> at java.io.DataInputStream.read(**DataInputStream.java:132)
>>> at net.sf.samtools.util.**BinaryCodec.readBytesOrFewer(**
>>> BinaryCodec.java:394)
>>> ... 13 more
>>>
>>> What does this mean?  Is there something wrong with my BAM file?  I've
>>> used
>>> AddOrReplaceReadGroup in the past without problems.
>>>
>>>
>