[Samtools-help] picard_MarkDuplicates:Exception closing FileOutputStream

SourceForge Headquarters 1320 Columbia Street Suite 310 San Diego, CA 92101 +1 (858) 422-6466

Dear Samtools-list,

Recently, I used Picardtools/MarkDuplicates as following:

java -Djava.io.tmpdir=/data/mira0501/agilentfastq/nonmulti/  -Xmx4g -Xms4g
-jar /home/mira0501/apps/picard-tools-1.119/MarkDuplicates.jar
MAX_FILE_HANDLES_FOR_READ_END
S_MAP=1000 I=e_ptgene.sorted.bam  O=e_ptgene.dedup.bam
M=e_ptgenedup.metrics AS=true VALIDATION_STRINGENCY=LENIENT

After 45 mins, the error message showed:
___________________________________________________________________
.......
INFO    2014-09-29 14:53:30     MarkDuplicates  Tracking 1004938 as yet
unmatched pairs. 89 records in RAM.
INFO    2014-09-29 14:56:08     MarkDuplicates  Read    12,000,000
records.  Elapsed time: 00:41:57s.  Time for last 1,000,000:  158s.  Last
read position: scaffold3685
03:22,683
INFO    2014-09-29 14:56:08     MarkDuplicates  Tracking 1078120 as yet
unmatched pairs. 36 records in RAM.
[Mon Sep 29 14:57:41 CDT 2014] picard.sam.MarkDuplicates done. Elapsed
time: 43.51 minutes.
Runtime.totalMemory()=3818389504
To get help, see http://picard.sourceforge.net/index.shtml#GettingHelp
Exception in thread "main" htsjdk.samtools.SAMException: Exception closing
FileOutputStream for
/data/mira0501/agilentfastq/nonmulti/mira0501/CSPI.5751822609546779855.t
mp/27421.tmp
        at
htsjdk.samtools.util.FileAppendStreamLRUCache$Functor.finalizeValue(FileAppendStreamLRUCache.java:74)
        at
htsjdk.samtools.util.FileAppendStreamLRUCache$Functor.finalizeValue(FileAppendStreamLRUCache.java:49)
        at
htsjdk.samtools.util.ResourceLimitedMap$1.removeEldestEntry(ResourceLimitedMap.java:59)
        at java.util.LinkedHashMap.addEntry(LinkedHashMap.java:431)
        at java.util.HashMap.put(HashMap.java:509)
        at
htsjdk.samtools.util.ResourceLimitedMap.get(ResourceLimitedMap.java:76)
        at
htsjdk.samtools.CoordinateSortedPairInfoMap.getOutputStreamForSequence(CoordinateSortedPairInfoMap.java:180)
        at
htsjdk.samtools.CoordinateSortedPairInfoMap.put(CoordinateSortedPairInfoMap.java:164)
        at picard.sam.DiskReadEndsMap.put(DiskReadEndsMap.java:67)
        at
picard.sam.MarkDuplicates.buildSortedReadEndLists(MarkDuplicates.java:449)
        at picard.sam.MarkDuplicates.doWork(MarkDuplicates.java:177)
        at
picard.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:183)
        at picard.sam.MarkDuplicates.main(MarkDuplicates.java:161)
Caused by: java.io.IOException: Input/output error
        at java.io.FileOutputStream.close0(Native Method)
        at java.io.FileOutputStream.close(FileOutputStream.java:393)
        at java.io.FilterOutputStream.close(FilterOutputStream.java:160)
        at
htsjdk.samtools.util.FileAppendStreamLRUCache$Functor.finalizeValue(FileAppendStreamLRUCache.java:71)
        ... 12 more
__________________________________________________________________

Then I tried it again,and  I did not change anything in  the command. This
time, it lasted longer, after 1hour, it reported the error:
_____________________________________________________________
......
INFO    2014-09-29 21:14:59     MarkDuplicates  Tracking 1509035 as yet
unmatched pairs. 119 records in RAM.
INFO    2014-09-29 21:15:37     MarkDuplicates  Read    45,000,000
records.  Elapsed time: 01:43:10s.  Time for last 1,000,000:   38s.  Last
read position: tscaffold375
9:199,308
INFO    2014-09-29 21:15:37     MarkDuplicates  Tracking 1476529 as yet
unmatched pairs. 78 records in RAM.
[Mon Sep 29 21:17:31 CDT 2014] picard.sam.MarkDuplicates done. Elapsed
time: 105.41 minutes.
Runtime.totalMemory()=3922722816
To get help, see http://picard.sourceforge.net/index.shtml#GettingHelp
Exception in thread "main" htsjdk.samtools.SAMException: Exception closing
FileOutputStream for
/data/mira0501/agilentfastq/nonmulti/mira0501/CSPI.2781500592681706581.t
mp/29668.tmp
        at
htsjdk.samtools.util.FileAppendStreamLRUCache$Functor.finalizeValue(FileAppendStreamLRUCache.java:74)
        at
htsjdk.samtools.util.FileAppendStreamLRUCache$Functor.finalizeValue(FileAppendStreamLRUCache.java:49)
        at
htsjdk.samtools.util.ResourceLimitedMap$1.removeEldestEntry(ResourceLimitedMap.java:59)
        at java.util.LinkedHashMap.addEntry(LinkedHashMap.java:431)
        at java.util.HashMap.put(HashMap.java:509)
        at
htsjdk.samtools.util.ResourceLimitedMap.get(ResourceLimitedMap.java:76)
        at
htsjdk.samtools.CoordinateSortedPairInfoMap.getOutputStreamForSequence(CoordinateSortedPairInfoMap.java:180)
        at
htsjdk.samtools.CoordinateSortedPairInfoMap.put(CoordinateSortedPairInfoMap.java:164)
        at picard.sam.DiskReadEndsMap.put(DiskReadEndsMap.java:67)
        at
picard.sam.MarkDuplicates.buildSortedReadEndLists(MarkDuplicates.java:449)
        at picard.sam.MarkDuplicates.doWork(MarkDuplicates.java:177)
        at
picard.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:183)
        at picard.sam.MarkDuplicates.main(MarkDuplicates.java:161)
Caused by: java.io.IOException: Input/output error
        at java.io.FileOutputStream.close0(Native Method)
        at java.io.FileOutputStream.close(FileOutputStream.java:393)
        at java.io.FilterOutputStream.close(FilterOutputStream.java:160)
        at
htsjdk.samtools.util.FileAppendStreamLRUCache$Functor.finalizeValue(FileAppendStreamLRUCache.java:71)
        ... 12 more
_________________________________________________________

Dear fellow picard users, could you advise what should I do for "Exception
in thread "main" htsjdk.samtools.SAMException: Exception closing
FileOutputStream" and "java.io.IOException: Input/output error",please?

Thanks very much!

Mengmeng