Menu
▾
▴
Re: [Samtools-help] Sort and rmdup
|
From: <go...@um...> - 2010-04-19 12:52:33
|
I don't think there is an easy relationship between size of the file to be sorted and memory requirements. If you were sorting a SAM file, memory likely would be comparable to file size. In a BAM, memory needs will probably exceed file size by a small integer factor but this will depend on compression ratio, etc. G -----Original Message----- From: Daniel Sobral <so...@eb...> Date: Mon, 19 Apr 2010 11:08:57 To: Tom Blackwell<tb...@um...> Cc: <sam...@li...> Subject: Re: [Samtools-help] Sort and rmdup Thanks for the suggestion. Nonetheless I think I saw someone mentioning the -m is not a very accurate estimate? Just a more technical question: what's the relationship between memory requirements and the size of the file to be sorted? Could I infer what would be the required -m by the size of the input file? Thanks, Daniel On 16/04/2010 18:20, Tom Blackwell wrote: > > For a quick fix, you might try using the '-m' flag to give samtools > sort enough memory to avoid any need for temp files. E.g. 'samtools > sort -m 8000000000 -o - temp.sorted | ...' for 8 Gb RAM, assuming you > have that much physical memory. > > - tom blackwell - > > On Fri, 16 Apr 2010, Daniel Sobral wrote: > >> I'm having similar problems. >> What seems to be happening in my case is that there are for some reason >> problems with the temp files that get created during sorting. >> >> Daniel >> >> On 16/04/2010 07:03, Henrikki Almusa wrote: >>> Hi, >>> >>> I've tried to use the piping from sort but it seems to fail. If I run >>> the two commands: >>> >>> samtools import ensembl-hs_GRCh37_56.fa.fai alignment-raw.sam - | >>> samtools sort - raw-sorted >>> samtools rmdup raw-sorted.bam sorted.bam >>> >>> This produces a 1.8Gb sorted bam file. However if I try to use the >>> piping the sort: >>> >>> samtools import ensembl-hs_GRCh37_56.fa.fai alignment-raw.sam - | >>> samtools sort -o - raw-sorted | samtools rmdup - sorted.bam >>> >>> I get 163 Mb bam file. What causes the difference? >>> >>> Thanks, >>> >> >> >> ------------------------------------------------------------------------------ >> >> Download Intel® Parallel Studio Eval >> Try the new software tools for yourself. Speed compiling, find bugs >> proactively, and fine-tune applications for parallel performance. >> See why Intel Parallel Studio got high marks during beta. >> http://p.sf.net/sfu/intel-sw-dev >> _______________________________________________ >> Samtools-help mailing list >> Sam...@li... >> https://lists.sourceforge.net/lists/listinfo/samtools-help >> >> >> ------------------------------------------------------------------------------ Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev _______________________________________________ Samtools-help mailing list Sam...@li... https://lists.sourceforge.net/lists/listinfo/samtools-help |
