Re: [Samtools-help] Sort and rmdup

[Tom B. <tb...@um...>]

Daniel  -

I think the -m value is used in the code as an upper limit. 
Samtools sort will take only as much memory as it needs, but 
not MORE than the -m value.  So you don't need to anticipate 
the actual need very precisely.

I haven't checked, but probably the memory footprint is never 
larger than the input .bam file.  (I'm guessing that what's 
in memory is already in compressed .bam format, rather than 
uncompressed.  Haven't checked this.)

 	 	 	 	 	-  tom blackwell  -

On Mon, 19 Apr 2010, Daniel Sobral wrote:

> Thanks for the suggestion.
> Nonetheless I think I saw someone mentioning the -m is not a very accurate 
> estimate?
>
> Just a more technical question: what's the relationship between memory 
> requirements and the size of the file to be sorted?
> Could I infer what would be the required -m by the size of the input file?
>
> Thanks,
> Daniel
>
> On 16/04/2010 18:20, Tom Blackwell wrote:
>> 
>> For a quick fix, you might try using the '-m' flag to give samtools sort 
>> enough memory to avoid any need for temp files. E.g.  'samtools sort -m 
>> 8000000000 -o - temp.sorted | ...' for 8 Gb RAM, assuming you have that 
>> much physical memory.
>>
>>                         -  tom blackwell  -
>> 
>> On Fri, 16 Apr 2010, Daniel Sobral wrote:
>> 
>>> I'm having similar problems.
>>> What seems to be happening in my case is that there are for some reason
>>> problems with the temp files that get created during sorting.
>>> 
>>> Daniel
>>> 
>>> On 16/04/2010 07:03, Henrikki Almusa wrote:
>>>> Hi,
>>>> 
>>>> I've tried to use the piping from sort but it seems to fail. If I run
>>>> the two commands:
>>>> 
>>>> samtools import ensembl-hs_GRCh37_56.fa.fai alignment-raw.sam - |
>>>>    samtools sort - raw-sorted
>>>> samtools rmdup raw-sorted.bam sorted.bam
>>>> 
>>>> This produces a 1.8Gb sorted bam file. However if I try to use the
>>>> piping the sort:
>>>> 
>>>> samtools import ensembl-hs_GRCh37_56.fa.fai alignment-raw.sam - |
>>>>    samtools sort -o - raw-sorted | samtools rmdup - sorted.bam
>>>> 
>>>> I get 163 Mb bam file. What causes the difference?
>>>> 
>>>> Thanks,
>>>> 
>>> 
>>>
>>> 
>>> ------------------------------------------------------------------------------ 
>>> Download Intel® Parallel Studio Eval
>>> Try the new software tools for yourself. Speed compiling, find bugs
>>> proactively, and fine-tune applications for parallel performance.
>>> See why Intel Parallel Studio got high marks during beta.
>>> http://p.sf.net/sfu/intel-sw-dev
>>> _______________________________________________
>>> Samtools-help mailing list
>>> Sam...@li...
>>> https://lists.sourceforge.net/lists/listinfo/samtools-help
>>> 
>>> 
>>> 
>
>
>
>