Re: [MUMmer-help] MUMmer Memory Cap/Problem?

[Adam P. <aph...@gm...>]

Hello Steve,
At first glance, it appears to be exhausting the memory of the machine. The
"536870908" number is an upper limit on the input sequence, regardless of
the hardware. Thus you could have a machine with unlimited RAM and mummer
could still only handle 536870908 bp of input. The general rule of thumb is
~17 bytes of RAM per base pair of input. That fluctuates based on the repeat
content of the input, but is a good guess. So, for 115 Mbp of input, mummer
will use about 2 GB of RAM. Since your machine has 12 GB, this should be
well below the limit. Are you running a 64-bit OS capable of addressing more
than 2 GB RAM per process? Or maybe there are competing processes using the
memory? If that doesn't seem to be the problem, let me know and I can
download and try one of the sequences that is giving you trouble.

Best,
Adam Phillippy


On 10/22/07, Steve Lane <dr...@rn...> wrote:
>
> Greetings.  My name is Steve Lane, and I am the sysadmin for Dr. Jennifer
> Doudna at UC Berkeley.  We are having a problem getting MUMmer to
> work with the human chromosomes larger than chromosome 13 (larger than
> ~115MBP).
>
> The fasta files were downloaded from the NCBI36.46 release, and the
> machine the jobs are being run on has 12GB of RAM, but, as can be seen
> below, the process seems to be running out of memory:
>
>   ---cut---
>   End of GetDescription for mapGO_tuschl_clean2_unique.fa with 22930 Lines
>   # reading input file "../HUGO/Homo_sapiens.NCBI36.46.dna.chromosome.
>   1.fa" of length 247249719
>   # construct suffix tree for sequence of length 247249719
>   # (maximum input length is 536870908)
>   # process 2472497 characters per dot
>   file "construct.c", line 878:  allocandusespaceviaptr(4,370874580)
>   failed:not enough memory
>   # reading input file "../HUGO/Homo_sapiens.NCBI36.46.dna.chromosome.
>   2.fa" of length 242951149
>   # construct suffix tree for sequence of length 242951149
>   # (maximum input length is 536870908)
>   # process 2429511 characters per dot
>   file "construct.c", line 878:  allocandusespaceviaptr(4,364426725)
>   failed:not enough memory
>   # reading input file "../HUGO/Homo_sapiens.NCBI36.46.dna.chromosome.
>   3.fa" of length 199501827
>   # construct suffix tree for sequence of length 199501827
>   # (maximum input length is 536870908)
>   # process 1995018 characters per dot
>   file "construct.c", line 878:  allocandusespaceviaptr(4,299252742)
>   failed:not enough memory
>   # reading input file "../HUGO/Homo_sapiens.NCBI36.46.dna.chromosome.
>   4.fa" of length 191273063
>   # construct suffix tree for sequence of length 191273063
>   # (maximum input length is 536870908)
>   # process 1912730 characters per dot
>   #.....................................................file
>   "construct.c", line 125:  allocandusespaceviaptr(4,315600555)
>   failed:not enough memory
>   # reading input file "../HUGO/Homo_sapiens.NCBI36.46.dna.chromosome.
>   5.fa" of length 180857866
>   # construct suffix tree for sequence of length 180857866
>   # (maximum input length is 536870908)
>   # process 1808578 characters per dot
>   #................................................# reading input file
>   "../HUGO/Homo_sapiens.NCBI36.46.dna.chromosome.6.fa" # reading input
>   file "../HUGO/Homo_sapiens.NCBI36.46.dna.chromosome.7.fa" # reading
>   input file "../HUGO/Homo_sapiens.NCBI36.46.dna.chromosome.8.fa"
>   ---cut---
>
> It doesn't seem that MUMmer is incapable of processing chromosomes of
> this size, as the output indicates that the suffix tree is for a sequence
> of length 247249719 and that the maximum input length is 536870908.
>
> We looked briefly at the C source code but couldn't find any obvious
> problem.
>
> Any help would be appreciated.   We are not subscribers to the list,
> so please reply directly.
>
> Thanks very much,
>
> --
> Steve Lane
> System, Network and Security Administrator
> Doudna Lab
> Biomolecular Structure and Mechanism Group
> UC Berkeley
>
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc.
> Still grepping through log files to find problems?  Stop.
> Now Search log events and configuration files using AJAX and a browser.
> Download your FREE copy of Splunk now >> http://get.splunk.com/
> _______________________________________________
> MUMmer-help mailing list
> MUM...@li...
> https://lists.sourceforge.net/lists/listinfo/mummer-help
>