#28 Suggestion: bbmap.sh: showing estimated memory ammount

[Jun Aruga]

Description

I tried to run bbmap.sh (BBMap latet version 38.73) to index on my laptop (Linux Fedora 30).
As the environment's total memory is 7.5GB, I ran it with -Xmx7G.
But bbmap.sh outputs still "This program ran out of memory. Try increasing the -Xmx flag and using tool-specific memory-related parameters.".

My suggestion is

How do you think about adding the estimated necessary memory amount in the error log message?
Maybe you can estimate necessary memory ammount to accomplish the task roughly.
Because I want to know how much addtional memory is necessary. The message helps me to select or prepare the running environment.

$ free -mh
              total        used        free      shared  buff/cache   available
Mem:          7.5Gi       1.3Gi       5.2Gi       197Mi       961Mi       5.7Gi
Swap:         7.5Gi       2.3Gi       5.2Gi

$ bbmap.sh -Xmx7G ref=hg19_main_mask_ribo_animal_allplant_allfungus.fa.gz
java -ea -Xmx7G -Xms7G -cp /home/root/local/bbmap-38.73/current/ align2.BBMap build=1 overwrite=true fastareadlen=500 -Xmx7G ref=hg19_main_mask_ribo_animal_allplant_allfungus.fa.gz
Executing align2.BBMap [build=1, overwrite=true, fastareadlen=500, -Xmx7G, ref=hg19_main_mask_ribo_animal_allplant_allfungus.fa.gz]
Version 38.73
...

Writing chunk 7
Set genome to 1

Loaded Reference:>0.020 seconds.
Loading index for chunk 1-7, build 1
No index available; generating from reference genome: /home/jaruga/data/yamp/ref/index/1/chr1-3_index_k13_c2_b1.block
Indexing threads started for block 0-3
java.lang.OutOfMemoryError: Java heap space
>-at shared.KillSwitch.allocInt1D(KillSwitch.java:195)
>-at align2.IndexMaker4$BlockMaker$CountThread.run(IndexMaker4.java:277)

This program ran out of memory.
Try increasing the -Xmx flag and using tool-specific memory-related parameters.

Environment

• CPU arch: x86_64
• OS: Fedora 30
• Java: 1.8.0_232

$ java -version
openjdk version "1.8.0_232"
OpenJDK Runtime Environment (build 1.8.0_232-b09)
OpenJDK 64-Bit Server VM (build 25.232-b09, mixed mode)

• bbmap.sh

$ bbmap.sh --version
java -ea -Xmx2230m -Xms2230m -cp /home/root/local/bbmap-38.73/current/ align2.BBMap build=1 overwrite=true fastareadlen=500 --version
BBMap version 38.73
For help, please run the shellscript with no parameters, or look in /docs/.

• Input reference fasta: hg19_main_mask_ribo_animal_allplant_allfungus.fa.gz 841MB.
  From http://seqanswers.com/forums/showthread.php?t=42552 .

## Full command log

```
$ bbmap.sh -Xmx7G ref=hg19_main_mask_ribo_animal_allplant_allfungus.fa.gz
java -ea -Xmx7G -Xms7G -cp /home/root/local/bbmap-38.73/current/ align2.BBMap build=1 overwrite=true fastareadlen=500 -Xmx7G ref=hg19_main_mask_ribo_animal_allplant_allfungus.fa.gz
Executing align2.BBMap [build=1, overwrite=true, fastareadlen=500, -Xmx7G, ref=hg19_main_mask_ribo_animal_allplant_allfungus.fa.gz]
Version 38.73

No output file.
Writing reference.
Executing dna.FastaToChromArrays2 [hg19_main_mask_ribo_animal_allplant_allfungus.fa.gz, 1, writeinthread=false, genscaffoldinfo=true, retain, waitforwriting=false, gz=true, maxlen=536670912, writechroms=true, minscaf=1, midpad=300, startpad=8000, stoppad=8000, nodisk=false]

Set genScaffoldInfo=true
Writing chunk 1
Writing chunk 2
Writing chunk 3
Writing chunk 4
Writing chunk 5
Writing chunk 6
Writing chunk 7
Set genome to 1

Loaded Reference: 0.020 seconds.
Loading index for chunk 1-7, build 1
No index available; generating from reference genome: /home/jaruga/data/yamp/ref/index/1/chr1-3_index_k13_c2_b1.block
Indexing threads started for block 0-3
java.lang.OutOfMemoryError: Java heap space
at shared.KillSwitch.allocInt1D(KillSwitch.java:195)
at align2.IndexMaker4$BlockMaker$CountThread.run(IndexMaker4.java:277)

This program ran out of memory.
Try increasing the -Xmx flag and using tool-specific memory-related parameters.
```

Thank you.