#29 mrsfast-ultra 3.3.1: --mem option & performance

[Fritjof]

Hi,

I'm currently mapping PE-reads with the following command and wondered about performance related to the --mem option. I observed a rapid decline in processing speed with --mem > 4. With --mem > 32 I get a segmentation fault after a while.

mrsfast --search genome.fa --pe --mem 4 --seq1 1.fastq --seq2 2.fastq -e 4 --threads 20 --discordant-vh --min 100 --max 500

The genome has 277711 scaffolds, I have 127924436 paired reads, each 90 bp long

The server I run it on, has 32 cores (64 threads) and 512 GB RAM. What's the most efficient way to run mrsfast. I thought increasing the memory usage would yield also yield better performance. Version is ultra-3.3.1.

Many thanks in advance

Fritjof

[Faraz Hach]

Thanks. We are investigating this issue.

[Iman Sarrafi]

Hi Fritjof,

Yes you definitely get a better running time by increasing the memory limit when dealing with large numbers of reads. This way mrsFAST-Ultra will be able to load and align more reads in each mapping iteration, and therefore will finish in fewer iterations.

I tried to re-create the bug with similar data and parameters as you provided but I am not getting a crash or segmentation fault. I am using a server with large memory to map 127924436 paired-end 90bp reads to a 277711 scaffold reference genome in pe-discordant mode, with --mem 32 and --threads 10 (different number of threads won't cause a problem). The program has been smoothly running for hours with no problem.

I should note that the previous version, mrsFAST-Ultra 3.3.0, had a known issue that caused segmentation fault in discordant mode on genomes with many scaffolds (similar to your case). Though this bug is fixed in version 3.3.1. Please make sure you are using the latest version 3.3.1 as you mentioned.

Yet if the problem persists, please provide us with more details about your data and where you receive the error, and we will further investigate.

Thanks,
Iman