Re: [wgs-assembler-users] mer, mertrim running single threaded on large SMP machine

[Brian W. <th...@gm...>]

The option you're looking for is mbtThreads, with a default of 4.

Also look into option mbtBatchSize, which sets how many reads to process
per job.  The default is 1 million, and you've already got at least 48
jobs, so this is probably not an issue.

You can increase the number of jobs running at once with mbtConcurrency.
You should be able to run 20 with the current job size.  Dropping the batch
size should decrease the memory used per job, and so you can then run more
jobs.

On the current jobs, are the WORKING files non-zero size?  If so, then the
compute should be in the multi-threaded stage, and it should be using 4
CPUs.  Check the mertrim.sh (or similar) script in the 0-mertrim directory
to verify that it has "-t 4".  Adding "-v" will make it report the number
of reads processed during the compute, but it won't tell you the number of
threads.  Both of these are to check that the job is done with the data
structure building -- after two days, it definitely should be.

So, in summary, I don't know why you're not getting multiple CPUs on
these.  You can work around the problem by dropping the batch size to make
jobs with about 8gb memory (smaller than 512/48), then run 48 jobs in
parallel.

b


On Wed, Jan 14, 2015 at 12:33 PM, mathog <ma...@ca...> wrote:

> It looks like I set the wgs spec file parameters wrong, because both mer
> and mertrim run single threaded (most of the time) on a machine with 48
> cpus and 530G of RAM. The data is 4 sets of illumina data.  The data was
> loaded into gatekeeper OK, and that was single threaded too, but I can
> see why that might be.  However, after that it began doing these very,
> very slowly:
>
> /home/wgs_project/do_illumina_wgs/./0-mertrim/mertrim.sh 48 \
>    > /home/wgs_project/do_illumina_wgs/./0-mertrim/..0048.err 2>&1
>
> Most of the times when I have checked this either mer or mertrim is
> running at 99% cpu.  On some occasions mertrim has gone a bit higher:
>
> 14097 wgsuser    39  19 18.7g  16g 1304 S 401.7  3.3  49:43.93 merTrim
>
> That is still poor use of this machine, leaving it mostly idle.
>
> Here is the spec file, minus all comments:
>
> utgErrorRate=0.03
> utgErrorLimit=2.5
> ovlErrorRate=0.06
> cnsErrorRate=0.10
> cgwErrorRate=0.10
> merSize = 22
> overlapper=ovl
> unitigger = bog
> utgBubblePopping = 1
> merylMemory   = 128000
> merylThreads    = 25
> ovlHashBits=25
> ovlHashBlockLength=180000000
> ovlThreads          = 2
> ovlConcurrency                = 20
> ovlRefBlockSize  = 32000000
> ovlStoreMemory = 8192 # Mbp
> frgCorrThreads    = 10
> frgCorrConcurrency = 3
> ovlCorrBatchSize  = 1000000
> ovlCorrConcurrency = 25
> cnsConcurrency   = 16
> useGrid          = 0
> scriptOnGrid     = 0
> s_300_qseq.frg
> s_1000_qseq.frg
> s_3000_qseq.frg
> s_5000_qseq.frg
>
> What needs to be tweaked to get mer, mertrim to use more of the
> machine's resources?  This spec file was based on a couple found here
> and there on the web, and I'm sure that many of the memory/threads
> parameters are not optimal.
>
> Thank you,
>
> David Mathog
> ma...@ca...
> Manager, Sequence Analysis Facility, Biology Division, Caltech
>
>
> ------------------------------------------------------------------------------
> New Year. New Location. New Benefits. New Data Center in Ashburn, VA.
> GigeNET is offering a free month of service with a new server in Ashburn.
> Choose from 2 high performing configs, both with 100TB of bandwidth.
> Higher redundancy.Lower latency.Increased capacity.Completely compliant.
> http://p.sf.net/sfu/gigenet
> _______________________________________________
> wgs-assembler-users mailing list
> wgs...@li...
> https://lists.sourceforge.net/lists/listinfo/wgs-assembler-users
>