Re: [wgs-assembler-users] multithreaded CGW ?

[Walenz, B. <bw...@jc...>]

Hi, Heiner, Ole-

The transitive edge reduction was easy to parallelize (I actually did it as
an escape from a harder problem).  The rest of CGW suffers from _lots_ of
thread unsafe code and inherently sequential algorithms.  We'd like to
parallelize, but its a huge project.

On most genomes, the parallel pieces are fast, a few hours at the most.  On
some genomes, they get 'stuck' and will burn all CPUs for days.  If you see
this, drop me a line and I'll tell you how to get around it.  I don't know
what causes it, and the 'fix' damages other assemblies so isn't good to
apply in general.

Glad to hear the overlap store builder is working for you!  It works
fantastic if your filer can support the I/O.  On my dev cluster, the
original ovlStore is faster.

b



On 3/24/13 7:09 AM, "Ole Kristian Tørresen" <o.k...@ib...> wrote:

> Hi Heiner,
> only some parts of cgw is multithreaded now. Among them is transistive
> edge reduction as you can see from the cvs log here:
> http://wgs-assembler.cvs.sourceforge.net/viewvc/wgs-assembler/src/AS_CGW/Trans
> itiveReduction_CGW.c?view=log
> (April 5th). I'm not sure what plans Brian have, but I think it's
> pretty hard to parallelize merge scaffolds aggressively, which is the
> most time-consuming step for me now.
> 
> Ole
> 
> On 24 March 2013 11:30, kuhl <ku...@mo...> wrote:
>> Dear Brian,
>> 
>> I was just wondering if cgw can be run in a parallel mode. I found
>> something saying "48 openmpi threads" in the cgw.out file, but it is still
>> running on a single core. Can I somehow configure cgw to use more cores? I
>> have installed the latest CA from CVS. I have just used the parallel
>> ovlstore creation which is great, thanks for that.
>> 
>> Best wishes, Heiner
>> 
>> 
----------------------------------------------------------------------------->>
-
>> Everyone hates slow websites. So do we.
>> Make your web apps faster with AppDynamics
>> Download AppDynamics Lite for free today:
>> http://p.sf.net/sfu/appdyn_d2d_mar
>> _______________________________________________
>> wgs-assembler-users mailing list
>> wgs...@li...
>> https://lists.sourceforge.net/lists/listinfo/wgs-assembler-users
> 
> ------------------------------------------------------------------------------
> Everyone hates slow websites. So do we.
> Make your web apps faster with AppDynamics
> Download AppDynamics Lite for free today:
> http://p.sf.net/sfu/appdyn_d2d_mar
> _______________________________________________
> wgs-assembler-users mailing list
> wgs...@li...
> https://lists.sourceforge.net/lists/listinfo/wgs-assembler-users