Re: [Bio-bwa-help] keep bwa index in memory
Status: Beta
Brought to you by:
lh3lh3
From: Heng Li <lh...@me...> - 2014-10-15 15:13:40
|
BTW, if your system has /dev/shm and it is large enough, you can copy the index to this directory. Loading index located here should be much faster. Heng On Oct 15, 2014, at 9:11, Ross Whetten <ros...@nc...> wrote: > Thanks, Shawn. That is consistent with my experience also. > Based on a trial run of Bowtie2 using the --mm option, memory usage is much lower than BWA v0.7.5, but the time required is much longer, so the time BWA spends loading the index into memory is less of an issue. Available RAM is not limiting for this case, so I will probably proceed with the bash loop to run BWA in series. > > ________________________________________________________ > | Ross Whetten > | Professor, Department of Forestry & Environmental Resources > | North Carolina State University, Raleigh NC 27695-8008, USA > > On Tue, Oct 14, 2014 at 11:22 PM, Shawn Driscoll <sdr...@sa...> wrote: > I don't know about other devs but if I comment out an option it is usually because related code is broken/incomplete/not yet implemented. > > On Oct 14, 2014 4:38 PM, "Ross Whetten" <ros...@nc...> wrote: > Heng, > Thanks for that pointer - that helped a lot. As I understand the code at > https://github.com/BenLangmead/bowtie2/blob/master/bt2_search.cpp > the --shmem option is commented out for Bowtie2, but the --mm option is available. I don't know enough C++ to predict the result if I uncomment the --shmem option and recompile, or if --mm will work for my purposes, but I can do those experiments and report back for the benefit of the list. > Regards, > Ross > > > ________________________________________________________ > | Ross Whetten > | Professor, Department of Forestry & Environmental Resources > | North Carolina State University, Raleigh NC 27695-8008, USA > > On Tue, Oct 14, 2014 at 6:19 PM, Heng Li <lh...@me...> wrote: > Bowtie2 can use shared memory (—shm). > > Heng > > On Oct 14, 2014, at 17:14, Ross Whetten <ros...@nc...> wrote: > > > Heng, > > Thanks for the quick response! I have not found any mention of that preloading option in the Bowtie2 web manual, but will keep looking. > > Regards, > > Ross > > > > > > ________________________________________________________ > > | Ross Whetten > > | Professor, Department of Forestry & Environmental Resources > > | North Carolina State University, Raleigh NC 27695-8008, USA > > > > On Tue, Oct 14, 2014 at 4:49 PM, Heng Li <lh...@me...> wrote: > > Yes, loading the index for a 22Gb genome takes a while. I am afraid that there are no good solutions. Bowtie2 and STAR both support preloading the index into RAM for repeated use. I have thought about the feature but haven’t got time to implement it. > > > > Heng > > > > On Oct 14, 2014, at 16:45, Ross Whetten <ros...@nc...> wrote: > > > > > Hi all, > > > We are using BWA mem to align paired-end reads for each of a few hundred samples to a large genome (22 Gb). When the program starts, memory usage increases but no output appears until about 40 Gb of RAM is occupied, then alignment results begin appearing on STDOUT. It seems that the program goes through this 'lag phase' (loading the index into RAM?) for each invocation, so when using a bash loop to process hundreds of files in series, the total time taken up by this process is significant. Is there a better way to align each of several hundred separate samples, or a command-line option that tells BWA mem to keep the index in memory for the next cycle through the loop? > > > > > > Thanks, > > > Ross > > > > > > ------------------------------------------------------------------------------ > > > Comprehensive Server Monitoring with Site24x7. > > > Monitor 10 servers for $9/Month. > > > Get alerted through email, SMS, voice calls or mobile push notifications. > > > Take corrective actions from your mobile device. > > > http://p.sf.net/sfu/Zoho_______________________________________________ > > > Bio-bwa-help mailing list > > > Bio...@li... > > > https://lists.sourceforge.net/lists/listinfo/bio-bwa-help > > > > > > > > ------------------------------------------------------------------------------ > Comprehensive Server Monitoring with Site24x7. > Monitor 10 servers for $9/Month. > Get alerted through email, SMS, voice calls or mobile push notifications. > Take corrective actions from your mobile device. > http://p.sf.net/sfu/Zoho > _______________________________________________ > Bio-bwa-help mailing list > Bio...@li... > https://lists.sourceforge.net/lists/listinfo/bio-bwa-help > > |