|
From: Richard H. <rd...@lb...> - 2011-04-19 18:52:38
|
Hi, That was my original bug report. We're using mod_perl 2.0.4 on Apache 2.2.9, perl 5.10.0. Consistently on our servers, it looks like gbrowse spawned child processes periodically never finish. Our typical apache processes run as: /usr/sbin/apache2 -f /scratch/www-data/apache2/browser/apache2.conf -k start whereas long running processes that eventually consume multiple gigs of RAM and >50% CPU show up in the process list as: /scratch/www-data -f /scratch/www-data/apache2/browser/apache2.conf -k start When I run strace against one of these runaway processes, this occasionally reports a constant stream of mysql queries against our Bio::SeqFeature::Store database running select statements for subfeatures. We've had to use Apache:Resource in our perl.conf to kill any process consuming more than 5 Gb of RAM: PerlSetEnv PERL_RLIMIT_DATA 1024:2048 PerlSetEnv PERL_RLIMIT_AS 5000:6000 PerlChildInitHandler Apache2::Resource and also have a cron job to restart/cycle Apache twice a day, but these are clearly bandaids over a larger problem. I am in the process of upgrading to BioPerl 1.6.9 and the latest Gbrowse 2 build on a test machine to see if those cumulative bug fixes help. (We're still on Gbrowse 2.16). Also, I've seen a few reports of potential memory leaks in either DBI or DBD::mysql that could be the culprit, although this will require more testing. https://rt.cpan.org/Public/Bug/Display.html?id=67435 https://rt.cpan.org/Public/Bug/Display.html?id=66596 Any and all comments welcome, thanks! -- Richard D. Hayes, Ph.D. Joint Genome Institute / Lawrence Berkeley National Lab http://www.phytozome.net On Tue, Apr 19, 2011 at 10:05 AM, Jayaraman, Pushkala <pja...@mc...>wrote: > RUN(): POST: http://test.rgd.mcw.edu/fgb2/gbrowse/rgd_904/action=retrieve_multiple;track_ids > > > =HumanSyntenyBlock;track_ids=RatSyntenyBlock;track_ids=dbSNP_Track;tk_HumanSyntenyBlock=26117e647b7417d62fcfd434df39df4b;tk_RatSyntenyBlock=9b2c1f8a09b0e005fd06638a9645eb78;tk_db > > SNP_Track=a653ea6516e414ecf2af259e986b2442;_= at > /usr/lib64/perl5/site_perl/5.8.8/x86_64-linux-thread-multi/Bio/Graphics/Browser2/Render.pm > line 201., referer: http://test.rgd.mc > > w.edu/fgb2/gbrowse/rgd_904/ > > [Tue Apr 19 11:56:46 2011] [warn] [client 141.106.161.52] mod_fcgid: > stderr: [11552] add_user_tracks() at > /usr/lib64/perl5/site_perl/5.8.8/x86_64-linux-, referer: http://test.rgd > > .mcw.edu/fgb2/gbrowse/rgd_904/ > > [Tue Apr 19 11:56:46 2011] [warn] [client 141.106.161.52] mod_fcgid: > stderr: thread-multi/Bio/Graphics/Browser2/Render.pm line 209., referer: > http://test.rgd.mcw.edu/fgb2/gbrowse > > /rgd_904/ > > [Tue Apr 19 11:56:46 2011] [warn] [client 141.106.161.52] mod_fcgid: > stderr: [11552] testing for asynchronous event() at > /usr/lib64/perl5/site_perl/5.8.8/x86_64-linux-thread-mult > > i/Bio/Graphics/Browser2/Render.pm line 214., referer: > http://test.rgd.mcw.edu/fgb2/gbrowse/rgd_904/ > > [Tue Apr 19 11:56:46 2011] [warn] [client 141.106.161.52] mod_fcgid: > stderr: [11552] asynchronous exit at > /usr/lib64/perl5/site_perl/5.8.8/x86_64-linux-thread-multi/Bio/Graphics/ > > Browser2/Render.pm line 216., referer: > http://test.rgd.mcw.edu/fgb2/gbrowse/rgd_904/ > > *[Tue Apr 19 11:57:44 2011] [warn] [client 141.106.161.52] mod_fcgid: read > data timeout in 40 seconds, referer: > http://test.rgd.mcw.edu/fgb2/gbrowse/rgd_904/* > > *[Tue Apr 19 11:57:44 2011] [error] [client 141.106.161.52] Premature end > of script headers: gbrowse, referer: > http://test.rgd.mcw.edu/fgb2/gbrowse/rgd_904/* > > *[Tue Apr 19 11:57:53 2011] [warn] [client 141.106.161.52] mod_fcgid: read > data timeout in 40 seconds, referer: > http://test.rgd.mcw.edu/fgb2/gbrowse/rgd_904/* > > *[Tue Apr 19 11:57:53 2011] [error] [client 141.106.161.52] Premature end > of script headers: gbrowse_img, referer: > http://test.rgd.mcw.edu/fgb2/gbrowse/rgd_904/*** > > > > I see these errors in my error logs.. I see a “read data timeout” .. how do > I counter that? > > > > Pushkala Jayaraman > > Programmer/Analyst > > Rat Genome Database > > Human and Molecular Genetics Center > > Medical College of Wisconsin > > Email: pja...@mc... > > Work: 414-955-2229 > > www.rgd.mcw.edu > > > > *From:* Jayaraman, Pushkala [mailto:pja...@mc...] > *Sent:* Tuesday, April 19, 2011 11:34 AM > *To:* gmo...@li...; gmo...@li... > *Subject:* [Gmod-gbrowse] Any colutions to the GBrowse memory leak > problem? > > > > Hello, > > I am referring to this particular issue that we seem to be experiencing as > well… > > http://comments.gmane.org/gmane.science.biology.gmod.gbrowse/7572 > > > > Our Gbrowse process simply hangs even if we have just three tracks open for > a range of 1MB. Reloading the page seems to make it okay and everything > renders quickly.. > > Our Gbrowse process is 2.15. > > > > The issue ( follow link) reports that it is some sort of memory leak in > gbrowse.. > > > > Would you guys out there know any solution to this problem.. or a fix that > we might have to integrate? > > > > > > Pushkala Jayaraman > > Programmer/Analyst > > Rat Genome Database > > Human and Molecular Genetics Center > > Medical College of Wisconsin > > Email: pja...@mc... > > Work: 414-955-2229 > > www.rgd.mcw.edu > > > > > ------------------------------------------------------------------------------ > Benefiting from Server Virtualization: Beyond Initial Workload > Consolidation -- Increasing the use of server virtualization is a top > priority.Virtualization can reduce costs, simplify management, and improve > application availability and disaster protection. Learn more about boosting > the value of server virtualization. http://p.sf.net/sfu/vmware-sfdev2dev > _______________________________________________ > Gmod-gbrowse mailing list > Gmo...@li... > https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse > > |