Menu
▾
▴
sqlgrey-users
|
From: Casey F. <cfe...@wi...> - 2006-09-11 18:31:02
|
We've been testing an implementation of sqlgrey 1.6.7 and have come across a memory leak fairly rapidly. Under our current load (30,000 msgs/day, 6000 users), sqlgrey's heap seems to be growing by about 1MB every 5 minutes. Another copy of sqlgrey not actively in use does not seem to be gaining any memory, so it looks like this issue only occurs when the daemon is in active use. I noticed a previous thread on this list about memory leaks in 1.6.7, but no resolution. Has anyone else seen this: Our configuration: Solaris 9 Postfix 2.2.10 Perl 5.8.4 Sqlgrey 1.6.7 Thanks for any help you may be able to provide. -- --------------------------------------------- Casey Feskens <cfe...@wi...> System Administrator/Network Svcs. Consultant Willamette Integrated Technology Services Willamette University, Salem, OR --------------------------------------------- |
|
From: Lionel B. <lio...@bo...> - 2006-09-11 19:21:53
|
Casey Feskens wrote the following on 11.09.2006 20:30 : > We've been testing an implementation of sqlgrey 1.6.7 and have > come across a memory leak fairly rapidly. Under our current > load (30,000 msgs/day, 6000 users), sqlgrey's heap seems to be > growing by about 1MB every 5 minutes. > > Another copy of sqlgrey not actively in use does not seem to be > gaining any memory, so it looks like this issue only occurs when > the daemon is in active use. > > I noticed a previous thread on this list about memory leaks in > 1.6.7, but no resolution. Has anyone else seen this: > > Our configuration: > > Solaris 9 > Postfix 2.2.10 > Perl 5.8.4 > Sqlgrey 1.6.7 > > Thanks for any help you may be able to provide. > > I recently had an unrelated problem with DBD-Pg and investigated the "Changes" files of DBI (Perl's database API) and DBD-Pg (Perl DBI's PostgreSQL driver) by downloading the latest versions available from CPAN. I've spotted reports of memory leaks fixed in DBI 1.52 (latest available) and DBD-Pg 1.44 (current is 1.49, which, for me, solved a prepared statement problem when DBD-Pg was compiled with GCC 4.1.1). You should check your DBI version and if you use PostgreSQL, please check DBD-Pg's version too. That said, given the simple algorithms used it's highly unlikely that SQLgrey would suffer from memory leaks (you have to do pretty fancy stuff to make Perl lost track of the memory it should free). I believe most of the currently reported problems with 1.6.x are either bugs in DBI or the DBD driver, which actually do pretty fancy stuff :-) Lionel. |
|
From: Casey F. <cfe...@wi...> - 2006-09-11 20:15:04
|
Lionel Bouton wrote: > Casey Feskens wrote the following on 11.09.2006 20:30 : > >> We've been testing an implementation of sqlgrey 1.6.7 and have >> come across a memory leak fairly rapidly. Under our current >> load (30,000 msgs/day, 6000 users), sqlgrey's heap seems to be >> growing by about 1MB every 5 minutes. >> >> Another copy of sqlgrey not actively in use does not seem to be >> gaining any memory, so it looks like this issue only occurs when >> the daemon is in active use. >> >> I noticed a previous thread on this list about memory leaks in >> 1.6.7, but no resolution. Has anyone else seen this: >> >> Our configuration: >> >> Solaris 9 >> Postfix 2.2.10 >> Perl 5.8.4 >> Sqlgrey 1.6.7 >> >> Thanks for any help you may be able to provide. >> >> >> > > I recently had an unrelated problem with DBD-Pg and investigated the > "Changes" files of DBI (Perl's database API) and DBD-Pg (Perl DBI's > PostgreSQL driver) by downloading the latest versions available from > CPAN. I've spotted reports of memory leaks fixed in DBI 1.52 (latest > available) and DBD-Pg 1.44 (current is 1.49, which, for me, solved a > prepared statement problem when DBD-Pg was compiled with GCC 4.1.1). You > should check your DBI version and if you use PostgreSQL, please check > DBD-Pg's version too. > > That said, given the simple algorithms used it's highly unlikely that > SQLgrey would suffer from memory leaks (you have to do pretty fancy > stuff to make Perl lost track of the memory it should free). I believe > most of the currently reported problems with 1.6.x are either bugs in > DBI or the DBD driver, which actually do pretty fancy stuff :-) > > Thanks, for the tip. We are running DBI 1.52 and DBD-Pg 1.49. I'm going to do some further profiling and see what I find out. -- --------------------------------------------- Casey Feskens <cfe...@wi...> System Administrator/Network Svcs. Consultant Willamette Integrated Technology Services Willamette University, Salem, OR --------------------------------------------- |
|
From: Andrew D. <and...@gm...> - 2006-09-12 02:45:12
|
Hello Casey, Monday, September 11, 2006, 2:14:51 PM, Casey Feskens wrote: > Lionel Bouton wrote: >> I recently had an unrelated problem with DBD-Pg and investigated the >> "Changes" files of DBI (Perl's database API) and DBD-Pg (Perl DBI's >> PostgreSQL driver) by downloading the latest versions available from >> CPAN. I've spotted reports of memory leaks fixed in DBI 1.52 (latest >> available) and DBD-Pg 1.44 (current is 1.49, which, for me, solved a >> prepared statement problem when DBD-Pg was compiled with GCC 4.1.1). You >> should check your DBI version and if you use PostgreSQL, please check >> DBD-Pg's version too. > Thanks, for the tip. We are running DBI 1.52 and DBD-Pg 1.49. > I'm going to do some further profiling and see what I find out. I use sqlgrey on a SuSE 10.0 box with postgres, and I've seen the postmaster leak. If I restart sqlgrey the memory in postmaster releases. Looks like I have perl-DBI-1.48-2, and I found DBD-Pg-1.47 in my .cpan directory, though no-where else, so I'm not even sure if I'm using DBD-Pg. -- Best regards, Andrew mailto:and...@gm... |
|
From: Dan M. <dan...@da...> - 2006-09-12 04:34:24
|
On Mon, Sep 11, 2006 at 08:44:57PM -0600, Andrew Diederich wrote: > Looks like I have perl-DBI-1.48-2, and I found DBD-Pg-1.47 > in my .cpan directory, though no-where else, so I'm not even sure if > I'm using DBD-Pg. Andrew, To determine which DBD you are using, just check the sqlgrey.conf. For example: muratore [~] > grep ^db_type /usr/local/etc/sqlgrey/sqlgrey.conf db_type = mysql Dan -- | Daniel R Mason | "An ignorant people is | | dan...@da... | the blind instrument of | | www.danmason.net | its own destruction." | | Unix Systems Engineer | Simon Bolivar, Liberator | |
|
From: Casey F. <cfe...@wi...> - 2006-09-12 14:25:40
|
Lionel Bouton wrote: > Casey Feskens wrote the following on 11.09.2006 20:30 : > >> We've been testing an implementation of sqlgrey 1.6.7 and have >> come across a memory leak fairly rapidly. Under our current >> load (30,000 msgs/day, 6000 users), sqlgrey's heap seems to be >> growing by about 1MB every 5 minutes. >> >> Another copy of sqlgrey not actively in use does not seem to be >> gaining any memory, so it looks like this issue only occurs when >> the daemon is in active use. >> >> I noticed a previous thread on this list about memory leaks in >> 1.6.7, but no resolution. Has anyone else seen this: >> >> Our configuration: >> >> Solaris 9 >> Postfix 2.2.10 >> Perl 5.8.4 >> Sqlgrey 1.6.7 >> >> Thanks for any help you may be able to provide. >> >> >> > > I recently had an unrelated problem with DBD-Pg and investigated the > "Changes" files of DBI (Perl's database API) and DBD-Pg (Perl DBI's > PostgreSQL driver) by downloading the latest versions available from > CPAN. I've spotted reports of memory leaks fixed in DBI 1.52 (latest > available) and DBD-Pg 1.44 (current is 1.49, which, for me, solved a > prepared statement problem when DBD-Pg was compiled with GCC 4.1.1). You > should check your DBI version and if you use PostgreSQL, please check > DBD-Pg's version too. > > That said, given the simple algorithms used it's highly unlikely that > SQLgrey would suffer from memory leaks (you have to do pretty fancy > stuff to make Perl lost track of the memory it should free). I believe > most of the currently reported problems with 1.6.x are either bugs in > DBI or the DBD driver, which actually do pretty fancy stuff :-) > > Lionel. > > > For what it's worth, after reading through a bunch of DBD-Pg release notes and noticing all the memory leaks fixed in the 1.4x series, I rolled back to DBD-Pg 1.32. Since then we've run nearly a day now without any gain in process size. -- --------------------------------------------- Casey Feskens <cfe...@wi...> System Administrator/Network Svcs. Consultant Willamette Integrated Technology Services Willamette University, Salem, OR --------------------------------------------- |
|
From: Lionel B. <lio...@bo...> - 2006-09-12 19:46:44
|
Casey Feskens wrote the following on 12.09.2006 16:25 : > [...] > For what it's worth, after reading through a bunch of DBD-Pg > release notes and noticing all the memory leaks fixed in the 1.4x > series, I rolled back to DBD-Pg 1.32. Since then we've run > nearly a day now without any gain in process size. > > Hum. Even with low traffic, I can see DBD-Pg 1.49 leaking on my system too (and clearly the PostgreSQL process is growing, which makes me think that there are server-side prepared statements being leaked by DBD-Pg). At least SQLgrey is stable... Too bad the driver is not on par with the database. I'll lurk around mailing-lists to see if the problem is known by the developper(s). Lionel. |
|
From: Steve H. <st...@th...> - 2006-10-05 15:53:08
|
On Tue, 2006-09-12 at 21:46 +0200, Lionel Bouton wrote: > Hum. Even with low traffic, I can see DBD-Pg 1.49 leaking on my system > too (and clearly the PostgreSQL process is growing, which makes me think > that there are server-side prepared statements being leaked by DBD-Pg). We have two servers running sqlgrey and sharing a database. One has a DBD-Pg version of 1.22 and the other has 1.49. The one with 1.49 is the m/c that runs the postgresql database We dont see the postgresql process growing too much. Load is about 7500 email per day shared across the two servers. Steve -- thorNET Internet Services, Consultancy & Training www.thornet.co.uk |
×
Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.
Thanks for helping keep SourceForge clean.
X