|
From: Sabrina L. <sla...@am...> - 2007-03-19 08:04:37
|
Hi, One of our developers got the following valgrind error with memcheck: Valgrind's memory management: out of memory: newSuperblock's request for 5881856 bytes failed. 5168607232 bytes have already been allocated. Valgrind cannot continue. Sorry. When I monitored the process using 'top', I checked that just before stopping, the value of the field VIRT was 5055MB and RES around 1.9GB. He's binary is 10MB big. I guess he faced a valgrind limitation, but before giving up, I wanted to check with you if there were a way to solve this Pb. Thx in advance for your help. Rgds, Sabrina |
|
From: Michael P. <md...@tr...> - 2007-07-27 03:22:27
|
I see the following error with Valgrind 3.2.3 when my program starts up on an amd64 machine with 4 GB RAM: --10800-- REDIR: 0x3F9C677750 (memmove) redirected to 0x4A07CD0 (memmove) --10800-- REDIR: 0x3F9C676950 (strncpy) redirected to 0x4A08010 (strncpy) --10800-- REDIR: 0x3F9C676890 (strncmp) redirected to 0x4A07A50 (strncmp) --10800-- REDIR: 0x3F9C677D00 (stpcpy) redirected to 0x4A07F30 (stpcpy) error: Unable to open database file 'pgsql.help' for reading: No such file or directory --10800-- REDIR: 0x3F9C677100 (memchr) redirected to 0x4A07B80 (memchr) Valgrind's memory management: out of memory: newSuperblock's request for 1048576 bytes failed. 61566976 bytes have already been allocated. Valgrind cannot continue. Sorry. With Ubuntu's valgrind_3.2.1-1ubuntu2 package from Feisty, it dies in almost the same place: --10831-- REDIR: 0x4EB2750 (memmove) redirected to 0x4A07BB0 (memmove) --10831-- REDIR: 0x4EB1950 (strncpy) redirected to 0x4A083D0 (strncpy) --10831-- REDIR: 0x4EB1890 (strncmp) redirected to 0x4A07980 (strncmp) --10831-- REDIR: 0x4EB2D00 (stpcpy) redirected to 0x4A07F10 (stpcpy) error: Unable to open database file 'pgsql.help' for reading: No such file or directory --10831-- REDIR: 0x4EB2100 (memchr) redirected to 0x4A07AB0 (memchr) warning: Unable to mmap database file 'mod-helpserv.help' (falling back to stdio): Cannot allocate memory Valgrind's memory management: out of memory: memcheck:allocate new SecMap's request for 16384 bytes failed. 64569344 bytes have already been allocated. Valgrind cannot continue. Sorry. Bypassing the mmap() path for reading those files only makes the ENOMEM line go away; the number of bytes allocated are not changed. A similar problem was reported in March. In my case, it does not appear to be a ulimit issue: % ulimit -a -t: cpu time (seconds) unlimited -f: file size (blocks) unlimited -d: data seg size (kbytes) unlimited -s: stack size (kbytes) 8192 -c: core file size (blocks) unlimited -m: resident set size (kbytes) unlimited -u: processes unlimited -n: file descriptors 32768 -l: locked-in-memory size (kb) unlimited -v: address space (kb) unlimited -x: file locks unlimited -i: pending signals unlimited -q: bytes in POSIX msg queues unlimited -e: max nice 20 -r: max rt priority 0 At that time, Julian asked the reporter how many 1 MB blocks could be allocated in a loop before running out of memory. Without valgrind, I interrupted the loop on my machine after it reported some 400 GB allocated (the machine was starting to swap; memory overcommit was on). With valgrind, I interrupted the loop after a little more than 6 GB, after which valgrind reported: ==11093== malloc/free: in use at exit: 6,800,015,360 bytes in 6,485 blocks. ==11093== malloc/free: 6,485 allocs, 0 frees, 6,800,015,360 bytes allocated. Since it is not simply memory exhaustion, how should I diagnose the "out of memory" error? Michael Poole |
|
From: Julian S. <js...@ac...> - 2007-03-19 11:20:57
|
> When I monitored the process using 'top', I checked that just before > stopping, the value of the field VIRT was 5055MB and RES around 1.9GB. > He's binary is 10MB big. > I guess he faced a valgrind limitation, No .. valgrind 3.2.X should be able to go up to 32GB on a 64-bit machine. Most likely this machine ran out of swap space. J |
|
From: Sabrina L. <sla...@am...> - 2007-03-19 15:49:23
|
> valgrind 3.2.X should be able to go up to 32GB on a 64-bit machine. Yes, that's what I thought too (the machine is a SuSe SLES9 on an x86_64 architecture - AMD Opteron). > Most likely this machine ran out of swap space. No, this is what I checked first of course. When I monitored the process, there was no swap space in use and only 17% of the memory was used. Rgds, Sabrina Julian Seward <js...@ac...> To val...@li... cc bcc Subject Re: [Valgrind-users] Valgrind's memory management: out of memory Julian Seward <js...@ac...> Sent by: val...@li... 19/03/2007 12:18 > When I monitored the process using 'top', I checked that just before > stopping, the value of the field VIRT was 5055MB and RES around 1.9GB. > He's binary is 10MB big. > I guess he faced a valgrind limitation, No .. valgrind 3.2.X should be able to go up to 32GB on a 64-bit machine. Most likely this machine ran out of swap space. J ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV _______________________________________________ Valgrind-users mailing list Val...@li... https://lists.sourceforge.net/lists/listinfo/valgrind-users |
|
From: Julian S. <js...@ac...> - 2007-03-19 16:01:43
|
Well, I don't know. Can you check: * you are using 3.2.3 ? * you don't have some per-user specific memory limits? (ulimit?) If you just write a normal program which mallocs 1M blocks in a loop, how much can it allocate before running out of memory? Both when running normally and running on V ? J On Monday 19 March 2007 15:48, Sabrina Lautier wrote: > > valgrind 3.2.X should be able to go up to 32GB on a 64-bit machine. > > Yes, that's what I thought too (the machine is a SuSe SLES9 on an x86_64 > architecture - AMD Opteron). > > > Most likely this machine ran out of swap space. > > No, this is what I checked first of course. > When I monitored the process, there was no swap space in use and only 17% > of the memory was used. > > Rgds, > Sabrina > > > > Julian Seward <js...@ac...> > To > val...@li... > cc > > bcc > > Subject > Re: [Valgrind-users] Valgrind's memory management: out of memory > > > > > > Julian Seward <js...@ac...> > Sent by: val...@li... > 19/03/2007 12:18 > > > When I monitored the process using 'top', I checked that just before > > stopping, the value of the field VIRT was 5055MB and RES around 1.9GB. > > He's binary is 10MB big. > > I guess he faced a valgrind limitation, > > No .. valgrind 3.2.X should be able to go up to 32GB on a 64-bit > machine. Most likely this machine ran out of swap space. > > J > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share > your > opinions on IT & business topics through brief surveys-and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Valgrind-users mailing list > Val...@li... > https://lists.sourceforge.net/lists/listinfo/valgrind-users |
|
From: Sabrina L. <sla...@am...> - 2007-03-21 10:38:12
|
> you are using 3.2.3 ? Yes, I confirm: $ valgrind --version valgrind-3.2.3 > you don't have some per-user specific memory limits? (ulimit?) No. Logged as the user, here is the output of the ulimit command: $ ulimit -a cpu time (seconds) unlimited file size (blocks) unlimited data seg size (kbytes) unlimited stack size (kbytes) unlimited core file size (blocks) 16777216 resident set size (kbytes) unlimited processes 73727 file descriptors 32768 locked-in-memory size (kb) unlimited address space (kb) 5242880 file locks unlimited About the prg, I'm going to create and launch it. I'll then let you know. Rgds, Sabtrina Julian Seward <js...@ac...> To val...@li... cc Sabrina Lautier <sla...@am...> bcc Subject Re: [Valgrind-users] Valgrind's memory management: out of memory Julian Seward <js...@ac...> 19/03/2007 16:58 Well, I don't know. Can you check: * you are using 3.2.3 ? * you don't have some per-user specific memory limits? (ulimit?) If you just write a normal program which mallocs 1M blocks in a loop, how much can it allocate before running out of memory? Both when running normally and running on V ? J On Monday 19 March 2007 15:48, Sabrina Lautier wrote: > > valgrind 3.2.X should be able to go up to 32GB on a 64-bit machine. > > Yes, that's what I thought too (the machine is a SuSe SLES9 on an x86_64 > architecture - AMD Opteron). > > > Most likely this machine ran out of swap space. > > No, this is what I checked first of course. > When I monitored the process, there was no swap space in use and only 17% > of the memory was used. > > Rgds, > Sabrina > > > > Julian Seward <js...@ac...> > To > val...@li... > cc > > bcc > > Subject > Re: [Valgrind-users] Valgrind's memory management: out of memory > > > > > > Julian Seward <js...@ac...> > Sent by: val...@li... > 19/03/2007 12:18 > > > When I monitored the process using 'top', I checked that just before > > stopping, the value of the field VIRT was 5055MB and RES around 1.9GB. > > He's binary is 10MB big. > > I guess he faced a valgrind limitation, > > No .. valgrind 3.2.X should be able to go up to 32GB on a 64-bit > machine. Most likely this machine ran out of swap space. > > J > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share > your > opinions on IT & business topics through brief surveys-and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Valgrind-users mailing list > Val...@li... > https://lists.sourceforge.net/lists/listinfo/valgrind-users |
|
From: Julian S. <js...@ac...> - 2007-03-21 11:16:37
|
> > you don't have some per-user specific memory limits? (ulimit?) > address space (kb) 5242880 This is a per-user memory (address space) limit - to around 5GB, which is what you said you observed. J |
|
From: Sabrina L. <sla...@am...> - 2007-03-21 15:30:56
|
Thx a lot, I missed it ! Thx again for your help. Rgds, Sabrina Julian Seward <js...@ac...> To Sabrina Lautier <sla...@am...> cc val...@li... bcc Subject Re: [Valgrind-users] Valgrind's memory management: out of memory Julian Seward <js...@ac...> 21/03/2007 11:51 > > you don't have some per-user specific memory limits? (ulimit?) > address space (kb) 5242880 This is a per-user memory (address space) limit - to around 5GB, which is what you said you observed. J |
|
From: Nicholas N. <nj...@cs...> - 2007-03-22 00:26:06
|
On Wed, 21 Mar 2007, Julian Seward wrote:
>>> you don't have some per-user specific memory limits? (ulimit?)
>
>> address space (kb) 5242880
>
> This is a per-user memory (address space) limit - to around 5GB, which
> is what you said you observed.
We should improve that error message. Here's my suggestion, can anyone
improve it?
Valgrind's memory management: out of memory:
newSuperblock's request for 5881856 bytes failed.
5168607232 bytes have already been allocated.
There are several possible reasons for this.
- You have some kind of memory limit in place. Look at the output
of 'ulimit -a'. Is there a limit on the size of virtual memory,
or address space?
- You have run out of swap space.
- Valgrind has a bug. If you think this is the case or you are not
sure, please let us know and we'll try to fix it.
Please note that programs can take substantially more memory than normal
when running under Valgrind tools, eg. up to twice, or more, depending
on the tool.
Either way, Valgrind cannot continue. Sorry.
Are there any other reasons why this might happen?
Nick
|
|
From: Julian S. <js...@ac...> - 2007-03-22 00:38:09
|
On Thursday 22 March 2007 00:25, Nicholas Nethercote wrote: > On Wed, 21 Mar 2007, Julian Seward wrote: > >>> you don't have some per-user specific memory limits? (ulimit?) > >> > >> address space (kb) 5242880 > > > > This is a per-user memory (address space) limit - to around 5GB, which > > is what you said you observed. > > We should improve that error message. I agree. > Are there any other reasons why this might happen? Not that I can think of. It'd be worth stating what the expected limits are on a 64-bit machine On a 64 bit machine, Valgrind should be able to make use of up 32GB memory. On a 32 bit machine, Valgrind should be able to use all the memory available to a single process, up to 4GB if that's how you have your kernel configured. Most 32 bit Linux setups allow a maximum of 3GB per process. A bit verbose, but it does give people a feel for what to expect. J |