|
From: Ashley P. <as...@qu...> - 2004-07-27 14:02:29
|
Hi, I've had a problem running valgrind on my desktop for a while (running debian unstable) but a recent upgrade has changed the failure mode from a segv to a assertion failure which allowed me to track the source of the problem better. I'm running the official debian package which contains reports itself as version 'valgrind-2.1.2' Basically V used to segv on startup and I was never able to get a stacktrace, however now I get the following assertion. I changed the code to enable the debugging printfs in the failing function: ashley:ashley> valgrind true client_base 0 (440MB) client_mapbase 1B8E4000 (883MB) client_end 52C00000 (1MB) shadow_base 52D00000 (1489MB) shadow_end AFE80000 (1MB) valgrind_base B0000000 (256MB) valgrind_end C0000000 valgrind: vg_main.c:553 (layout_remaining_space): Assertion `(void*)-1 != vres' failed. ashley:ashley> If i strace the process then I get the following: mmap2(0x52c00000, 1048576, PROT_NONE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x52c00000 munmap(0, 1388314624) = 0 mmap2(0x52d00000, 1561853952, PROT_NONE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = -1 ENOMEM (Cannot allocate memory) write(1, "\nvalgrind: vg_main.c:553 (layout"..., 89 valgrind: vg_main.c:553 (layout_remaining_space): Assertion `(void*)-1 != vres' failed. ) = 89 exit_group(2) = ? Does anybody recognise this problem or do I need to do some more digging? Oddly enough I can't run --help without specifying --tool=none or again the program crashes. Running --tool=none or --tool=corecheck runs fine. Ashley, |
|
From: Tom H. <th...@cy...> - 2004-07-27 14:21:11
|
In message <1090936850.24525.361.camel@ashley>
Ashley Pittman <as...@qu...> wrote:
> If i strace the process then I get the following:
>
> mmap2(0x52c00000, 1048576, PROT_NONE,
> MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x52c00000
> munmap(0, 1388314624) = 0
> mmap2(0x52d00000, 1561853952, PROT_NONE,
> MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = -1 ENOMEM (Cannot allocate
> memory)
> write(1, "\nvalgrind: vg_main.c:553 (layout"..., 89
> valgrind: vg_main.c:553 (layout_remaining_space): Assertion `(void*)-1
> != vres' failed.
> ) = 89
> exit_group(2) = ?
>
> Does anybody recognise this problem or do I need to do some more
> digging?
You got a address space limit set that means there isn't enough
space for valgrind to run - try "ulimit -a" and see what it says.
Tom
--
Tom Hughes (th...@cy...)
Software Engineer, Cyberscience Corporation
http://www.cyberscience.com/
|
|
From: Nicholas N. <nj...@ca...> - 2004-07-27 14:35:44
|
On Tue, 27 Jul 2004, Ashley Pittman wrote: > I've had a problem running valgrind on my desktop for a while (running > debian unstable) but a recent upgrade has changed the failure mode from > a segv to a assertion failure which allowed me to track the source of > the problem better. Good! > I'm running the official debian package which contains reports itself as > version 'valgrind-2.1.2' > > Basically V used to segv on startup and I was never able to get a > stacktrace, however now I get the following assertion. I changed the > code to enable the debugging printfs in the failing function: > > ashley:ashley> valgrind true > client_base 0 (440MB) > client_mapbase 1B8E4000 (883MB) > client_end 52C00000 (1MB) > shadow_base 52D00000 (1489MB) > shadow_end AFE80000 (1MB) > valgrind_base B0000000 (256MB) > valgrind_end C0000000 That looks as expected... > valgrind: vg_main.c:553 (layout_remaining_space): Assertion `(void*)-1 > != vres' failed. > ashley:ashley> Valgrind is trying to grab a large slice of address space for itself here. Does this failure occur for all programs? Do you know if any library loads itself at a fixed high address in the range 0x52d00000..0xafe80000? If you don't know, assuming you have a source distribution (if not, you'll need to get one from valgrind.kde.org), can you please do this: - insert "VG_(read)(0,0,1)" just before line 553 in vg_main.c. - recompile Valgrind, run it - when it pauses on the VG_(read)() line, find out its process id (use "ps -af" in another xterm), open up /proc/<pid>/maps, and tell us the contents. Hopefully that should indicate what the problem is. Thanks. N |
|
From: Ashley P. <as...@qu...> - 2004-07-27 14:43:29
|
On Tue, 2004-07-27 at 15:20, Tom Hughes wrote: > In message <1090936850.24525.361.camel@ashley> > Ashley Pittman <as...@qu...> wrote: > > > If i strace the process then I get the following: > > > > mmap2(0x52c00000, 1048576, PROT_NONE, > > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x52c00000 > > munmap(0, 1388314624) = 0 > > mmap2(0x52d00000, 1561853952, PROT_NONE, > > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = -1 ENOMEM (Cannot allocate > > memory) > > write(1, "\nvalgrind: vg_main.c:553 (layout"..., 89 > > valgrind: vg_main.c:553 (layout_remaining_space): Assertion `(void*)-1 > > != vres' failed. > > ) = 89 > > exit_group(2) = ? > > > > Does anybody recognise this problem or do I need to do some more > > digging? > > You got a address space limit set that means there isn't enough > space for valgrind to run - try "ulimit -a" and see what it says. Should all be normal to the best of my knowledge. I have only got 256M of ram but V is trying to mmap() 1.5Gb which can't be right can it? ashley:ashley> ulimit -a core file size (blocks, -c) unlimited data seg size (kbytes, -d) unlimited file size (blocks, -f) unlimited max locked memory (kbytes, -l) unlimited max memory size (kbytes, -m) unlimited open files (-n) 1024 pipe size (512 bytes, -p) 8 stack size (kbytes, -s) 8192 cpu time (seconds, -t) unlimited max user processes (-u) unlimited virtual memory (kbytes, -v) unlimited |
|
From: Tom H. <th...@cy...> - 2004-07-27 14:52:24
|
In message <1090939066.5887.4.camel@ashley>
Ashley Pittman <as...@qu...> wrote:
> On Tue, 2004-07-27 at 15:20, Tom Hughes wrote:
>
>> You got a address space limit set that means there isn't enough
>> space for valgrind to run - try "ulimit -a" and see what it says.
>
> Should all be normal to the best of my knowledge. I have only got 256M
> of ram but V is trying to mmap() 1.5Gb which can't be right can it?
That's what swap is for ;-)
Seriously, valgrind isn't actually going to touch that address space
so you don't actually need physical memory (or even swap) for it
unless you've turned off overcommitting of swap for some reason.
I have just remembered something else however, which is that certain
older kernels didn't seem to like those very large mmap calls. What
distribution are you using, and what kernel?
Tom
--
Tom Hughes (th...@cy...)
Software Engineer, Cyberscience Corporation
http://www.cyberscience.com/
|
|
From: Ashley P. <as...@qu...> - 2004-07-27 14:57:36
|
On Tue, 2004-07-27 at 15:35, Nicholas Nethercote wrote: > - when it pauses on the VG_(read)() line, find out its process id (use > "ps -af" in another xterm), open up /proc/<pid>/maps, and tell us the > contents. Here you go, hope the formatting works ok... ashley:ashley> cat /proc/7094/maps 52c00000-52d00000 ---p 00000000 00:00 0 52d00000-b0000000 ---p 4ac0c000 03:06 43776 /tmp/.pad.7094.1 (deleted) b0000000-b009c000 r-xp 00000000 03:07 674313 /mnt/redhat/debian-usr/usr/lib/valgrind/stage2 b009c000-b009d000 rw-p 0009b000 03:07 674313 /mnt/redhat/debian-usr/usr/lib/valgrind/stage2 b009d000-b01f7000 rw-p 00000000 00:00 0 b01f7000-b1000000 ---p 00000000 03:06 43776 /tmp/.pad.7094.1 (deleted) b1000000-b1016000 r-xp 00000000 03:06 27068 /lib/ld-2.3.2.so b1016000-b1017000 rw-p 00015000 03:06 27068 /lib/ld-2.3.2.so b1017000-b1018000 rw-p 00000000 00:00 0 b1018000-b1140000 r-xp 00000000 03:07 675833 /mnt/redhat/debian-usr/usr/lib/debug/libc-2.3.2.so b1140000-b1148000 rw-p 00127000 03:07 675833 /mnt/redhat/debian-usr/usr/lib/debug/libc-2.3.2.so b1148000-b124b000 rw-p 00000000 00:00 0 b124b000-b1264000 r-xp 00000000 03:07 674393 /mnt/redhat/debian-usr/usr/lib/valgrind/vgskin_memcheck.so b1264000-b1265000 rw-p 00018000 03:07 674393 /mnt/redhat/debian-usr/usr/lib/valgrind/vgskin_memcheck.so b1265000-b1377000 rw-p 00000000 00:00 0 bfffe000-c0000000 rwxp fffff000 00:00 0 |
|
From: Ashley P. <as...@qu...> - 2004-07-27 15:29:41
|
On Tue, 2004-07-27 at 15:52, Tom Hughes wrote:
> I have just remembered something else however, which is that certain
> older kernels didn't seem to like those very large mmap calls. What
> distribution are you using, and what kernel?
I'm running a fully up-to-date debian unstable but a elderly RedHat
kernel, I took the (unwise) precaution of not installing grub or a cd
rom drive when I installed my desktop so had to bootstrap off a Redhat
rescue floppy some time ago. I'm dreading the next power cut :(
/proc/version says this: A bit old perhaps but should still be ok...
Linux version 2.4.18-5 (bhc...@st...) (gcc
version 2.96 20000731 (Red Hat Linux 7.3 2.96-110)) #1 Mon Jun 10
15:38:12 EDT 2002
for completeness:
ashley:ashley> cat /proc/meminfo
total: used: free: shared: buffers: cached:
Mem: 260972544 253939712 7032832 0 6410240 99799040
Swap: 268419072 198647808 69771264
MemTotal: 254856 kB
MemFree: 6868 kB
MemShared: 0 kB
Buffers: 6260 kB
Cached: 91792 kB
SwapCached: 5668 kB
Active: 138568 kB
Inact_dirty: 83948 kB
Inact_clean: 4804 kB
Inact_target: 45464 kB
HighTotal: 0 kB
HighFree: 0 kB
LowTotal: 254856 kB
LowFree: 6868 kB
SwapTotal: 262128 kB
SwapFree: 68136 kB
Committed_AS: 512292 kB
Ashley,
|
|
From: Tom H. <th...@cy...> - 2004-07-27 15:34:28
|
In message <1090941474.5891.19.camel@ashley>
Ashley Pittman <as...@qu...> wrote:
> On Tue, 2004-07-27 at 15:52, Tom Hughes wrote:
>
>> I have just remembered something else however, which is that certain
>> older kernels didn't seem to like those very large mmap calls. What
>> distribution are you using, and what kernel?
>
> I'm running a fully up-to-date debian unstable but a elderly RedHat
> kernel, I took the (unwise) precaution of not installing grub or a cd
> rom drive when I installed my desktop so had to bootstrap off a Redhat
> rescue floppy some time ago. I'm dreading the next power cut :(
>
> /proc/version says this: A bit old perhaps but should still be ok...
> Linux version 2.4.18-5 (bhc...@st...) (gcc
> version 2.96 20000731 (Red Hat Linux 7.3 2.96-110)) #1 Mon Jun 10
> 15:38:12 EDT 2002
I suspect this is your problem as I have seen this problem on
various RedHat 7.2 and 7.3 machines before and updating to the
latest kernel from the updates site seemed to fix it.
In fact as I recall even rebooting tended to fix it, at least
for a while.
Tom
--
Tom Hughes (th...@cy...)
Software Engineer, Cyberscience Corporation
http://www.cyberscience.com/
|
|
From: Ashley P. <as...@qu...> - 2004-08-05 15:45:03
|
On Tue, 2004-07-27 at 16:34, Tom Hughes wrote: > I suspect this is your problem as I have seen this problem on > various RedHat 7.2 and 7.3 machines before and updating to the > latest kernel from the updates site seemed to fix it. I've rebooted into the official debian kernel, version 2.4.26-1-386 and all is fine now. Thanks for your time. Ashley, |
|
From: Nicholas N. <nj...@ca...> - 2004-07-28 08:26:39
|
On Tue, 27 Jul 2004, Ashley Pittman wrote: > ashley:ashley> cat /proc/7094/maps > 52c00000-52d00000 ---p 00000000 00:00 0 > 52d00000-b0000000 ---p 4ac0c000 03:06 43776 /tmp/.pad.7094.1 (deleted) > b0000000-b009c000 r-xp 00000000 03:07 674313 /mnt/redhat/debian-usr/usr/lib/valgrind/stage2 > b009c000-b009d000 rw-p 0009b000 03:07 674313 /mnt/redhat/debian-usr/usr/lib/valgrind/stage2 > b009d000-b01f7000 rw-p 00000000 00:00 0 > b01f7000-b1000000 ---p 00000000 03:06 43776 /tmp/.pad.7094.1 (deleted) > b1000000-b1016000 r-xp 00000000 03:06 27068 /lib/ld-2.3.2.so > b1016000-b1017000 rw-p 00015000 03:06 27068 /lib/ld-2.3.2.so > b1017000-b1018000 rw-p 00000000 00:00 0 > b1018000-b1140000 r-xp 00000000 03:07 675833 /mnt/redhat/debian-usr/usr/lib/debug/libc-2.3.2.so > b1140000-b1148000 rw-p 00127000 03:07 675833 /mnt/redhat/debian-usr/usr/lib/debug/libc-2.3.2.so > b1148000-b124b000 rw-p 00000000 00:00 0 > b124b000-b1264000 r-xp 00000000 03:07 674393 /mnt/redhat/debian-usr/usr/lib/valgrind/vgskin_memcheck.so > b1264000-b1265000 rw-p 00018000 03:07 674393 /mnt/redhat/debian-usr/usr/lib/valgrind/vgskin_memcheck.so > b1265000-b1377000 rw-p 00000000 00:00 0 > bfffe000-c0000000 rwxp fffff000 00:00 0 That looks as expected too. Maybe it's the too-big-mmap problem Tom mentioned. You could try breaking the big mmap() on line 550 up into several smaller mmap()s? Sorry I can't be more helpful. N |
|
From: Ashley P. <as...@qu...> - 2004-07-28 08:46:46
|
On Wed, 2004-07-28 at 09:26, Nicholas Nethercote wrote: > That looks as expected too. Maybe it's the too-big-mmap problem Tom > mentioned. You could try breaking the big mmap() on line 550 up into > several smaller mmap()s? Sorry I can't be more helpful. I'm working on installing a new kernel now but it's going to have to wait until later in the week as this is my everyday desktop machine. Ashley, |