|
From: Madhan S. <mad...@gm...> - 2007-04-10 15:24:03
|
My question is pertaining to 2.4.1 Valgrind addrcheck,
but its pretty much a general question.
My application executable is around 1.73 G in size after
rebuilding most of the code with debug information (dwarf2) using
Intel 9 compiler.
When starting this application with valgrind (addrcheck) I see a mmap failure
when valgrind tries to read symbols from the binary.
Looking at the objdump -x output for the sections in the application
27 .debug_info 5256cc6e 00000000 00000000 0affad39 2**0
CONTENTS, READONLY, DEBUGGING
28 .debug_abbrev 0038ff18 00000000 00000000 5d5679a7 2**0
CONTENTS, READONLY, DEBUGGING
29 .debug_line 021f7453 00000000 00000000 5d8f78bf 2**0
CONTENTS, READONLY, DEBUGGING
30 .debug_frame 003f9030 00000000 00000000 5faeed12 2**0
CONTENTS, READONLY, DEBUGGING
It looks like ".debug_info" takes up most of the space in the binary (1.2 G)
and valgrind does not seem to be requiring it. I could only find
".debug_line" being searched for in valgrind code.
Using
objcopy -R .debug_info <org> <new>
I am removing the ".debug_info" section and the resulting binary
is only 236 M. This is comfortably handled by Valgrind.
I see line numbers in the error stack traces of addrcheck.
Is it ok to rip off the ".debug_info" section and will this cause
valgrind's checking or information in log files to degrade.
I have been advised long back to move to the latest version of
valgrind. But with such huge executable it looks like the problem
will be found there too.
Thanks,
Madhan.
|
|
From: Nicholas N. <nj...@cs...> - 2007-04-10 22:07:25
|
On Tue, 10 Apr 2007, Madhan Sadasivam wrote: > My question is pertaining to 2.4.1 Valgrind addrcheck, > but its pretty much a general question. > [...] > I have been advised long back to move to the latest version of > valgrind. But with such huge executable it looks like the problem > will be found there too. Why do you assume that? Recent versions have much better memory handling. Also, post-3.2.0 Memcheck's memory usage is roughly 1/4 that of pre-3.2.0. Nick |
|
From: Madhan S. <mad...@gm...> - 2007-04-11 10:20:20
|
Hello Nicholas,
The problem is present in 3.2.3 too. The problem is present in 3.2.3
even after removing the ".debug_info" section. This happens a lot
earlier before the tools start using any memory.
--------------------------------
On 3.2.3 memcheck - big executable (size 1862236727)
--------------------------------
On the shell I see the following:
valgrind: mmap(0x38000000, 1253376) failed in UME with error 22
(Invalid argument).
valgrind: this can be caused by executables with very large text, data
or bss segments.
Valgrind log file:
==28646== Memcheck, a memory error detector.
==28646== Copyright (C) 2002-2007, and GNU GPL'd, by Julian Seward et al.
==28646== Using LibVEX rev 1732, a library for dynamic binary translation.
==28646== Copyright (C) 2004-2007, and GNU GPL'd, by OpenWorks LLP.
==28646== Using valgrind-3.2.3, a dynamic binary instrumentation framework.
==28646== Copyright (C) 2000-2007, and GNU GPL'd, by Julian Seward et al.
[...]
--28646-- Contents of /proc/version:
--28646-- Linux version 2.6.9-34.0.1.0.11.ELsmp (bbarbe@ca-build1) (gcc ver
sion 3.4.5 20051201 (Red Hat 3.4.5-2)) #1 SMP Thu Oct 26 14:26:54 PDT 2006
--28646-- Arch and hwcaps: X86, x86-sse1-sse2
--28646-- Page sizes: currently 4096, max supported 4096
--28646-- Reading syms from /lib/ld-2.3.4.so (0x532000)
--28646-- Reading syms from /scratch/local/bin/app (0x8048000)
==28646== warning: mmap failed on /scratch/local/bin/app
==28646== no symbols or debug info loaded
--------------------------------
On 3.2.3 memcheck - executable without ".debug_info" (size 236327131)
--------------------------------
valgrind: mmap(0x38000000, 1253376) failed in UME with error 22
(Invalid argument).
valgrind: this can be caused by executables with very large text, data
or bss segments.
--------------------------------
On 2.4.1 addrcheck
--------------------------------
==27153== Addrcheck, a fine-grained address checker for x86-linux.
==27153== Copyright (C) 2002-2005, and GNU GPL'd, by Julian Seward et al.
==27153== Using valgrind-2.4.1, a program supervision framework for x86-linux.
==27153== Copyright (C) 2000-2005, and GNU GPL'd, by Julian Seward et al.
[...]
==27153==
==27153== Contents of /proc/version:
==27153== Linux version 2.6.9-34.0.1.0.11.ELsmp (bbarbe@ca-build1)
(gcc version 3.4.5 20051201 (Red Hat 3.4.5-2)) #1 SMP Thu Oct 26
14:26:54 PDT 2006
==27153== Reading syms from /scratch/local/bin/app (0x8048000)
==27153== warning: mmap failed on /scratch/local/bin/app
==27153== no symbols or debug info loaded
==27153== Warning: set address range perms: large range 179920896, a 0
--------------------------------
objdump -x app (partial) - from the big executable
--------------------------------
11 .text 0976da54 08483e70 08483e70 0043be70 2**4
CONTENTS, ALLOC, LOAD, READONLY, CODE
12 .text1 00000000 11bf18c4 11bf18c4 09ba98c4 2**2
CONTENTS, ALLOC, LOAD, READONLY, CODE
23 .data 0003f040 12be2280 12be2280 0ab99280 2**5
CONTENTS, ALLOC, LOAD, DATA
24 .data1 00064a80 12c212c0 12c212c0 0abd82c0 2**5
CONTENTS, ALLOC, LOAD, DATA
25 .bss 00026cb4 12c85d40 12c85d40 0ac3cd40 2**5
ALLOC
27 .debug_info 5256cc6e 00000000 00000000 0affadb9 2**0
CONTENTS, READONLY, DEBUGGING
28 .debug_abbrev 0038ff18 00000000 00000000 5d567a27 2**0
CONTENTS, READONLY, DEBUGGING
29 .debug_line 021f7453 00000000 00000000 5d8f793f 2**0
CONTENTS, READONLY, DEBUGGING
30 .debug_frame 003f9030 00000000 00000000 5faeed92 2**0
CONTENTS, READONLY, DEBUGGING
Thanks,
Madhan.
On 4/11/07, Nicholas Nethercote <nj...@cs...> wrote:
> On Tue, 10 Apr 2007, Madhan Sadasivam wrote:
>
> > My question is pertaining to 2.4.1 Valgrind addrcheck,
> > but its pretty much a general question.
> > [...]
> > I have been advised long back to move to the latest version of
> > valgrind. But with such huge executable it looks like the problem
> > will be found there too.
>
> Why do you assume that? Recent versions have much better memory handling.
> Also, post-3.2.0 Memcheck's memory usage is roughly 1/4 that of pre-3.2.0.
>
> Nick
>
|
|
From: Julian S. <js...@ac...> - 2007-04-11 10:25:09
|
> -------------------------------- > On 3.2.3 memcheck - big executable (size 1862236727) > -------------------------------- > On the shell I see the following: > > valgrind: mmap(0x38000000, 1253376) failed in UME with error 22 > (Invalid argument). (comment to V devs ..) Why is it trying to do mmap at 0x38000000 ? This is the load address for memcheck itself. Something funny about this. Madhan: does 3.2.3 work for other executables on the machine (for example, ls, date, etc)? J |
|
From: Madhan S. <mad...@gm...> - 2007-04-11 12:18:08
|
Julian, date, ls are working perfectly fine. $ valgrind /bin/date ==25147== Memcheck, a memory error detector. ==25147== Copyright (C) 2002-2007, and GNU GPL'd, by Julian Seward et al. ==25147== Using LibVEX rev 1732, a library for dynamic binary translation. ==25147== Copyright (C) 2004-2007, and GNU GPL'd, by OpenWorks LLP. ==25147== Using valgrind-3.2.3, a dynamic binary instrumentation framework. ==25147== Copyright (C) 2000-2007, and GNU GPL'd, by Julian Seward et al. ==25147== For more details, rerun with: -v ==25147== Wed Apr 11 05:11:40 PDT 2007 ==25147== ==25147== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 16 from 1) ==25147== malloc/free: in use at exit: 0 bytes in 0 blocks. ==25147== malloc/free: 72 allocs, 72 frees, 8,809 bytes allocated. ==25147== For counts of detected errors, rerun with: -v ==25147== All heap blocks were freed -- no leaks are possible. Thanks, Madhan. On 4/11/07, Julian Seward <js...@ac...> wrote: > > > -------------------------------- > > On 3.2.3 memcheck - big executable (size 1862236727) > > -------------------------------- > > On the shell I see the following: > > > > valgrind: mmap(0x38000000, 1253376) failed in UME with error 22 > > (Invalid argument). > > (comment to V devs ..) > Why is it trying to do mmap at 0x38000000 ? This is the load address > for memcheck itself. Something funny about this. > > Madhan: does 3.2.3 work for other executables on the machine > (for example, ls, date, etc)? > > J > |
|
From: Julian S. <js...@ac...> - 2007-04-11 16:06:09
|
> On Wednesday 11 April 2007 13:18, Madhan Sadasivam wrote: > > date, ls are working perfectly fine. Strange. Can you send the results of valgrind -v -v -d -d <big_exe_which_fails> Also, try running your big exe with "--tool=none". Does that make any difference? J |
|
From: Julian S. <js...@ac...> - 2007-04-11 20:04:10
|
> Attached is the output of the run with "-v -v -d -d" > using both memcheck and "none" tool. > "none" behaviour is the same. So in fact Valgrind does successfully start your app. However, the app then tries to exec the valgrind tool binary itself, which is why it fails: --13062:1:syswrap Exec of /net/stacj16/scratch/msadasiv/sware/2.4.1/test/bin/valgrind valgrind: mmap(0x38000000, 1253376) failed in UME with error 22 (Invalid argument). So you're looking for an exec which is using an executable whose name has been created in some strange way (by looking in /proc, maybe ?) Does that make any sense? J |
|
From: Madhan S. <mad...@gm...> - 2007-04-12 08:49:05
Attachments:
valg_memcheck.txt
|
Just because there was some error involving mmap I did not verify if I
was doing everything correct.
Here's what I do inside my main() function.
if(RUNNING_ON_VALGRIND){
}else{
exec( "valgrind", <binary>, <binary args>);
}
This allows my binary to be always started with Valgrind. If it finds
that it is not running on valgrind it will reexec itself with
Valgrind. Unfortunately I had the 2.4.1 version of valgrind.h included
in this c file. Hence all the confusions. Sorry for that.
I have included the 3.2.3 version of valgrind.h and here's the output
now. I am able to see the mmap failure while reading symbols, just
like the one I saw on 2.4.1. However with 3.2.3 when I remove the
".debug_info" section I am not seeing line numbers in the stack
traces.
I have attached the output thrown to the shell.
==21757== Memcheck, a memory error detector.
==21757== Copyright (C) 2002-2007, and GNU GPL'd, by Julian Seward et al.
==21757== Using LibVEX rev 1732, a library for dynamic binary translation.
==21757== Copyright (C) 2004-2007, and GNU GPL'd, by OpenWorks LLP.
==21757== Using valgrind-3.2.3, a dynamic binary instrumentation framework.
==21757== Copyright (C) 2000-2007, and GNU GPL'd, by Julian Seward et al.
==21757==
==21757== My PID = 21757, parent PID = 3965. Prog and args are:
==21757== /scratch/local/bin/app
==21757==
--21757--
--21757-- Command line
--21757-- /scratch/local/bin/app
--21757-- Startup, with flags:
--21757-- --log-file=/tmp/valg_app
--21757-- -v
--21757-- -v
--21757-- -d
--21757-- -d
--21757-- Contents of /proc/version:
--21757-- Linux version 2.6.9-34.0.1.0.11.ELsmp (bbarbe@ca-build1)
(gcc version 3.4.5 20051201 (Red Hat 3.4.5-2)) #1 SMP Thu Oct 26
14:26:54 PDT 2006
--21757-- Arch and hwcaps: X86, x86-sse1-sse2
--21757-- Page sizes: currently 4096, max supported 4096
--21757-- Valgrind library directory:
/net/stacj16/scratch/msadasiv/sware/2.4.1/test/lib/valgrind
--21757-- TT/TC: VG_(init_tt_tc) (startup of code management)
--21757-- TT/TC: cache: 8 sectors of 20127744 bytes each = 161021952 total
--21757-- TT/TC: table: 524168 total entries, max occupancy 419328 (80%)
--21757-- Reading syms from /lib/ld-2.3.4.so (0x532000)
--21757-- svma 0x0000532000, avma 0x0000532000
--21757-- Reading syms from /scratch/local/bin/app (0x8048000)
==21757== warning: mmap failed on /scratch/local/bin/app
==21757== no symbols or debug info loaded
--21757-- Reading syms from
/net/stacj16/scratch/msadasiv/sware/2.4.1/test/lib/valgrind/x86-linux/memcheck
(0x38000000)
--21757-- svma 0x0038000000, avma 0x0038000000
--21757-- object doesn't have a dynamic symbol table
==21757== Warning: set address range perms: large range 179920896 (defined)
Thanks,
Madhan.
On 4/12/07, Julian Seward <js...@ac...> wrote:
>
> > Attached is the output of the run with "-v -v -d -d"
> > using both memcheck and "none" tool.
> > "none" behaviour is the same.
>
> So in fact Valgrind does successfully start your app. However, the
> app then tries to exec the valgrind tool binary itself, which is why
> it fails:
>
> --13062:1:syswrap Exec of /net/stacj16/scratch/msadasiv/sware/2.4.1/test/bin/valgrind
> valgrind: mmap(0x38000000, 1253376) failed in UME with error 22 (Invalid argument).
>
> So you're looking for an exec which is using an executable whose name has been
> created in some strange way (by looking in /proc, maybe ?)
> Does that make any sense?
>
> J
>
|
|
From: Madhan S. <mad...@gm...> - 2007-04-18 05:00:16
|
Hope you noticed the following in my mail
> ==21757== warning: mmap failed on /scratch/local/bin/app
> ==21757== no symbols or debug info loaded
This results in stack traces without symbols and line numbers.
Hence I wanted to know if knocking off ".debug_info" section in the
binary is acceptable.
Please let me know if I am missing something.
Thanks,
Madhan.
On 4/12/07, Madhan Sadasivam <mad...@gm...> wrote:
> Just because there was some error involving mmap I did not verify if I
> was doing everything correct.
>
> Here's what I do inside my main() function.
>
> if(RUNNING_ON_VALGRIND){
> }else{
> exec( "valgrind", <binary>, <binary args>);
> }
>
> This allows my binary to be always started with Valgrind. If it finds
> that it is not running on valgrind it will reexec itself with
> Valgrind. Unfortunately I had the 2.4.1 version of valgrind.h included
> in this c file. Hence all the confusions. Sorry for that.
>
> I have included the 3.2.3 version of valgrind.h and here's the output
> now. I am able to see the mmap failure while reading symbols, just
> like the one I saw on 2.4.1. However with 3.2.3 when I remove the
> ".debug_info" section I am not seeing line numbers in the stack
> traces.
>
> I have attached the output thrown to the shell.
>
>
> ==21757== Memcheck, a memory error detector.
> ==21757== Copyright (C) 2002-2007, and GNU GPL'd, by Julian Seward et al.
> ==21757== Using LibVEX rev 1732, a library for dynamic binary translation.
> ==21757== Copyright (C) 2004-2007, and GNU GPL'd, by OpenWorks LLP.
> ==21757== Using valgrind-3.2.3, a dynamic binary instrumentation framework.
> ==21757== Copyright (C) 2000-2007, and GNU GPL'd, by Julian Seward et al.
> ==21757==
> ==21757== My PID = 21757, parent PID = 3965. Prog and args are:
> ==21757== /scratch/local/bin/app
> ==21757==
> --21757--
> --21757-- Command line
> --21757-- /scratch/local/bin/app
> --21757-- Startup, with flags:
> --21757-- --log-file=/tmp/valg_app
> --21757-- -v
> --21757-- -v
> --21757-- -d
> --21757-- -d
> --21757-- Contents of /proc/version:
> --21757-- Linux version 2.6.9-34.0.1.0.11.ELsmp (bbarbe@ca-build1)
> (gcc version 3.4.5 20051201 (Red Hat 3.4.5-2)) #1 SMP Thu Oct 26
> 14:26:54 PDT 2006
> --21757-- Arch and hwcaps: X86, x86-sse1-sse2
> --21757-- Page sizes: currently 4096, max supported 4096
> --21757-- Valgrind library directory:
> /net/stacj16/scratch/msadasiv/sware/2.4.1/test/lib/valgrind
> --21757-- TT/TC: VG_(init_tt_tc) (startup of code management)
> --21757-- TT/TC: cache: 8 sectors of 20127744 bytes each = 161021952 total
> --21757-- TT/TC: table: 524168 total entries, max occupancy 419328 (80%)
> --21757-- Reading syms from /lib/ld-2.3.4.so (0x532000)
> --21757-- svma 0x0000532000, avma 0x0000532000
> --21757-- Reading syms from /scratch/local/bin/app (0x8048000)
> ==21757== warning: mmap failed on /scratch/local/bin/app
> ==21757== no symbols or debug info loaded
> --21757-- Reading syms from
> /net/stacj16/scratch/msadasiv/sware/2.4.1/test/lib/valgrind/x86-linux/memcheck
> (0x38000000)
> --21757-- svma 0x0038000000, avma 0x0038000000
> --21757-- object doesn't have a dynamic symbol table
> ==21757== Warning: set address range perms: large range 179920896 (defined)
>
>
>
> Thanks,
> Madhan.
>
>
>
> On 4/12/07, Julian Seward <js...@ac...> wrote:
> >
> > > Attached is the output of the run with "-v -v -d -d"
> > > using both memcheck and "none" tool.
> > > "none" behaviour is the same.
> >
> > So in fact Valgrind does successfully start your app. However, the
> > app then tries to exec the valgrind tool binary itself, which is why
> > it fails:
> >
> > --13062:1:syswrap Exec of /net/stacj16/scratch/msadasiv/sware/2.4.1/test/bin/valgrind
> > valgrind: mmap(0x38000000, 1253376) failed in UME with error 22 (Invalid argument).
> >
> > So you're looking for an exec which is using an executable whose name has been
> > created in some strange way (by looking in /proc, maybe ?)
> > Does that make any sense?
> >
> > J
> >
>
>
|