|
From: Pierre H. <mad...@de...> - 2011-05-12 11:57:50
|
Hi, in the packaging readme[0] for valgrind it's stated that it's a bad
idea to strip what's in /usr/lib/valgrind/ because it makes bad reports.
Though if I test, here is what I get:
unstripped:
┌─(13:43)────
└[apollon] valgrind ./a
==20112== Memcheck, a memory error detector
==20112== Copyright (C) 2002-2010, and GNU GPL'd, by Julian Seward et al.
==20112== Using Valgrind-3.6.1 and LibVEX; rerun with -h for copyright info
==20112== Command: ./a
==20112==
==20112== Invalid free() / delete / delete[]
==20112== at 0x4C268FE: free (vg_replace_malloc.c:366)
==20112== by 0x4004F1: main (in /home/madcoder/a)
==20112== Address 0x14 is not stack'd, malloc'd or (recently) free'd
==20112==
==20112==
==20112== HEAP SUMMARY:
==20112== in use at exit: 0 bytes in 0 blocks
==20112== total heap usage: 0 allocs, 1 frees, 0 bytes allocated
==20112==
==20112== All heap blocks were freed -- no leaks are possible
==20112==
==20112== For counts of detected and suppressed errors, rerun with: -v
==20112== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 4 from 4)
valgrind ./a 2.82s user 0.10s system 98% cpu 2.969 total
stripped:
┌─(13:43)────
└[apollon] valgrind ./a
==20117== Memcheck, a memory error detector
==20117== Copyright (C) 2002-2010, and GNU GPL'd, by Julian Seward et al.
==20117== Using Valgrind-3.6.1 and LibVEX; rerun with -h for copyright info
==20117== Command: ./a
==20117==
==20117== Invalid free() / delete / delete[]
==20117== at 0x4C268FE: free (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==20117== by 0x4004F1: main (in /home/madcoder/a)
==20117== Address 0x14 is not stack'd, malloc'd or (recently) free'd
==20117==
==20117==
==20117== HEAP SUMMARY:
==20117== in use at exit: 0 bytes in 0 blocks
==20117== total heap usage: 0 allocs, 1 frees, 0 bytes allocated
==20117==
==20117== All heap blocks were freed -- no leaks are possible
==20117==
==20117== For counts of detected and suppressed errors, rerun with: -v
==20117== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 4 from 4)
valgrind ./a 2.08s user 0.08s system 99% cpu 2.165 total
IOW I'm unable to reproduce the sample on [0] since the "free" symbol is still
present. Sure I do lack the line and file in vg_replace_malloc.c but who cares?
FWIW this is a matter of 227Mb large /usr/lib/valgrind versus 71Mb without
debug. Which is kind of huge for a doubtful benefit. So is there something I
miss or does the restriction could be lifted?
Secondly, valgrinds installs a few archives namely:
/usr/lib/valgrind/libcoregrind-amd64-linux.a
/usr/lib/valgrind/libcoregrind-x86-linux.a
/usr/lib/valgrind/libreplacemalloc_toolpreload-amd64-linux.a
/usr/lib/valgrind/libreplacemalloc_toolpreload-x86-linux.a
/usr/lib/valgrind/libvex-amd64-linux.a
/usr/lib/valgrind/libvex-x86-linux.a
What are those used for, are those useful for anything else but valgrind
tools development ? IOW does it makes sense to provide them by default ?
TIA.
[0] http://valgrind.org/docs/manual/dist.readme-packagers.html
--
·O· Pierre Habouzit
··O mad...@de...
OOO http://www.madism.org
|
|
From: John R. <jr...@bi...> - 2011-05-12 14:58:03
|
> Hi, in the packaging readme[0] for valgrind it's stated that it's a bad > idea to strip what's in /usr/lib/valgrind/ ... > FWIW this is a matter of 227Mb large /usr/lib/valgrind versus 71Mb without > debug. Which machine architecture and compiler? There's a difference between stripped (applying /usr/bin/strip, /usr/bin/objcopy, flags such as -s -S to /usr/bin/ld, etc.) and non-debug (no -g in CFLAGS). Only developers of valgrind need debug symbols (-g) inside valgrind itself. After installing a recent Fedora 15 .rpm for valgrind-3.6.1: ----- $ ls -l /usr/bin/valgrind /usr/lib64/valgrind/memcheck-amd64-linux -rwxr-xr-x. 1 root root 43260 May 3 02:49 /usr/bin/valgrind -rwxr-xr-x. 1 root root 13742515 May 3 02:49 /usr/lib64/valgrind/memcheck-amd64-linux $ file /usr/bin/valgrind /usr/lib64/valgrind/memcheck-amd64-linux /usr/bin/valgrind: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.32, not stripped /usr/lib64/valgrind/memcheck-amd64-linux: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, not stripped ----- > Secondly, valgrinds installs a few archives namely: > /usr/lib/valgrind/libcoregrind-amd64-linux.a > /usr/lib/valgrind/libcoregrind-x86-linux.a > /usr/lib/valgrind/libreplacemalloc_toolpreload-amd64-linux.a > /usr/lib/valgrind/libreplacemalloc_toolpreload-x86-linux.a > /usr/lib/valgrind/libvex-amd64-linux.a > /usr/lib/valgrind/libvex-x86-linux.a These are needed only for development of valgrind tools. They are not needed for ordinary invocation of valgrind tools. -- |
|
From: Pierre H. <mad...@de...> - 2011-05-12 15:09:29
|
On Thu, May 12, 2011 at 07:58:14AM -0700, John Reiser wrote:
> > Hi, in the packaging readme[0] for valgrind it's stated that it's a bad
> > idea to strip what's in /usr/lib/valgrind/ ...
>
> > FWIW this is a matter of 227Mb large /usr/lib/valgrind versus 71Mb without
> > debug.
>
> Which machine architecture and compiler?
amd64 gcc, but indeed with -g, because it's how binaries are supposed to
be built in Debian.
> There's a difference between stripped (applying /usr/bin/strip, /usr/bin/objcopy,
> flags such as -s -S to /usr/bin/ld, etc.) and non-debug (no -g in CFLAGS).
> Only developers of valgrind need debug symbols (-g) inside valgrind itself.
I know about the difference, but stupidly thought that valgrind needed
the latter (debug symbols).
By default, after build, debian strip:
- libraries with:
strip --remove-section=.comment --remove-section=.note --strip-unnedded
- exectutables with:
strip --remove-section=.comment --remove-section=.note
Would this be okay to do that? (I think so since my earlier test proves
it works).
> After installing a recent Fedora 15 .rpm for valgrind-3.6.1:
> -----
> $ ls -l /usr/bin/valgrind /usr/lib64/valgrind/memcheck-amd64-linux
> -rwxr-xr-x. 1 root root 43260 May 3 02:49 /usr/bin/valgrind
> -rwxr-xr-x. 1 root root 13742515 May 3 02:49 /usr/lib64/valgrind/memcheck-amd64-linux
I'm in the same numbers, which means that fedora does the same as Debian
(and distros usually do that btw), and build in -g and do not strip.
total 232036
drwxr-xr-x 2 root root 4096 12 mai 13:49 .
drwxr-xr-x 231 root root 90112 10 mai 23:02 ..
-rwxr-xr-x 1 root root 11694337 12 mai 13:37 cachegrind-amd64-linux
-rwxr-xr-x 1 root root 6349469 12 mai 13:37 cachegrind-x86-linux
-rwxr-xr-x 1 root root 12049332 12 mai 13:37 callgrind-amd64-linux
-rwxr-xr-x 1 root root 6660945 12 mai 13:37 callgrind-x86-linux
-rwxr-xr-x 1 root root 12104936 12 mai 13:37 drd-amd64-linux
-rwxr-xr-x 1 root root 6662606 12 mai 13:37 drd-x86-linux
-rwxr-xr-x 1 root root 11532388 12 mai 13:37 exp-bbv-amd64-linux
-rwxr-xr-x 1 root root 6231331 12 mai 13:37 exp-bbv-x86-linux
-rwxr-xr-x 1 root root 11563895 12 mai 13:37 exp-dhat-amd64-linux
-rwxr-xr-x 1 root root 6255989 12 mai 13:37 exp-dhat-x86-linux
-rwxr-xr-x 1 root root 11844280 12 mai 13:37 exp-ptrcheck-amd64-linux
-rwxr-xr-x 1 root root 6469495 12 mai 13:37 exp-ptrcheck-x86-linux
-rwxr-xr-x 1 root root 12001111 12 mai 13:37 helgrind-amd64-linux
-rwxr-xr-x 1 root root 6603351 12 mai 13:37 helgrind-x86-linux
-rwxr-xr-x 1 root root 11542032 12 mai 13:37 lackey-amd64-linux
-rwxr-xr-x 1 root root 6243031 12 mai 13:37 lackey-x86-linux
-rwxr-xr-x 1 root root 11616620 12 mai 13:37 massif-amd64-linux
-rwxr-xr-x 1 root root 6296640 12 mai 13:37 massif-x86-linux
-rwxr-xr-x 1 root root 12585174 12 mai 13:37 memcheck-amd64-linux
-rwxr-xr-x 1 root root 6911323 12 mai 13:37 memcheck-x86-linux
-rwxr-xr-x 1 root root 11517333 12 mai 13:37 none-amd64-linux
-rwxr-xr-x 1 root root 6218296 12 mai 13:37 none-x86-linux
-rw-r--r-- 1 root root 8736 12 mai 13:37 vgpreload_core-amd64-linux.so
-rw-r--r-- 1 root root 6611 12 mai 13:37 vgpreload_core-x86-linux.so
-rw-r--r-- 1 root root 246346 12 mai 13:37 vgpreload_drd-amd64-linux.so
-rw-r--r-- 1 root root 171160 12 mai 13:37 vgpreload_drd-x86-linux.so
-rw-r--r-- 1 root root 53878 12 mai 13:37 vgpreload_exp-dhat-amd64-linux.so
-rw-r--r-- 1 root root 41312 12 mai 13:37 vgpreload_exp-dhat-x86-linux.so
-rw-r--r-- 1 root root 81584 12 mai 13:37 vgpreload_exp-ptrcheck-amd64-linux.so
-rw-r--r-- 1 root root 62454 12 mai 13:37 vgpreload_exp-ptrcheck-x86-linux.so
-rw-r--r-- 1 root root 157372 12 mai 13:37 vgpreload_helgrind-amd64-linux.so
-rw-r--r-- 1 root root 119497 12 mai 13:37 vgpreload_helgrind-x86-linux.so
-rw-r--r-- 1 root root 53878 12 mai 13:37 vgpreload_massif-amd64-linux.so
-rw-r--r-- 1 root root 41312 12 mai 13:37 vgpreload_massif-x86-linux.so
-rw-r--r-- 1 root root 141577 12 mai 13:37 vgpreload_memcheck-amd64-linux.so
-rw-r--r-- 1 root root 105811 12 mai 13:37 vgpreload_memcheck-x86-linux.so
If you strip --strip-debug (which basically should be the same as not
building with -g if I'm correct) then memcheck-amd64-linux here is:
-rwxr-xr-x 1 madcoder madcoder 4751991 12 mai 17:06 /home/madcoder/memcheck-amd64-linux
which is basically 4M instead of 11.5… du your /usr/lib/valgrind you'll see it's huge!
> > Secondly, valgrinds installs a few archives namely:
> > /usr/lib/valgrind/libcoregrind-amd64-linux.a
> > /usr/lib/valgrind/libcoregrind-x86-linux.a
> > /usr/lib/valgrind/libreplacemalloc_toolpreload-amd64-linux.a
> > /usr/lib/valgrind/libreplacemalloc_toolpreload-x86-linux.a
> > /usr/lib/valgrind/libvex-amd64-linux.a
> > /usr/lib/valgrind/libvex-x86-linux.a
>
> These are needed only for development of valgrind tools. They are
> not needed for ordinary invocation of valgrind tools.
I assumed so, thanks.
--
·O· Pierre Habouzit
··O mad...@de...
OOO http://www.madism.org
|
|
From: John R. <jr...@bi...> - 2011-05-12 15:55:21
|
>>> FWIW this is a matter of 227Mb large /usr/lib/valgrind versus 71Mb without >>> debug. >> >> Which machine architecture and compiler? > > amd64 gcc, but indeed with -g, because it's how binaries are supposed to > be built in Debian. For many [most?] packages, Fedora handles this by building _with_ -g in CFLAGS, saving everything in a <package>-debuginfo-<version>.rpm, then [effectively] applying "strip --strip-debug" before constructing <package>-<version>.rpm. Ordinary installs get the smaller pieces without debuginfo, but the corresponding full -g is available for developers to use when handling bug reports. [However, in this specific case I cannot find a valgrind-debuginfo package.] > By default, after build, debian strip: > > - libraries with: > strip --remove-section=.comment --remove-section=.note --strip-unnedded > - exectutables with: > strip --remove-section=.comment --remove-section=.note > > Would this be okay to do that? (I think so since my earlier test proves > it works). The earlier test checks only the 'free' routine, so applying --strip-unneeded would rely on all "interesting" routines having similar behavior. > If you strip --strip-debug (which basically should be the same as not > building with -g if I'm correct) then memcheck-amd64-linux here is: > > -rwxr-xr-x 1 madcoder madcoder 4751991 12 mai 17:06 /home/madcoder/memcheck-amd64-linux > > which is basically 4M instead of 11.5… du your /usr/lib/valgrind you'll see it's huge! I see around 12MB per <tool>-amd64-linux, and about 5MB or less after "strip --strip-debug". If that much is too big, then consider removing infrequently-used tools, or mounting a compressed filesystem on /usr/lib/valgrind, or perhaps compressing individual executables with a compressor such as upx (http://upx.sourceforge.net). upx does compress them to about 2MB to 2.6MB each, and they do invoke; but valgrind and its tools might make assumptions about layout of address space that are more demanding than a usual executable program. -- |
|
From: Pierre H. <mad...@de...> - 2011-05-12 18:00:00
|
On Thu, May 12, 2011 at 08:55:32AM -0700, John Reiser wrote:
> >>> FWIW this is a matter of 227Mb large /usr/lib/valgrind versus 71Mb without
> >>> debug.
> >>
> >> Which machine architecture and compiler?
> >
> > amd64 gcc, but indeed with -g, because it's how binaries are supposed to
> > be built in Debian.
>
> For many [most?] packages, Fedora handles this by building _with_ -g in CFLAGS,
> saving everything in a <package>-debuginfo-<version>.rpm, then [effectively]
> applying "strip --strip-debug" before constructing <package>-<version>.rpm.
> Ordinary installs get the smaller pieces without debuginfo, but the corresponding
> full -g is available for developers to use when handling bug reports.
> [However, in this specific case I cannot find a valgrind-debuginfo package.]
Debian does the same, mostly. And I plan to do the same. I'm just unsure
whether doing this will leave valgrind in a usable state or not. I think it
will I just wanted some confirmation.
> > By default, after build, debian strip:
> >
> > - libraries with:
> > strip --remove-section=.comment --remove-section=.note --strip-unnedded
> > - exectutables with:
> > strip --remove-section=.comment --remove-section=.note
> >
> > Would this be okay to do that? (I think so since my earlier test proves
> > it works).
>
> The earlier test checks only the 'free' routine, so applying --strip-unneeded
> would rely on all "interesting" routines having similar behavior.
Sure, though I fail to see why it would be any different for the other
ones. the vgpreload .so files define a bunch of symbols:
0000000000003890 T _vgrZU_libcZdsoZa_free
0000000000004c00 T _vgrZU_libcZdsoZa_index
0000000000002980 T _vgrZU_libcZdsoZa_mallinfo
0000000000004730 T _vgrZU_libcZdsoZa_malloc
0000000000002950 T _vgrZU_libcZdsoZa_malloc_get_state
0000000000002960 T _vgrZU_libcZdsoZa_malloc_set_state
0000000000002a60 T _vgrZU_libcZdsoZa_malloc_size
0000000000002970 T _vgrZU_libcZdsoZa_malloc_stats
0000000000002930 T _vgrZU_libcZdsoZa_malloc_trim
0000000000002b40 T _vgrZU_libcZdsoZa_malloc_usable_size
0000000000002920 T _vgrZU_libcZdsoZa_mallopt
and so forth, and obviously what will produce a proper stack is a way to find
out about those symbols which in my knowledge should still work with the way
Debian strips binaries.
I think I'll hence try to "mildly" strip the binaries and see where that
goes. With a warning to the users so that they know about what is
failing if something begins to fail miserably after that change.
> > If you strip --strip-debug (which basically should be the same as not
> > building with -g if I'm correct) then memcheck-amd64-linux here is:
> >
> > -rwxr-xr-x 1 madcoder madcoder 4751991 12 mai 17:06 /home/madcoder/memcheck-amd64-linux
> >
> > which is basically 4M instead of 11.5… du your /usr/lib/valgrind you'll see it's huge!
>
> I see around 12MB per <tool>-amd64-linux, and about 5MB or less after "strip --strip-debug".
> If that much is too big, then consider removing infrequently-used tools,
> or mounting a compressed filesystem on /usr/lib/valgrind, or perhaps compressing
> individual executables with a compressor such as upx (http://upx.sourceforge.net).
> upx does compress them to about 2MB to 2.6MB each, and they do invoke; but valgrind
> and its tools might make assumptions about layout of address space that are more
> demanding than a usual executable program.
Please, it's too big because it's a BW waste for the mirrors and so
forth, plus it uses 200Mb where 60 are enough, yeah that's a waste,
that's with those kind of reasonings that you end up with needing 20G
for a decent system nowadays :/
--
·O· Pierre Habouzit
··O mad...@de...
OOO http://www.madism.org
|