valgrind-users — General discussion list for Valgrind users

Re: [Valgrind-users] Valgrind doesn't detect leaks on Linux/ARM with uClibc

[Julian S. <js...@ac...>]

On 05/21/2014 05:24 PM, Andrejs Bogdanovs wrote:
> Hi Julian!
> 
> Soname for malloc intercepts should be one defined in VG_Z_LIBC_SONAME?
> include/pub_tool_redir.h:244:#  define  VG_Z_LIBC_SONAME  libcZdsoZa
>            // libc.so*
> And soname for libuClibc-0.9.33.2.so is libc.so.0:
> [andrejs@arch ~]$ readelf -a ./dev_libs/lib/libuClibc-0.9.33.2.so | grep soname
>  0x0000000e (SONAME)                     Library soname: [libc.so.0]

That soname looks ok, then.  But from the log you posted
later, it is clear that there is no malloc intercepting happening.

For malloc intercepting to work, the files vgpreload_core-arm-linux.so
and vgpreload_memcheck-amd64-linux.so need to be loaded into the process.

Comparing your log with a run with a glibc based system, I see this

  Reading syms from [..]/vgpreload_core-amd64-linux.so
  Reading syms from [..]/vgpreload_memcheck-amd64-linux.so

which is missing in your log.

Does uCLibc implement LD_PRELOAD?  I think that is how Valgrind causes
those two objects to be loaded into the process.

J

Re: [Valgrind-users] Valgrind doesn't detect leaks on Linux/ARM with uClibc

[Andrejs B. <sin...@gm...>]

On 5/21/14, Philippe Waroquiers wrote:
>
> If Valgrind does not preload the above, then the redirections will not
> happen.
>
> Running
>    valgrind -v -v -v -d -d -d env
> might give some lights on what does not work
> In particular, what is the value of the LD_PRELOAD env variable ?
>
> Philippe
>

Hi Philippe!
I tried command 'valgrind -v -v -v -d -d -d env'
And got:
LD_PRELOAD=/usr/lib/valgrind/vgpreload_core-arm-linux.so:/usr/lib/valgrind/vgpreload_memcheck-arm-linux.so

Full log: http://pastebin.com/JmzjaRKp

Re: [Valgrind-users] Valgrind doesn't detect leaks on Linux/ARM with uClibc

[Philippe W. <phi...@sk...>]

On Wed, 2014-05-21 at 14:14 +0300, Andrejs Bogdanovs wrote:

> On Linux/x86-64 leak was detected.
> On Linux/ARM using --trace-redir=yes option I've found that
> vgpreload_core-arm-linux.so and vgpreload_memcheck-arm-linux.so files
> are not being read.
> Using strace on Valgrind I found that vgpreload_memcheck-arm-linux.so
> is accessed, but not used
> (access("/usr/lib/valgrind/vgpreload_memcheck-arm-linux.so", R_OK) = 0).

If Valgrind does not preload the above, then the redirections will not
happen.

Running 
   valgrind -v -v -v -d -d -d env
might give some lights on what does not work
In particular, what is the value of the LD_PRELOAD env variable ?

Philippe

Re: [Valgrind-users] Valgrind doesn't detect leaks on Linux/ARM with uClibc

[Andrejs B. <sin...@gm...>]

Hi Julian!

Soname for malloc intercepts should be one defined in VG_Z_LIBC_SONAME?
include/pub_tool_redir.h:244:#  define  VG_Z_LIBC_SONAME  libcZdsoZa
           // libc.so*
And soname for libuClibc-0.9.33.2.so is libc.so.0:
[andrejs@arch ~]$ readelf -a ./dev_libs/lib/libuClibc-0.9.33.2.so | grep soname
 0x0000000e (SONAME)                     Library soname: [libc.so.0]

Best regards,
Andrejs

On 5/21/14, Julian Seward wrote:
>
> This is most likely to be a soname problem, in that ucLibc's soname
> is different from the one used for the malloc/free intercepts.
>
> J
>
>

Re: [Valgrind-users] Valgrind doesn't detect leaks on Linux/ARM with uClibc

[Julian S. <js...@ac...>]

This is most likely to be a soname problem, in that ucLibc's soname
is different from the one used for the malloc/free intercepts.

J

[Valgrind-users] Valgrind doesn't detect leaks on Linux/ARM with uClibc

[Andrejs B. <sin...@gm...>]

Hi there! I'm quite new to Linux and was trying to use Valgrind for
finding leaks on 32-bit Linux ARMv7 (3.6.0, armv7l).
I crosscompiled Valgrind using non-stripped version of uClibc library.
I tried Valgrind version 3.9.0 and then newest from SVN (13985).

ldd's output for valgrind:
/# ldd /usr/bin/valgrind
ldd: can't open cache '/etc/ld.so.cache'
        libgcc_s.so.1 => /lib/libgcc_s.so.1 (0xb6f27000)
        libc.so.0 => /lib/libc.so.0 (0xb6ed2000)
        ld-uClibc.so.0 => /lib/ld-uClibc.so.0 (0xb6f3a000)


I tried detecting obvious leak in an example:
/* leak.c */
#include <stdlib.h>

int main()
{
	int *x = malloc(10 * sizeof(int));
	return 0;
}

On Linux/x86-64 leak was detected.
On Linux/ARM using --trace-redir=yes option I've found that
vgpreload_core-arm-linux.so and vgpreload_memcheck-arm-linux.so files
are not being read.
Using strace on Valgrind I found that vgpreload_memcheck-arm-linux.so
is accessed, but not used
(access("/usr/lib/valgrind/vgpreload_memcheck-arm-linux.so", R_OK) = 0).


I've checked ld-uClibc-0.9.33.2.so, libuClibc-0.9.33.2.so and
Valgrind's *.so files using 'file' and 'readelf'. They are not
stripped.
Also, have checked libuClibc-0.9.33.2.so for 'malloc' with 'readelf -a
libuClibc-0.9.33.2.so | grep malloc'. It's there.
Have checked sonames. They are same as shown in Valgrind's log.


Full Valgrind log on Linux/ARM:
/# valgrind --trace-redir=yes ./leak
==3526== Memcheck, a memory error detector
==3526== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al.
==3526== Using Valgrind-3.10.0.SVN and LibVEX; rerun with -h for copyright info
==3526== Command: ./leak
==3526==
--3526-- <<
--3526--    ------ REDIR STATE after VG_(redir_initialise) ------
--3526--    TOPSPECS of soname (hardwired)
--3526--      ld-linux-armhf.so.3       memcpy             R->
(0000.0) 0x38075c50
--3526--      ld-linux.so.3             memcpy                 R->
(0000.0) 0x38075c50
--3526--      ld-linux-armhf.so.3       strlen                  R->
(0000.0) 0x38075c24
--3526--      ld-linux.so.3             strlen
R-> (0000.0) 0x38075c24
--3526--    ------ ACTIVE ------
--3526-- >>
--3526-- Reading syms from /transit/misc/leak
--3526--    svma 0x0000008338, avma 0x0000008338
--3526-- <<
--3526--    ------ REDIR STATE after VG_(redir_notify_new_DebugInfo) ------
--3526--    TOPSPECS of soname NONE filename /transit/misc/leak
--3526--    TOPSPECS of soname (hardwired)
--3526--      ld-linux-armhf.so.3       memcpy             R->
(0000.0) 0x38075c50
--3526--      ld-linux.so.3             memcpy                 R->
(0000.0) 0x38075c50
--3526--      ld-linux-armhf.so.3       strlen                  R->
(0000.0) 0x38075c24
--3526--      ld-linux.so.3             strlen
R-> (0000.0) 0x38075c24
--3526--    ------ ACTIVE ------
--3526-- >>
--3526-- Reading syms from /lib/ld-uClibc-0.9.33.2.so
--3526--    svma 0x0000000dc0, avma 0x0004000dc0
--3526-- <<
--3526--    ------ REDIR STATE after VG_(redir_notify_new_DebugInfo) ------
--3526--    TOPSPECS of soname ld-uClibc.so.0 filename
/lib/ld-uClibc-0.9.33.2.so
--3526--    TOPSPECS of soname NONE filename /transit/misc/leak
--3526--    TOPSPECS of soname (hardwired)
--3526--      ld-linux-armhf.so.3       memcpy             R->
(0000.0) 0x38075c50
--3526--      ld-linux.so.3             memcpy                 R->
(0000.0) 0x38075c50
--3526--      ld-linux-armhf.so.3       strlen                  R->
(0000.0) 0x38075c24
--3526--      ld-linux.so.3             strlen
R-> (0000.0) 0x38075c24
--3526--    ------ ACTIVE ------
--3526-- >>
--3526-- Reading syms from /usr/lib/valgrind/memcheck-arm-linux
--3526--    svma 0x00380000c0, avma 0x00380000c0
--3526-- <<
--3526--    ------ REDIR STATE after VG_(redir_notify_new_DebugInfo) ------
--3526--    TOPSPECS of soname NONE filename
/usr/lib/valgrind/memcheck-arm-linux
--3526--    TOPSPECS of soname ld-uClibc.so.0 filename
/lib/ld-uClibc-0.9.33.2.so
--3526--    TOPSPECS of soname NONE filename /transit/misc/leak
--3526--    TOPSPECS of soname (hardwired)
--3526--      ld-linux-armhf.so.3       memcpy             R->
(0000.0) 0x38075c50
--3526--      ld-linux.so.3             memcpy                 R->
(0000.0) 0x38075c50
--3526--      ld-linux-armhf.so.3       strlen                  R->
(0000.0) 0x38075c24
--3526--      ld-linux.so.3             strlen
R-> (0000.0) 0x38075c24
--3526--    ------ ACTIVE ------
--3526-- >>
--3526-- Reading syms from /lib/libgcc_s.so.1
--3526--    svma 0x0000003c58, avma 0x0004812c58
--3526-- <<
--3526--    ------ REDIR STATE after VG_(redir_notify_new_DebugInfo) ------
--3526--    TOPSPECS of soname libgcc_s.so.1 filename /lib/libgcc_s.so.1
--3526--    TOPSPECS of soname NONE filename
/usr/lib/valgrind/memcheck-arm-linux
--3526--    TOPSPECS of soname ld-uClibc.so.0 filename
/lib/ld-uClibc-0.9.33.2.so
--3526--    TOPSPECS of soname NONE filename /transit/misc/leak
--3526--    TOPSPECS of soname (hardwired)
--3526--      ld-linux-armhf.so.3       memcpy             R->
(0000.0) 0x38075c50
--3526--      ld-linux.so.3             memcpy                 R->
(0000.0) 0x38075c50
--3526--      ld-linux-armhf.so.3       strlen                  R->
(0000.0) 0x38075c24
--3526--      ld-linux.so.3             strlen
R-> (0000.0) 0x38075c24
--3526--    ------ ACTIVE ------
--3526-- >>
--3526-- Reading syms from /lib/libuClibc-0.9.33.2.so
--3526--    svma 0x000000a7b0, avma 0x000482c7b0
--3526-- <<
--3526--    ------ REDIR STATE after VG_(redir_notify_new_DebugInfo) ------
--3526--    TOPSPECS of soname libc.so.0 filename /lib/libuClibc-0.9.33.2.so
--3526--    TOPSPECS of soname libgcc_s.so.1 filename /lib/libgcc_s.so.1
--3526--    TOPSPECS of soname NONE filename
/usr/lib/valgrind/memcheck-arm-linux
--3526--    TOPSPECS of soname ld-uClibc.so.0 filename
/lib/ld-uClibc-0.9.33.2.so
--3526--    TOPSPECS of soname NONE filename /transit/misc/leak
--3526--    TOPSPECS of soname (hardwired)
--3526--      ld-linux-armhf.so.3       memcpy             R->
(0000.0) 0x38075c50
--3526--      ld-linux.so.3             memcpy                 R->
(0000.0) 0x38075c50
--3526--      ld-linux-armhf.so.3       strlen                  R->
(0000.0) 0x38075c24
--3526--      ld-linux.so.3             strlen
R-> (0000.0) 0x38075c24
--3526--    ------ ACTIVE ------
--3526-- >>
==3526== Invalid read of size 4
==3526==    at 0x4005550: _dl_get_ready_to_run (in /lib/ld-uClibc-0.9.33.2.so)
==3526==  Address 0xbde78a9c is just below the stack ptr.  To
suppress, use: --workaround-gcc296-bugs=yes
==3526==
==3526== Invalid read of size 4
==3526==    at 0x4864C34: __uClibc_main (in /lib/libuClibc-0.9.33.2.so)
==3526==  Address 0xbde78c6c is just below the stack ptr.  To
suppress, use: --workaround-gcc296-bugs=yes
==3526==
==3526== Invalid read of size 4
==3526==    at 0x4864A30: __uClibc_fini (in /lib/libuClibc-0.9.33.2.so)
==3526==  Address 0xbde78c4c is just below the stack ptr.  To
suppress, use: --workaround-gcc296-bugs=yes
==3526==
==3526== Invalid read of size 4
==3526==    at 0x4000E1C: ??? (in /lib/ld-uClibc-0.9.33.2.so)
==3526==  Address 0xbde78c34 is just below the stack ptr.  To
suppress, use: --workaround-gcc296-bugs=yes
==3526==
==3526==
==3526== HEAP SUMMARY:
==3526==     in use at exit: 0 bytes in 0 blocks
==3526==   total heap usage: 0 allocs, 0 frees, 0 bytes allocated
==3526==
==3526== All heap blocks were freed -- no leaks are possible
==3526==
==3526== For counts of detected and suppressed errors, rerun with: -v
==3526== ERROR SUMMARY: 32 errors from 4 contexts (suppressed: 0 from 0)


On Linux/x86-64 using --trace-redir=yes I've found, that functions
from libc.so.0 are being redirected after
vgpreload_memcheck-amd64-linux.so is being read.
Excerpt from log:
>>
--6671-- Reading syms from /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so
--6671--    svma 0x0000002f10, avma 0x0004c28f10
--6671-- <<
--6671--    ------ REDIR STATE after VG_(redir_notify_new_DebugInfo) ------
--6671--    TOPSPECS of soname NONE filename
/usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so
--6671--      libc.so*                  setenv                W->
(0000.0) 0x04c2f8a0
--6671--      libc.so*                  unsetenv            W->
(0000.0) 0x04c2f800
--6671--      libc.so*                  putenv                W->
(0000.0) 0x04c2f760
--6671--      libc.so*                  wcslen                R->
(2037.0) 0x04c2f740
--6671--      libc.so*                  strcasestr           R->
(2035.0) 0x04c2f680
--6671--      libc.so*                  strspn                 R->
(2034.0) 0x04c2f600


Also hardwired sonames are redirected on x86-64, but not on ARM.
But that's maybe cause soname 'ld-uClibc.so.0' differs from
'ld-linux.so.3' and 'ld-linux-armhf.so.3'.

Thank you very much in advance!
Best regards,
Andrejs

Re: [Valgrind-users] m_cpuid.S: Assembler messages:

[Andy H. <an...@ga...>]

On 04/17/2014 06:08 PM, John Reiser wrote:
>> I am trying to compile and install valgrind on a 64 bits CENT OS 6.5 system.
>>
>> When I compile valgrind (with make), I get the following errors:
>>
>> m_cpuid.S:40: Error: suffix or operands invalid for `push'
> 
>> m_cpuid.S:43: Error: suffix or operands invalid for `pushf'
> 
>> m_cpuid.S:45: Error: suffix or operands invalid for `pop'
> 
>> gcc (GCC) 4.7.3
> 
> Unfortunately that version of gcc/gas/bfd/binutils has a bug
> in not allowing the 'l' on "pushl %ebp", "popl %ebp", "pushfl".
> (They changed gcc and binutils together to omit the 'l',
> but forgot that the rest of the world requires
> backward compatibility with the old spellings of the opcodes.)
> 
> Either change the source to remove the 'l' on the indicated lines,
> or upgrade to gcc-4.8.x.
> 

I just ran into the same problem under CentOS 5.10. I removed the 'l', but still get the
same problem. gcc (GCC) 4.1.2 20080704 (Red Hat 4.1.2-54)

m_cpuid.S:40: Error: suffix or operands invalid for `push'

Line 40 now has:
        push   %ebp

I tried to make -k to see what else has problems:

m_syscall.c
m_trampoline.S
dispatch-x86-linux.S
...

Along with a bunch of others. I think something else must be wrong, though I don't know
what. Make is running:

gcc -DHAVE_CONFIG_H -I. -I..  -I.. -I../include -I../VEX/pub -DVGA_x86=1 -DVGO_linux=1
-DVGP_x86_linux=1 -DVGPV_x86_linux_vanilla=1 -I../coregrind
-DVG_LIBDIR="\"/usr/lib64/valgrind"\" -DVG_PLATFORM="\"x86-linux\""  -m32 -g
-Wno-long-long -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector
--param=ssp-buffer-size=4 -m64 -mtune=generic -MT libcoregrind_x86_linux_a-m_cpuid.o -MD
-MP -MF .deps/libcoregrind_x86_linux_a-m_cpuid.Tpo -c -o
libcoregrind_x86_linux_a-m_cpuid.o `test -f 'm_cpuid.S' || echo './'`m_cpuid.S

Regards,

	Andy

Re: [Valgrind-users] Instrumentation time

[Philippe W. <phi...@sk...>]

On Thu, 2014-05-15 at 21:06 +0000, Domingues Luis Filipe wrote:
> Hello,
> 
> I have a question about how Valgrind works for the instrumentation.
> Did he instrument the code during the run time, or did he make the
> instrumentation of all the program and the run the instrumented code on
> a low level virtual machine?

The instrumentation is done at runtime.

This a.o. allows to run program that generates code (e.g. firefox and
its Javascript JIT compiler).

Philippe

[Valgrind-users] Instrumentation time

[Domingues L. F. <Lui...@ed...>]

Hello,

I have a question about how Valgrind works for the instrumentation.
Did he instrument the code during the run time, or did he make the
instrumentation of all the program and the run the instrumented code on
a low level virtual machine?

Regards,
Luis Domingues

Re: [Valgrind-users] [callgrind] CGAL profiling error, over 100% of instruction count

[Jin T. L. <jl...@co...>]

Thanks Josef!

It helps me a lot!

Jintack

On May 15, 2014, at 1:47 PM, Josef Weidendorfer <Jos...@gm...> wrote:

> Hi Jin,
> 
> thanks, I can reproduce it.
> 
> This is a bug in KCachegrind, and not in callgrind itself.
> 
> For some reason, the caller/callee view does not correctly
> recognize this as recursive function call. It simply should
> leave the cost column empty.
> 
> Simple workaround: just ignore.
> 
> Josef
> 
> 
> Am 11.05.2014 22:55, schrieb Jin Tack Lim:
>> 
>> The version of valgrind: valgrind-3.9.0
>> HW/OS: Macbook pro retina(core: i7), Ubuntu 12.04 on Parallels. 
>> 
>> Following link has the callgrind file.
>> https://www.dropbox.com/s/241sovlq14nrxmi/callgrind.tar.gz
>> 
>> I’m not sure the function name,
>> but the caller of the function is “create_internal_node_use_extension”,
>> incl. is 70.73%
>> This function is a directly recursive function.
>> If you see the callees tab, you can see one of the callee takes 408.45%
>> However, if you see call graph, this callee is disappeared.
>> 
>> I turned on “Relative”, “Cycle Detection”, “Relative to Parent” option
>> in kcachegrind.
>> 
>> 
>> On May 11, 2014, at 2:17 PM, John Reiser <jr...@Bi...
>> <mailto:jr...@Bi...>> wrote:
>> 
>>>> While I was profiling CGAL4.4 Spatial searching with callgrind,
>>>> I found that some function takes more than 100% of instruction. :(
>>>> I compiled CGAL with -g and -O0 options.
>>>> 
>>>> Is this common for other applications?
>>> 
>>> No.
>>> 
>>>> Any idea??
>>> 
>>> Tell us which version of callgrind.
>>> Tell us the name of the hardware architecture and operating system and
>>> runtime library.
>>> Tell us the name of the function, and the actual counts which seem bad.
>>> Is there only one function with bad counts?
>>> Is it directly recursive (calls itself)?
>>> Is it indirectly recursive (is in a recursive loop of multiple functions)?
>>> 
>>> 
>>> 
>>> 
>>> ------------------------------------------------------------------------------
>>> Is your legacy SCM system holding you back? Join Perforce May 7 to
>>> find out:
>>> &#149; 3 signs your SCM is hindering your productivity
>>> &#149; Requirements for releasing software faster
>>> &#149; Expert tips and advice for migrating your SCM now
>>> http://p.sf.net/sfu/perforce
>>> _______________________________________________
>>> Valgrind-users mailing list
>>> Val...@li...
>>> https://lists.sourceforge.net/lists/listinfo/valgrind-users
>> 
>> 
>> 
>> ------------------------------------------------------------------------------
>> Is your legacy SCM system holding you back? Join Perforce May 7 to find out:
>> &#149; 3 signs your SCM is hindering your productivity
>> &#149; Requirements for releasing software faster
>> &#149; Expert tips and advice for migrating your SCM now
>> http://p.sf.net/sfu/perforce
>> 
>> 
>> 
>> _______________________________________________
>> Valgrind-users mailing list
>> Val...@li...
>> https://lists.sourceforge.net/lists/listinfo/valgrind-users
>> 
> 
> ------------------------------------------------------------------------------
> "Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
> Instantly run your Selenium tests across 300+ browser/OS combos.
> Get unparalleled scalability from the best Selenium testing platform available
> Simple to use. Nothing to install. Get started now for free."
> http://p.sf.net/sfu/SauceLabs
> _______________________________________________
> Valgrind-users mailing list
> Val...@li...
> https://lists.sourceforge.net/lists/listinfo/valgrind-users

Re: [Valgrind-users] [callgrind] CGAL profiling error, over 100% of instruction count

[Josef W. <Jos...@gm...>]

Hi Jin,

thanks, I can reproduce it.

This is a bug in KCachegrind, and not in callgrind itself.

For some reason, the caller/callee view does not correctly
recognize this as recursive function call. It simply should
leave the cost column empty.

Simple workaround: just ignore.

Josef


Am 11.05.2014 22:55, schrieb Jin Tack Lim:
> 
> The version of valgrind: valgrind-3.9.0
> HW/OS: Macbook pro retina(core: i7), Ubuntu 12.04 on Parallels. 
> 
> Following link has the callgrind file.
> https://www.dropbox.com/s/241sovlq14nrxmi/callgrind.tar.gz
> 
> I’m not sure the function name,
> but the caller of the function is “create_internal_node_use_extension”,
> incl. is 70.73%
> This function is a directly recursive function.
> If you see the callees tab, you can see one of the callee takes 408.45%
> However, if you see call graph, this callee is disappeared.
> 
> I turned on “Relative”, “Cycle Detection”, “Relative to Parent” option
> in kcachegrind.
> 
> 
> On May 11, 2014, at 2:17 PM, John Reiser <jr...@Bi...
> <mailto:jr...@Bi...>> wrote:
> 
>>> While I was profiling CGAL4.4 Spatial searching with callgrind,
>>> I found that some function takes more than 100% of instruction. :(
>>> I compiled CGAL with -g and -O0 options.
>>>
>>> Is this common for other applications?
>>
>> No.
>>
>>> Any idea??
>>
>> Tell us which version of callgrind.
>> Tell us the name of the hardware architecture and operating system and
>> runtime library.
>> Tell us the name of the function, and the actual counts which seem bad.
>> Is there only one function with bad counts?
>> Is it directly recursive (calls itself)?
>> Is it indirectly recursive (is in a recursive loop of multiple functions)?
>>
>>
>>
>>
>> ------------------------------------------------------------------------------
>> Is your legacy SCM system holding you back? Join Perforce May 7 to
>> find out:
>> &#149; 3 signs your SCM is hindering your productivity
>> &#149; Requirements for releasing software faster
>> &#149; Expert tips and advice for migrating your SCM now
>> http://p.sf.net/sfu/perforce
>> _______________________________________________
>> Valgrind-users mailing list
>> Val...@li...
>> https://lists.sourceforge.net/lists/listinfo/valgrind-users
> 
> 
> 
> ------------------------------------------------------------------------------
> Is your legacy SCM system holding you back? Join Perforce May 7 to find out:
> &#149; 3 signs your SCM is hindering your productivity
> &#149; Requirements for releasing software faster
> &#149; Expert tips and advice for migrating your SCM now
> http://p.sf.net/sfu/perforce
> 
> 
> 
> _______________________________________________
> Valgrind-users mailing list
> Val...@li...
> https://lists.sourceforge.net/lists/listinfo/valgrind-users
> 

Re: [Valgrind-users] understanding massif output when run against a simple executable

[Alan M. <ala...@jp...>]

I'm not the original questioner, but this brings up an interesting 
question: what books are out there?  I'm familiar with all of this 
already, but I know of no single books that cover the language, along 
with compilation, along with the OS aspects.  If there is one out there, 
I'd love to know.

-- Alan


On 5/14/2014 2:46 PM, Eliot Moss wrote:
> You may benefit from finding documentation or a textbook about how 
> compiling, linking, loading, etc., works for C programs in Unix/Linux, 
> etc. (The Windows world is similar, though of course different in some 
> details.)

Re: [Valgrind-users] understanding massif output when run against a simple executable

[Eliot M. <mo...@cs...>]

On 5/14/2014 2:21 PM, Rob Taylor wrote:
> Thank you Milian Wolff and Eliot Moss,

Typical users of tools like valgrind have more background in
how all this works -- not a criticism of you, just a difference
in expectation.

The volume of code that results from compiling a program
consisting of a main function that simply returns is quite
small.  It is stored on disk in the executable file produced
by the C compiling and linking process.  That file also
describes any global / static storage needed for the program.
When a process is created to run the program, the operating
system sets up an area in low memory (not the stack) for
that data.  Local / automatic (in C terminology) variables
are allocated in the stack, and done so by each function as
you enter the function.  The operating system grows the stack
(from high addresses toward lower ones) automatically.

It is also possible to allocate, and "free", memory on request
using library functions such as malloc and free.

Any libraries used will be linked in (connected to your code)
on demand, and they may also have static data areas and may
malloc/free data as well.

You may benefit from finding documentation or a textbook about
how compiling, linking, loading, etc., works for C programs in
Unix/Linux, etc.  (The Windows world is similar, though of course
different in some details.)

Regards -- Eliot Moss

Re: [Valgrind-users] understanding massif output when run against a simple executable

[Rob T. <rta...@gm...>]

Thank you Milian Wolff and Eliot Moss,

When attempting to profile the total memory usage of my small application,
which I called tail according to the Excercise 5-13 of the C Programming
Language (second edition) by Brian W. Kernighan and Denis M. Richie, if I
try to run valgrind like this:

cat ~/my_lfs_7_4_notes_for_1.6.2.txt | valgrind --tool=massif --stacks=yes
--time-unit=B ./tail -n1000

It looks like the program uses a max of 3,360 bytes of memory (all stack
usage).

But in 2 of my source files I have allocations like this:

#define MAXLINES 1000L /* max lines to be stored */
static char *lineptr[MAXLINES]; /* pointers to text lines */

#define BUFSIZE 33000UL /* size of available buffer space */
static char buf[BUFSIZE]; /* array of BUFSIZE to store lines */

#define MAXLENGTH 1000 /* set the maximum line length stored */
char s[MAXLENGTH]; /* storage for get_line function */

>From this I estimate memory usage should be in excess of 42,000 bytes.

The closest that I can seem to get to a real representation of memory used
is if I subtract  the amount reported for an empty program from the amount
reported for this program when I use the option --pages-as-heap=yes as
follows:

cat ~/my_lfs_7_4_notes_for_1.6.2.txt | valgrind --tool=massif
--pages-as-heap=yes --time-unit=B ./tail -n1000
valgrind --tool=massif --pages-as-heap=yes --time-unit=B ./just_return

Then calculate the difference in memory usage reported for the 2 runs...

6,234,112 - 6,189,056 = 45,056

This seems more like the impact that my code has had on memory usage and
represents the part I can change or optimize, since I have no control over
whatever created the other 6 million bytes of usage even with an entirely
empty c program (or whether the libraries are shared or not). Is this a
fair assessment?

I'm guessing the C code stored on disk is also in memory somewhere when the
program is loaded. Though I have difficulty seeing that in these memory
outputs and can only guess how the size of my compiled code affects memory
usage.

Thank you for the feedback. Books and documentation just never seems to be
enough to learn this stuff.
Rob Taylor


On Wed, May 14, 2014 at 2:43 AM, Milian Wolff <ma...@mi...> wrote:

> On Tuesday 13 May 2014 19:13:23 Eliot Moss wrote:
> > On 5/13/2014 6:49 PM, Rob Taylor wrote:
> > > I recently discovered valgrind while searching for a way to see how
> much
> > > memory (total memory footprint) my small program represented.
> > >
> > > I was perplexed by the results I was seeing, since I expected at most
> to
> > > use perhaps tens of thousands of bites not Millions. My application is
> > > not dynamically requesting/allocating memory, all memory consumed is in
> > > the form of statically assigned character arrays, automatic stack usage
> > > etc..
> > >
> > > So I did a test by running valgrind against an empty C program. The
> > > program does nothing but return. Can someone explain why the output
> shows
> > > continually increasing memory used and in total over a 6 MB peak?
> >
> > Dynamically linked libraries and their data.  Some of that
> > (most of the code areas) can be shared in real memory with
> > other programs using the same library.  There's a lot of
> > mechanism there, even if you are not using much of it, and
> > even if you are using parts of it without realizing it ...
>
> Yep, you are investigating on the page level, which is different from what
> you
> allocate. Remove that argument, but try --stacks=yes instead.
>
> Bye
>
> --
> Milian Wolff
> ma...@mi...
> http://milianw.de
>
>
> ------------------------------------------------------------------------------
> "Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
> Instantly run your Selenium tests across 300+ browser/OS combos.
> Get unparalleled scalability from the best Selenium testing platform
> available
> Simple to use. Nothing to install. Get started now for free."
> http://p.sf.net/sfu/SauceLabs
> _______________________________________________
> Valgrind-users mailing list
> Val...@li...
> https://lists.sourceforge.net/lists/listinfo/valgrind-users
>

Re: [Valgrind-users] understanding massif output when run against a simple executable

[Milian W. <ma...@mi...>]

On Tuesday 13 May 2014 19:13:23 Eliot Moss wrote:
> On 5/13/2014 6:49 PM, Rob Taylor wrote:
> > I recently discovered valgrind while searching for a way to see how much
> > memory (total memory footprint) my small program represented.
> > 
> > I was perplexed by the results I was seeing, since I expected at most to
> > use perhaps tens of thousands of bites not Millions. My application is
> > not dynamically requesting/allocating memory, all memory consumed is in
> > the form of statically assigned character arrays, automatic stack usage
> > etc..
> > 
> > So I did a test by running valgrind against an empty C program. The
> > program does nothing but return. Can someone explain why the output shows
> > continually increasing memory used and in total over a 6 MB peak?
> 
> Dynamically linked libraries and their data.  Some of that
> (most of the code areas) can be shared in real memory with
> other programs using the same library.  There's a lot of
> mechanism there, even if you are not using much of it, and
> even if you are using parts of it without realizing it ...

Yep, you are investigating on the page level, which is different from what you 
allocate. Remove that argument, but try --stacks=yes instead.

Bye

-- 
Milian Wolff
ma...@mi...
http://milianw.de

Re: [Valgrind-users] understanding massif output when run against a simple executable

[Eliot M. <mo...@cs...>]

On 5/13/2014 6:49 PM, Rob Taylor wrote:
> I recently discovered valgrind while searching for a way to see how much memory (total memory
> footprint) my small program represented.
>
> I was perplexed by the results I was seeing, since I expected at most to use perhaps tens of
> thousands of bites not Millions. My application is not dynamically requesting/allocating memory, all
> memory consumed is in the form of statically assigned character arrays, automatic stack usage etc..

> So I did a test by running valgrind against an empty C program. The program does nothing but return.
> Can someone explain why the output shows continually increasing memory used and in total over a 6 MB
> peak?

Dynamically linked libraries and their data.  Some of that
(most of the code areas) can be shared in real memory with
other programs using the same library.  There's a lot of
mechanism there, even if you are not using much of it, and
even if you are using parts of it without realizing it ...

Regards -- Eliot Moss

[Valgrind-users] understanding massif output when run against a simple executable

[Rob T. <rta...@gm...>]

I recently discovered valgrind while searching for a way to see how much
memory (total memory footprint) my small program represented.

I was perplexed by the results I was seeing, since I expected at most to
use perhaps tens of thousands of bites not Millions. My application is not
dynamically requesting/allocating memory, all memory consumed is in the
form of statically assigned character arrays, automatic stack usage etc..

valgrind --version
valgrind-3.9.0

I'm running on LFS 7.4
linux-3.11
glibc-2.18
gcc-4.8.1

So I did a test by running valgrind against an empty C program. The program
does nothing but return. Can someone explain why the output shows
continually increasing memory used and in total over a 6 MB peak?

user [ ~/ccode/string_buffer ]> cat > ./just_return.c << "EOF"
> int main(void)
> {
> return 0;
> }
> EOF
user [ ~/ccode/string_buffer ]> gcc -O0 -Wall -g -o just_return
just_return.c
user [ ~/ccode/string_buffer ]> valgrind --tool=massif --pages-as-heap=yes
--time-unit=B ./just_return
==11740== Massif, a heap profiler
==11740== Copyright (C) 2003-2013, and GNU GPL'd, by Nicholas Nethercote
==11740== Using Valgrind-3.9.0 and LibVEX; rerun with -h for copyright info
==11740== Command: ./just_return
==11740==
==11740==
user [ ~/ccode/string_buffer ]> ms_print massif.out.11740
--------------------------------------------------------------------------------
Command:            ./just_return
Massif arguments:   --pages-as-heap=yes --time-unit=B
ms_print arguments: massif.out.11740
--------------------------------------------------------------------------------


    MB
5.902^
:

|                                                                       #

|                                                                       #

|                                                                       #

|                                                                       #

|                                                                       #

|                                                                       #

|                                                                       #

|                                                                       #

|                                                                       #

|                                                                       #

|                                                                       #

|                                                                       #
     |
:::::::::::::::::::::::::::::::::::::::::::::#
     |
:                                            #
     |
:                                            #
     |
:                                            #
     |
:                                            #
     |
:                                            #
     |
:                                            #
   0
+----------------------------------------------------------------------->MB
     0
5.918

Number of snapshots: 21
 Detailed snapshots: [9, 19 (peak)]

--------------------------------------------------------------------------------
  n        time(B)         total(B)   useful-heap(B) extra-heap(B)
stacks(B)
--------------------------------------------------------------------------------
  0          4,096            4,096            4,096
0            0
  1          8,192            8,192            8,192
0            0
  2        139,264          139,264          139,264
0            0
  3        147,456          147,456          147,456
0            0
  4        151,552          151,552          151,552
0            0
  5        155,648          155,648          155,648
0            0
  6        159,744          159,744          159,744
0            0
  7        167,936          167,936          167,936
0            0
  8        172,032          172,032          172,032
0            0
  9        172,032          172,032          172,032
0            0
100.00% (172,032B) (page allocation syscalls) mmap/mremap/brk, --alloc-fns,
etc.
->100.00% (172,032B) 0xFFFFFFFFFFFFFFFF: ???

--------------------------------------------------------------------------------
  n        time(B)         total(B)   useful-heap(B) extra-heap(B)
stacks(B)
--------------------------------------------------------------------------------
 10        176,128          176,128          176,128
0            0
 11      2,277,376        2,277,376        2,277,376
0            0
 12      2,281,472        2,281,472        2,281,472
0            0
 13      2,297,856        2,297,856        2,297,856
0            0
 14      6,139,904        6,139,904        6,139,904
0            0
 15      6,164,480        6,164,480        6,164,480
0            0
 16      6,180,864        6,180,864        6,180,864
0            0
 17      6,184,960        6,184,960        6,184,960
0            0
 18      6,189,056        6,189,056        6,189,056
0            0
 19      6,201,344        6,176,768        6,176,768
0            0
100.00% (6,176,768B) (page allocation syscalls) mmap/mremap/brk,
--alloc-fns, etc.
->97.21% (6,004,736B) 0x4017119: mmap (in /lib/ld-2.18.so)
| ->96.22% (5,943,296B) 0x40064F0: _dl_map_object_from_fd (in /lib/
ld-2.18.so)
| | ->96.22% (5,943,296B) 0x4007F93: _dl_map_object (in /lib/ld-2.18.so)
| |   ->62.20% (3,842,048B) 0x400C230: openaux (in /lib/ld-2.18.so)
| |   | ->62.20% (3,842,048B) 0x400E682: _dl_catch_error (in /lib/ld-2.18.so
)
| |   |   ->62.20% (3,842,048B) 0x400C483: _dl_map_object_deps (in /lib/
ld-2.18.so)
| |   |     ->62.20% (3,842,048B) 0x4002D52: dl_main (in /lib/ld-2.18.so)
| |   |       ->62.20% (3,842,048B) 0x40150B3: _dl_sysdep_start (in /lib/
ld-2.18.so)
| |   |         ->62.20% (3,842,048B) 0x40049A4: _dl_start (in /lib/
ld-2.18.so)
| |   |           ->62.20% (3,842,048B) 0x4001136: ??? (in /lib/ld-2.18.so)
| |   |
| |   ->34.02% (2,101,248B) 0x400131C: map_doit (in /lib/ld-2.18.so)
| |     ->34.02% (2,101,248B) 0x400E682: _dl_catch_error (in /lib/ld-2.18.so
)
| |       ->34.02% (2,101,248B) 0x4000F78: do_preload (in /lib/ld-2.18.so)
| |         ->34.02% (2,101,248B) 0x40040C6: dl_main (in /lib/ld-2.18.so)
| |           ->34.02% (2,101,248B) 0x40150B3: _dl_sysdep_start (in /lib/
ld-2.18.so)
| |             ->34.02% (2,101,248B) 0x40049A4: _dl_start (in /lib/
ld-2.18.so)
| |               ->34.02% (2,101,248B) 0x4001136: ??? (in /lib/ld-2.18.so)
| |
| ->00.99% (61,440B) in 1+ places, all below ms_print's threshold (01.00%)
|
->02.79% (172,032B) 0xFFFFFFFFFFFFFFFF: ???

--------------------------------------------------------------------------------
  n        time(B)         total(B)   useful-heap(B) extra-heap(B)
stacks(B)
--------------------------------------------------------------------------------
 20      6,205,440        6,172,672        6,172,672
0            0


Thanks,
Rob Taylor

Re: [Valgrind-users] Does Helgrind work effectively with condition variables and pthread_cancel?

[Steven Stewart-G. <sst...@my...>]

> I think I observed a strange behaviour on Linux which does not comply
> with POSIX.
> That is, POSIX mandates that a thread blocked on pthread_cond_wait()
> and canceled
> via pthread_cancel() gets unblocked, the corresponding mutex
> re-acquired and _then_
> cancellation cleanup handlers called.
> However on Linux this was not true - thread unblocked from
> pthread_cond_wait() did
> not re-acquire the associated mutex.



Thank you, Ivo Raisr. That makes sense and seems to fix my problem. I
think the relevant section of POSIX is:

> A side-effect of acting on a cancellation request while a thread is
> blocked on a condition variable is to re-acquire the mutex before
> calling any of the cancellation cleanup handlers.

I find it strange that GLibc does not implement this case properly.

[Valgrind-users] Does Helgrind work effectively with condition variables and pthread_cancel?

[Steven Stewart-G. <sst...@my...>]

The FAQ mentions that there are some problems with condition variables and I
seem to get false reports about destroying waited on condition variables from
Helgrind. I'm using really simple code like the following:

struct linted_array_queue
{
    pthread_mutex_t mutex;
    pthread_cond_t on_empty;
    pthread_cond_t on_full;
    size_t message_size;
    bool occupied;
    char message_buffer[];
};

static void unlock_routine(void* arg)
{
    pthread_mutex_t* mutex = arg;
    pthread_mutex_unlock(mutex);
}

void linted_array_queue_send(struct linted_array_queue* queue,
                             void const* message)
{
    pthread_mutex_lock(&queue->mutex);
    pthread_cleanup_push(unlock_routine, &queue->mutex);

    while (queue->occupied) {
        pthread_cond_wait(&queue->on_empty, &queue->mutex);
    }

    queue->occupied = true;
    memcpy(queue->message_buffer, message, queue->message_size);

    pthread_cond_signal(&queue->on_full);

    pthread_cleanup_pop(true);
}

I think it is correct but maybe Helgrind can't deal with condition variables and
cancellation. Is there any metadata or markup I can use to please Helgrind?

Re: [Valgrind-users] [callgrind] CGAL profiling error, over 100% of instruction count

[Jin T. L. <jl...@co...>]

The version of valgrind: valgrind-3.9.0
HW/OS: Macbook pro retina(core: i7), Ubuntu 12.04 on Parallels. 

Following link has the callgrind file.
https://www.dropbox.com/s/241sovlq14nrxmi/callgrind.tar.gz

I’m not sure the function name,
but the caller of the function is “create_internal_node_use_extension”, incl. is 70.73%
This function is a directly recursive function.
If you see the callees tab, you can see one of the callee takes 408.45%
However, if you see call graph, this callee is disappeared.

I turned on “Relative”, “Cycle Detection”, “Relative to Parent” option in kcachegrind.


On May 11, 2014, at 2:17 PM, John Reiser <jr...@Bi...> wrote:

>> While I was profiling CGAL4.4 Spatial searching with callgrind,
>> I found that some function takes more than 100% of instruction. :(
>> I compiled CGAL with -g and -O0 options.
>> 
>> Is this common for other applications?
> 
> No.
> 
>> Any idea??
> 
> Tell us which version of callgrind.
> Tell us the name of the hardware architecture and operating system and runtime library.
> Tell us the name of the function, and the actual counts which seem bad.
> Is there only one function with bad counts?
> Is it directly recursive (calls itself)?
> Is it indirectly recursive (is in a recursive loop of multiple functions)?
> 
> 
> 
> 
> ------------------------------------------------------------------------------
> Is your legacy SCM system holding you back? Join Perforce May 7 to find out:
> &#149; 3 signs your SCM is hindering your productivity
> &#149; Requirements for releasing software faster
> &#149; Expert tips and advice for migrating your SCM now
> http://p.sf.net/sfu/perforce
> _______________________________________________
> Valgrind-users mailing list
> Val...@li...
> https://lists.sourceforge.net/lists/listinfo/valgrind-users

Re: [Valgrind-users] [callgrind] CGAL profiling error, over 100% of instruction count

[John R. <jr...@Bi...>]

> While I was profiling CGAL4.4 Spatial searching with callgrind,
> I found that some function takes more than 100% of instruction. :(
> I compiled CGAL with -g and -O0 options.
> 
> Is this common for other applications?

No.

> Any idea??

Tell us which version of callgrind.
Tell us the name of the hardware architecture and operating system and runtime library.
Tell us the name of the function, and the actual counts which seem bad.
Is there only one function with bad counts?
Is it directly recursive (calls itself)?
Is it indirectly recursive (is in a recursive loop of multiple functions)?

[Valgrind-users] [callgrind] CGAL profiling error, over 100% of instruction count

[Jin T. L. <jl...@co...>]

Hi,


While I was profiling CGAL4.4 Spatial searching with callgrind,
I found that some function takes more than 100% of instruction. :(
I compiled CGAL with -g and -O0 options.

Is this common for other applications?
Any idea??

Re: [Valgrind-users] Marking blocks as write-once/grow-only

[Julian S. <js...@ac...>]

On 05/08/2014 10:26 AM, Alex Bennée wrote:
> 
> Julian Seward <js...@ac...> writes:
> 
>>> Is it possible for the DRD/Helgrind tools to detect this sort of
>>> double-write access behaviour?
>>
>> Both of them should be able to detect a write-vs-write race, if
>> that's what you mean.
> 
> But is that only if two threads race to write the same location at the
> same time?

Well, yes.  But isn't that what you care about?  Or are you concerned
about a race at a higher level of granularity?


> What I think has happened is:
> 
> (1) thread a writes from code_buf_start a series of operations
> (2) thread b writes from code_buf_start over the top of thread a
> (3) thread b finalises the code buffer at code_buf_end<b>
> (4) thread a finalises the code buffer at code_buf_end<a>

I'm still confused.  What does this sequence look like if you
add in the lock acquire/release points?  This sounds like it
ought to be straightforward, but you need to say what locks
exist, what they protect, and when they are supposed to be
acquired/released.

In the absence of further info I'd assume that there is just
one lock, which protects the entire code buffer and any
"finalisation info".  And that is supposed to be acquired and
held for the entire period in which a thread writes to and
finalises the code buffer.

In which case the above sequence can't happen, since (2) will
block as soon as it tries to acquire the lock.

J

Re: [Valgrind-users] Marking blocks as write-once/grow-only

[Alex B. <ker...@be...>]

Julian Seward <js...@ac...> writes:

>> Is it possible for the DRD/Helgrind tools to detect this sort of
>> double-write access behaviour?
>
> Both of them should be able to detect a write-vs-write race, if
> that's what you mean.

But is that only if two threads race to write the same location at the
same time?

>
>> Could I instrument QEMU so it marked the
>> codegen buffer as one that should only grow upwards (modulo-patchable
>> bits) so if anything re-wrote the buffer it could trigger an error?
>
> This is confusing.  Both tools are able to detect races at a byte level
> granularity.  If you can show that QEMU doesn't race on individual writes
> to its code buffer, isn't that good enough from a correctness
> perspective?

What I think has happened is:

  * thread a writes from code_buf_start a series of operations
  * thread b writes from code_buf_start over the top of thread a
  * thread b finalises the code buffer at code_buf_end<b>
  * thread a finalises the code buffer at code_buf_end<a>

The result is a corrupted code buffer where one threads output has been
stomped on by the other. I don't think they raced on the individual
writes, just over wrote the previous work on the next schedule.
>
> J

-- 
Alex Bennée

Re: [Valgrind-users] VG_(message) floating points

[Florian K. <fl...@ei...>]

On 05/07/2014 10:27 AM, Domingues Luis Filipe wrote:
> Hello,
> 
> On a tool, I try to print on the stdout the value of a floating point.
> But when I use the %f formating in VG(printf) or VG_(message) it print
> my string, but not my float/double.
> 
> It is a bug? Or there are another way to print floating points?

It's a missing feature. Search the valgrind-developers mailing list for
a thread entitled "VG_(printf) '%f' patch" which was meant to add
support for printing floating point numbers. But it was never integrated
into the source tree (not sure why; perhaps a license issue).
The patch may solve your problem.

  Florian