valgrind-users — General discussion list for Valgrind users

Re: [Valgrind-users] Who's doing valgrind on Solaris?

[Ivosh <iv...@iv...>]

Skip Montanaro <skip <at> pobox.com> writes:

> 
> On the valgrind platforms page I saw this:
> 
>     Less mature ports for x86/Solaris and x86/NetBSD are also being worked on.
> 
> Who is doing the Solaris port?  I can't help with directly with the
development
> (don't have the necessary background), but would be happy to help test it,
> provide build/bug reports, etc.
> 

There is a work-in-progress:
https://bitbucket.org/setupji/valgrind-solaris

Re: [Valgrind-users] mandatory function redirection error

[John R. <jr...@bi...>]

> I'm on a beaglebone white with vanilla Angstrom (linux 3.2)
> distribution.  Valgrind fails like this:

>    ==13719== Using Valgrind-3.8.1 and LibVEX; rerun with -h for copyright info

>    valgrind:  A must-be-redirected function
>    valgrind:  whose name matches the pattern:      memcpy
>    valgrind:  in an object with soname matching:   ld-linux.so.3
>    valgrind:  was not found whilst processing
>    valgrind:  symbols from the object with soname: ld-linux.so.3

> I've installed libc6-dbg as it says.  Same problem.

Which other Linux distribution is Angstrom derived from, or is most similar to?

Try running the previous version 3.7 of valgrind.
Also post the output from:
  readelf --symbols ld-linux.so.3  |  grep mem

-- 

[Valgrind-users] disInstr(arm): unhandled instruction: 0x69746E65

[王功庆 <wg...@gm...>]

Hi all,
    I'm using valgrind 3.81 to debug a memory overflow issue, and
encountered the exception  mentioned in the title.
    My configuration is listed below, and more information can be got at
https://bugs.kde.org/show_bug.cgi?id=319968 , where i have already filed a
bug.
   If it was caused by my error using, please be kindly give me some light,
thanks very much!

1) os: Linux yyyyy 2.6.35 #11 SMP PREEMPT Thu May 16 16:33:16 CST 2013
armv7l GNU/Linux
2) valgrind version: Current release: valgrind-3.8.1
3) build configure: ./configure --prefix==$(pwd)../build
--host=armv7a_xxxx451_001_vfp-linux-gnueabi
4) valgrind command log:
(130517_20:38:40.128)VALGRIND_LIB=/mnt/sda1/valgrind/lib/valgrind
(130517_20:38:40.128)/mnt/sda1/valgrind/bin/valgrind -v --leak-check=full
/mnt/sda1/sysroot/ld-2.12.2.so /mnt/sda1/app -Q source_type= -jz -T 7
5) valgrind exception info:
(130517_20:38:40.362)--1071-- REDIR: 0x11ec00 (memcpy) redirected to
0x380443d0 (???)
(130517_20:38:40.440)--1071-- REDIR: 0x11e050 (strlen) redirected to
0x380443a4 (???)
(130517_20:38:40.893)disInstr(arm): unhandled instruction: 0x69746E65
(130517_20:38:40.893)                 cond=6(0x6) 27:20=151(0x97) 4:4=0
3:0=5(0x5)
(130517_20:38:40.893)disInstr(arm): unhandled instruction: 0x69746E65
(130517_20:38:40.893)                 cond=6(0x6) 27:20=151(0x97) 4:4=0
3:0=5(0x5)

Best Regards,
Gongqing.Wang @ China/Wuhan

[Valgrind-users] mandatory function redirection error

[Britton K. <bri...@gm...>]

I'm on a beaglebone white with vanilla Angstrom (linux 3.2)
distribution.  Valgrind fails like this:

   root@bboneumh2:~/software# ~/local/bin/valgrind --leak-check=yes ./my_program
   ==13719== Memcheck, a memory error detector
   ==13719== Copyright (C) 2002-2012, and GNU GPL'd, by Julian Seward et al.
   ==13719== Using Valgrind-3.8.1 and LibVEX; rerun with -h for copyright info
   ==13719== Command: ./heat_and_water_meter
   ==13719==

   valgrind:  Fatal error at startup: a function redirection
   valgrind:  which is mandatory for this platform-tool combination
   valgrind:  cannot be set up.  Details of the redirection are:
   valgrind:
   valgrind:  A must-be-redirected function
   valgrind:  whose name matches the pattern:      memcpy
   valgrind:  in an object with soname matching:   ld-linux.so.3
   valgrind:  was not found whilst processing
   valgrind:  symbols from the object with soname: ld-linux.so.3
   valgrind:
   valgrind:  Possible fixes: (1, short term): install glibc's debuginfo
   valgrind:  package on this machine.  (2, longer term): ask the packagers
   valgrind:  for your Linux distribution to please in future ship a non-
   valgrind:  stripped ld.so (or whatever the dynamic linker .so is called)
   valgrind:  that exports the above-named function using the standard
   valgrind:  calling conventions for this platform.  The package you need
   valgrind:  to install for fix (1) is called
   valgrind:
   valgrind:    On Debian, Ubuntu:                 libc6-dbg
   valgrind:    On SuSE, openSuSE, Fedora, RHEL:   glibc-debuginfo
   valgrind:
   valgrind:  Cannot continue -- exiting now.  Sorry.


I've installed libc6-dbg as it says.  Same problem.

While installing libc6-dbg, opkg complains like this:

   bash-dbg: unsatisfied recommendation for ncurses-libtinfo-dbg
   libc6-dbg: unsatisfied recommendation for libsegfault-dbg
   libc6-dbg: unsatisfied recommendation for eglibc-thread-db-dbg
   libc6-dbg: unsatisfied recommendation for eglibc-extra-nss-dbg
   libc6-dbg: unsatisfied recommendation for libcidn-dbg

But trying 'opkg install eglibc-thread-db-dbg' says its an unknown
package.

Any clues?  Am I missing some other step not mentioned here?

Thanks,
Britton

Re: [Valgrind-users] Fwd: "__malloc_initialize_hook is deprecated"&&co. warnings while building ast-open.2013-04-22 ...

[Philippe W. <phi...@sk...>]

On Thu, 2013-05-16 at 22:14 +0200, Roland Mainz wrote:
> On Thu, May 16, 2013 at 9:13 PM, Philippe Waroquiers
> > However, the impact of this part is not as easy.
> 
> I know... I studied the valgrind code in the meantime and started to
> gnaw my fingernails off... ;-/
> 
> > This implies to change the basic way the malloc interception is done,
> > by adding an additional "grouping" parameter, and store this in each
> > memory "chunk" managed by memcheck.
> > More impact on memory, and on the interface between the core and
> > the tools replacing the malloc, and a lot more difficult to make
> > "generic".
> > I suspect this will imply also some possibly heavy changes to
> > the "core" redirection logic.
> 
> Right... but some day it needs to be done... ;-/
> ... for AFAIK (at least) two reasons:
For sure, the below are valid reasons. But there is a balance between
the additional complexity/cpu/memory in Valgrind versus the possible
use.
Valgrind already provides client requests which allows to instrument
"mempool" libraries to have (some) support of bug detection by memcheck.
Note that this support provides less bug detection capabilities than
the replacement of the "standard" malloc/free/... but it looks
unclear to me how to e.g. properly add redzone, freelist, etc
to mempools. So, it looks unlikely to me that the mempool
support can be made much better than what is provided by the
 *MEMPOOL* client requests in valgrind.h include file.

For what concerns mixing chicken malloc and pterodactylus free:
interesting functionality, but again IMO, the balance cost/benefit
is not clear.

The balance cost/benefit of a generalised --soname-synonyms
looks however better to me. I might take the time to look
at this. This would allow any redirection (e.g. the pthread
redirections for helgrind or drd) to have synonyms, allowing
e.g. these tools to work with static libraries.

Philippe

Re: [Valgrind-users] Fwd: "__malloc_initialize_hook is deprecated"&&co. warnings while building ast-open.2013-04-22 ...

[Roland M. <rol...@nr...>]

On Thu, May 16, 2013 at 9:13 PM, Philippe Waroquiers
<phi...@sk...> wrote:
> On Tue, 2013-05-14 at 04:28 +0200, Roland Mainz wrote:
>> On Thu, Apr 25, 2013 at 1:42 PM, Sebastian Feld
>> <seb...@gm...> wrote:
>> > On Wed, Apr 24, 2013 at 11:10 PM, Roland Mainz <rol...@nr...> wrote:
>> >> On Wed, Apr 24, 2013 at 10:14 PM, Roland Mainz <rol...@nr...> wrote:
>> >>> On Wed, Apr 24, 2013 at 12:45 AM, John Reiser <jr...@bi...> wrote:
>
>> >> $ valgrind  "--allocator-sym-redirect=sh_malloc=malloc,sh_free=free,sh_calloc=calloc"
>> >> ... # would instruct valgrind to take function |sh_malloc()| as an
>> >> alternative |malloc(), |sh_free()| as alternative |free()| version
>> >> etc. etc.
>> >>
>> >> The only issue is that if multiple allocators are active within a
>> >> single process we may need some kind of "grouping" to explain valgrind
>> >> that memory allocated by |sh_malloc()| can not be freed by |tcfree()|
>> >> or |_ast_free()| ... maybe it could be done using '{'- and '}'-pairs,
>> >> e.g. $ valgrind
>> >> "--allocator-sym-redirect={sh_malloc=malloc,sh_free=free,sh_calloc=calloc},{_ast_malloc=malloc,_ast_free=free,_ast_calloc=calloc}"
>> >> ... #
>> >
>> > The idea of (finally!) providing such an option sounds like a very
>> > good idea. Until now the only way to probe python and bash4 via
>> > valgrind is to poke in the valgrind sources (which should never
>> > happen).
> I think it would not be very difficult to extend the command line option
>     --soname-synonyms=syn1=pattern1,syn2=pattern2,... synonym soname
>               specify patterns for function wrapping or replacement.
>               To use a non-libc malloc library that is
>                   in the main exe:  --soname-synonyms=somalloc=NONE
>                   in libxyzzy.so:   --soname-synonyms=somalloc=libxyzzy.so
> to support also to give synonym for the "function" part of a redirection.
> Now that I understand better all this area, it should be relatively
> easy to allow to give synonyms for any (existing) redirection
> (library part or function part).
> In other words, to make -soname-synonyms "generic".

That would be helpfull (in our case it's the AST toolkit which has
it's own utility library called libast.so.1 and to avoid namespace
collisions on old operating systems many functions (including their
|malloc()| version) are prefixed with |_ast|, e.g. |_ast_malloc()|,
|_ast_free()| etc. on demand.

>> > I also think the idea to let valgrind detect mixing of different
>> > allocators is a very valuable feature since this has been a source of
>> > more and more bugs. Usually happens in complex projects with use many
>> > different shared libraries, all with their own memory allocators.
>
> However, the impact of this part is not as easy.

I know... I studied the valgrind code in the meantime and started to
gnaw my fingernails off... ;-/

> This implies to change the basic way the malloc interception is done,
> by adding an additional "grouping" parameter, and store this in each
> memory "chunk" managed by memcheck.
> More impact on memory, and on the interface between the core and
> the tools replacing the malloc, and a lot more difficult to make
> "generic".
> I suspect this will imply also some possibly heavy changes to
> the "core" redirection logic.

Right... but some day it needs to be done... ;-/
... for AFAIK (at least) two reasons:
1. Big applications depending on many utility libraries sooner or
later hit the issue of having multiple |malloc()| versions. For
passing a pointer obtained from |_chicken_malloc()| to
|_pterodactylus_free()| is likely causing some very interesting
trouble if the process doesn't get a SIGSEGV/SIGBUS walloped over the
head on the first try... ;-/
2. Some memory allocators (for example used by commercial C++
libraries... or take a look at Sun/Oracle is doing for the VM2 project
in Solaris 11.1/12) allow "pools" of memory (e.g. you create a pool
and then use the pool as argument to a |malloc()|-like function) where
from each pool memory can be allocated independently (this is usually
done to boost performance (for example each thread owns it's own pool
and doesn't have to use atomic operations unless new "backing store"
needs to be obtained), fight fragmentation, make
dealloc_everything_from_this_pool-in-one-go easier/faster etc.).
Beyond the usual trouble some implementations allow nesting of pools,
e.g. one pool provides the "backing store" for a couple of sub-pools
etc. (for example there is a "global" pool which is thread-safe and
the threads maintain their own pools (without the need for atomic
operations) and use the global pool to obtain the memory)

>> Uhm... was there any feedback yet for that idea ?
>
> Some feedback above :).

Thanks... :-)

----

Bye,
Roland

-- 
  __ .  . __
 (o.\ \/ /.o) rol...@nr...
  \__\/\/__/  MPEG specialist, C&&JAVA&&Sun&&Unix programmer
  /O /==\ O\  TEL +49 641 3992797
 (;O/ \/ \O;)

Re: [Valgrind-users] Fwd: "__malloc_initialize_hook is deprecated"&&co. warnings while building ast-open.2013-04-22 ...

[Philippe W. <phi...@sk...>]

On Tue, 2013-05-14 at 04:28 +0200, Roland Mainz wrote:
> On Thu, Apr 25, 2013 at 1:42 PM, Sebastian Feld
> <seb...@gm...> wrote:
> > On Wed, Apr 24, 2013 at 11:10 PM, Roland Mainz <rol...@nr...> wrote:
> >> On Wed, Apr 24, 2013 at 10:14 PM, Roland Mainz <rol...@nr...> wrote:
> >>> On Wed, Apr 24, 2013 at 12:45 AM, John Reiser <jr...@bi...> wrote:

> >> $ valgrind  "--allocator-sym-redirect=sh_malloc=malloc,sh_free=free,sh_calloc=calloc"
> >> ... # would instruct valgrind to take function |sh_malloc()| as an
> >> alternative |malloc(), |sh_free()| as alternative |free()| version
> >> etc. etc.
> >>
> >> The only issue is that if multiple allocators are active within a
> >> single process we may need some kind of "grouping" to explain valgrind
> >> that memory allocated by |sh_malloc()| can not be freed by |tcfree()|
> >> or |_ast_free()| ... maybe it could be done using '{'- and '}'-pairs,
> >> e.g. $ valgrind
> >> "--allocator-sym-redirect={sh_malloc=malloc,sh_free=free,sh_calloc=calloc},{_ast_malloc=malloc,_ast_free=free,_ast_calloc=calloc}"
> >> ... #
> >
> > The idea of (finally!) providing such an option sounds like a very
> > good idea. Until now the only way to probe python and bash4 via
> > valgrind is to poke in the valgrind sources (which should never
> > happen).
I think it would not be very difficult to extend the command line option
    --soname-synonyms=syn1=pattern1,syn2=pattern2,... synonym soname
              specify patterns for function wrapping or replacement.
              To use a non-libc malloc library that is
                  in the main exe:  --soname-synonyms=somalloc=NONE
                  in libxyzzy.so:   --soname-synonyms=somalloc=libxyzzy.so
to support also to give synonym for the "function" part of a redirection.
Now that I understand better all this area, it should be relatively
easy to allow to give synonyms for any (existing) redirection
(library part or function part).
In other words, to make -soname-synonyms "generic".


> >
> > I also think the idea to let valgrind detect mixing of different
> > allocators is a very valuable feature since this has been a source of
> > more and more bugs. Usually happens in complex projects with use many
> > different shared libraries, all with their own memory allocators.
However, the impact of this part is not as easy.
This implies to change the basic way the malloc interception is done,
by adding an additional "grouping" parameter, and store this in each
memory "chunk" managed by memcheck.
More impact on memory, and on the interface between the core and
the tools replacing the malloc, and a lot more difficult to make
"generic".
I suspect this will imply also some possibly heavy changes to
the "core" redirection logic.

> Uhm... was there any feedback yet for that idea ?
Some feedback above :).

Philippe

Re: [Valgrind-users] Helgrind data race question

[David F. <fa...@kd...>]

On Tuesday 14 May 2013 20:18:44 Phil Longstaff wrote:
> int* my_ptr = new int;
> *my_ptr = 10;
> pthread_mutex_lock(&lock);
> shared_ptr = my_ptr;
> pthread_mutex_unlock(&lock);
> 
> Thread 2:
> pthread_mutex_lock(&lock);
> int* my_ptr = shared_ptr;
> pthread_mutex_unlock(&lock);
> ... = *my_ptr;

You're reading a region of memory outside mutex protection, and that region of 
memory was written to, outside mutex protection. That's the basic definition 
of a data race.

Getting the address of that region of memory within the mutex doesn't change 
that.

You see it as non-racy because "how could *my_ptr ever be something else than 
10" ... but if you think about a multi-processor system, the write of the 
value 10 might not get propagated to the cache of the other processor where 
the read happens, since the system had no reason to perform that 
synchronisation.

-- 
David Faure, fa...@kd..., http://www.davidfaure.fr
Working on KDE, in particular KDE Frameworks 5

[Valgrind-users] Helgrind data race question

[Phil L. <plo...@sa...>]

According to the helgrind manual "When a mutex is unlocked by thread T1 and later (or immediately) locked by thread T2, then the memory accesses in T1 prior to the unlock must happen-before those in T2 after it acquires the lock".

Two possible ways to interpret this:

Scenario #1:
int shared_value;
pthread_mutex_t lock;

Thread 1:

pthread_mutex_lock(&lock);
shared_value = 10;
pthread_mutex_unlock(&lock);

Thread 2:

pthread_mutex_lock(&lock);
... = shared_value;
pthread_mutex_unlock(&lock);

For this, no data race on shared_value should be reported;

Scenario #2:

pthread_mutex_t lock;
int* shared_ptr;

Thread 1:

int* my_ptr = new int;
*my_ptr = 10;
pthread_mutex_lock(&lock);
shared_ptr = my_ptr;
pthread_mutex_unlock(&lock);

Thread 2:
pthread_mutex_lock(&lock);
int* my_ptr = shared_ptr;
pthread_mutex_unlock(&lock);
... = *my_ptr;

In this case, there should be no data race on shared_ptr, similar to scenario #1.  Will helgrind report a data race on the memory shared_ptr points to, or is memory only deemed to be safe while a semaphore is locked.

Phil
-----
Phil Longstaff
Senior Software Engineer
x2904

Re: [Valgrind-users] Fwd: "__malloc_initialize_hook is deprecated"&&co. warnings while building ast-open.2013-04-22 ...

[Roland M. <rol...@nr...>]

On Thu, Apr 25, 2013 at 1:42 PM, Sebastian Feld
<seb...@gm...> wrote:
> On Wed, Apr 24, 2013 at 11:10 PM, Roland Mainz <rol...@nr...> wrote:
>> On Wed, Apr 24, 2013 at 10:14 PM, Roland Mainz <rol...@nr...> wrote:
>>> On Wed, Apr 24, 2013 at 12:45 AM, John Reiser <jr...@bi...> wrote:
>>>>> Does valgrind provide any replacements for glibc's
>>>>> |__malloc_initialize_hook()| ? It seems this call and it's |*hook*()|
>>>>> siblings are depreciated now (at least in SuSE >=12.3) ...
>>>>
>>>> There is no glibc replacement.  [And the reasoning is correct.]
>>>> There is no valgrind replacement.
>>>> You must change your basic approach.
>>>>
>>>> We went through this just 6 months ago.
>>>> Check the archives of this mailing list:
>>>>
>>>>    [Valgrind-users] __malloc_hook
>>>>    Amir Szekely  <ki...@gm...>
>>>>    10/19/2012
>>>>
>>>> That thread contains code that works.
>>>> [The modification to detect the first use is obvious.]
>>>
>>> Grumpf... I tried that... but the combination how the stuff I'd like
>>> to instrument+debug is build+used makes that solution more or less
>>> impossible (for example... the allocator system lives in a seperate
>>> namespace, e.g. it has |malloc()|&&|free()| etc. but all symbols are
>>> prefixed with |_ast|, e.g. |_ast_malloc()|, |_ast_free()| etc.).
>>>
>>> I tried to work around the issues with the API provided in
>>> <valgrind/valgrind.h> ... but it seems this doesn't detect any
>>> read-from-unallocated etc. or even the plain double-free situations
>>> (patch below) ... erm... is the API around
>>> |VALGRIND_MALLOCLIKE_BLOCK()| know to work in valgrind-3.8.1 ?
>>>
>>> -- snip --
>>> --- src/lib/libast/vmalloc/vmbest.c      2012-06-28 22:12:14.000000000 +0200
>>> +++ src/lib/libast/vmalloc/vmbest.c     2013-04-24 03:03:44.207373019 +0200
>>> @@ -10,40 +10,42 @@
>>>  *          http://www.eclipse.org/org/documents/epl-v10.html           *
>>>  *         (with md5 checksum b35adb5213ca9657e911e9befb180842)         *
>>>  *                                                                      *
>>>  *              Information and Software Systems Research               *
>>>  *                            AT&T Research                             *
>>>  *                           Florham Park NJ                            *
>>>  *                                                                      *
>>>  *                 Glenn Fowler <gs...@re...>                  *
>>>  *                  David Korn <dg...@re...>                   *
>>>  *                   Phong Vo <kp...@re...>                    *
>>>  *                                                                      *
>>>  ***********************************************************************/
>>>  #if defined(_UWIN) && defined(_BLD_ast)
>>>
>>>  void _STUB_vmbest(){}
>>>
>>>  #else
>>>
>>>  #include       "vmhdr.h"
>>>
>>> +#include <valgrind/valgrind.h>
>>> +
>>>  /*     Best-fit allocation method. This is based on a best-fit strategy
>>>  **     using a splay tree for storage of lists of free blocks of the same
>>>  **     size. Recent free blocks may be cached for fast reuse.
>>>  **
>>>  **     Written by Kiem-Phong Vo, kp...@re..., 01/16/94.
>>>  */
>>>
>>>  #ifdef DEBUG
>>>  static int     N_free;         /* # of free calls                      */
>>>  static int     N_alloc;        /* # of alloc calls                     */
>>>  static int     N_resize;       /* # of resize calls                    */
>>>  static int     N_wild;         /* # allocated from the wild block      */
>>>  static int     N_last;         /* # allocated from last free block     */
>>>  static int     N_reclaim;      /* # of bestreclaim calls               */
>>>  #endif /*DEBUG*/
>>>
>>>  #define COMPACT                8       /* factor to decide when to
>>> compact     */
>>>
>>>  /* Check to see if a block is in the free tree */
>>>  #if __STD_C
>>> @@ -692,41 +694,44 @@
>>>
>>>                         if(VMWILD(vd,np))
>>>                         {       SIZE(np) &= ~BITS;
>>>                                 SELF(np) = np;
>>>                                 ap = NEXT(np); /**/ASSERT(ISBUSY(SIZE(ap)));
>>>                                 SETPFREE(SIZE(ap));
>>>                                 vd->wild = np;
>>>                         }
>>>                         else    vd->free = np;
>>>                 }
>>>
>>>                 SETBUSY(SIZE(tp));
>>>         }
>>>
>>>  done:
>>>         if(tp && !local && (vd->mode&VM_TRACE) && _Vmtrace &&
>>> VMETHOD(vd) == VM_MTBEST)
>>>                 (*_Vmtrace)(vm,NIL(Vmuchar_t*),(Vmuchar_t*)DATA(tp),orgsize,0);
>>>
>>>         CLRLOCK(vm,local); /**/ASSERT(_vmbestcheck(vd, NIL(Block_t*)) == 0);
>>>
>>> -       return tp ? DATA(tp) : NIL(Void_t*);
>>> +       void *res= tp ? DATA(tp) : NIL(Void_t*);
>>> +       if (!local)
>>> +               VALGRIND_MALLOCLIKE_BLOCK(res, size, 0, 0);
>>> +       return res;
>>>  }
>>>
>>>  #if __STD_C
>>>  static long bestaddr(Vmalloc_t* vm, Void_t* addr, int local )
>>>  #else
>>>  static long bestaddr(vm, addr, local)
>>>  Vmalloc_t*     vm;     /* region allocating from       */
>>>  Void_t*                addr;   /* address to check             */
>>>  int            local;
>>>  #endif
>>>  {
>>>         reg Seg_t*      seg;
>>>         reg Block_t     *b, *endb;
>>>         reg long        offset;
>>>         reg Vmdata_t*   vd = vm->data;
>>>
>>>         /**/ASSERT(local ? (vd->lock == 1) : 1 );
>>>         SETLOCK(vm, local);
>>>
>>>         offset = -1L; b = endb = NIL(Block_t*);
>>> @@ -816,40 +821,43 @@
>>>                         vd->free = bp;
>>>                 else
>>>                 {       /**/ASSERT(!vmonlist(CACHE(vd)[S_CACHE], bp) );
>>>                         LINK(bp) = CACHE(vd)[S_CACHE];
>>>                         CACHE(vd)[S_CACHE] = bp;
>>>                 }
>>>
>>>                 /* coalesce on freeing large blocks to avoid fragmentation */
>>>                 if(SIZE(bp) >= 2*vd->incr)
>>>                 {       bestreclaim(vd,NIL(Block_t*),0);
>>>                         if(vd->wild && SIZE(vd->wild) >= COMPACT*vd->incr)
>>>                                 KPVCOMPACT(vm,bestcompact);
>>>                 }
>>>         }
>>>
>>>         if(!local && _Vmtrace && (vd->mode&VM_TRACE) && VMETHOD(vd) ==
>>> VM_MTBEST )
>>>                 (*_Vmtrace)(vm,(Vmuchar_t*)data,NIL(Vmuchar_t*), (s&~BITS), 0);
>>>
>>>         CLRLOCK(vm, local); /**/ASSERT(_vmbestcheck(vd, NIL(Block_t*)) == 0);
>>>
>>> +       if (!local)
>>> +               VALGRIND_FREELIKE_BLOCK(data, 0);
>>> +
>>>         return 0;
>>>  }
>>>
>>>  #if __STD_C
>>>  static Void_t* bestresize(Vmalloc_t* vm, Void_t* data, reg size_t
>>> size, int type, int local)
>>>  #else
>>>  static Void_t* bestresize(vm, data, size, type, local)
>>>  Vmalloc_t*     vm;             /* region allocating from       */
>>>  Void_t*                data;           /* old block of data            */
>>>  reg size_t     size;           /* new size                     */
>>>  int            type;           /* !=0 to move, <0 for not copy */
>>>  int            local;
>>>  #endif
>>>  {
>>>         reg Block_t     *rp, *np, *t;
>>>         size_t          s, bs;
>>>         size_t          oldz = 0,  orgsize = size;
>>>         Void_t          *oldd = 0, *orgdata = data;
>>>         Vmdata_t        *vd = vm->data;
>>>
>>> @@ -936,40 +944,46 @@
>>>                         {       if(type&VM_RSCOPY)
>>>                                         memcpy(data, oldd, bs);
>>>
>>>                         do_free: /* reclaim these right away */
>>>                                 SETJUNK(SIZE(rp));
>>>                                 LINK(rp) = CACHE(vd)[S_CACHE];
>>>                                 CACHE(vd)[S_CACHE] = rp;
>>>                                 bestreclaim(vd, NIL(Block_t*), S_CACHE);
>>>                         }
>>>                 }
>>>         }
>>>
>>>         if(data && (type&VM_RSZERO) && (size = SIZE(BLOCK(data))&~BITS) > oldz )
>>>                 memset((Void_t*)((Vmuchar_t*)data + oldz), 0, size-oldz);
>>>
>>>         if(!local && _Vmtrace && data && (vd->mode&VM_TRACE) &&
>>> VMETHOD(vd) == VM_MTBEST)
>>>                 (*_Vmtrace)(vm, (Vmuchar_t*)orgdata, (Vmuchar_t*)data,
>>> orgsize, 0);
>>>
>>>         CLRLOCK(vm, local); /**/ASSERT(_vmbestcheck(vd, NIL(Block_t*)) == 0);
>>>
>>> +       if (!local)
>>> +       {
>>> +               VALGRIND_FREELIKE_BLOCK(orgdata, 0);
>>> +               VALGRIND_MALLOCLIKE_BLOCK(data, size, 0, 0);
>>> +       }
>>> +
>>>         return data;
>>>  }
>>>
>>>  #if __STD_C
>>>  static long bestsize(Vmalloc_t* vm, Void_t* addr, int local )
>>>  #else
>>>  static long bestsize(vm, addr, local)
>>>  Vmalloc_t*     vm;     /* region allocating from       */
>>>  Void_t*                addr;   /* address to check             */
>>>  int            local;
>>>  #endif
>>>  {
>>>         Seg_t           *seg;
>>>         Block_t         *b, *endb;
>>>         long            size;
>>>         Vmdata_t        *vd = vm->data;
>>>
>>>         SETLOCK(vm, local);
>>>
>>>         size = -1L;
>>> -- snip --
>>
>> ... aaand more digging: I found
>> http://code.google.com/p/valgrind-variant/source/browse/trunk/valgrind/coregrind/m_replacemalloc/vg_replace_malloc.c#1175
>> which seems to be from one of the valgrind forks... what about
>> talking-up that idea and provide a command line option called
>> --allocator-sym-redirect which works like passing down a small list of
>> symbol mappings to instruct valgrind that it should monitor some extra
>> allocators.
>>
>> Example:
>> $ valgrind  "--allocator-sym-redirect=sh_malloc=malloc,sh_free=free,sh_calloc=calloc"
>> ... # would instruct valgrind to take function |sh_malloc()| as an
>> alternative |malloc(), |sh_free()| as alternative |free()| version
>> etc. etc.
>>
>> The only issue is that if multiple allocators are active within a
>> single process we may need some kind of "grouping" to explain valgrind
>> that memory allocated by |sh_malloc()| can not be freed by |tcfree()|
>> or |_ast_free()| ... maybe it could be done using '{'- and '}'-pairs,
>> e.g. $ valgrind
>> "--allocator-sym-redirect={sh_malloc=malloc,sh_free=free,sh_calloc=calloc},{_ast_malloc=malloc,_ast_free=free,_ast_calloc=calloc}"
>> ... #
>
> The idea of (finally!) providing such an option sounds like a very
> good idea. Until now the only way to probe python and bash4 via
> valgrind is to poke in the valgrind sources (which should never
> happen).
>
> I also think the idea to let valgrind detect mixing of different
> allocators is a very valuable feature since this has been a source of
> more and more bugs. Usually happens in complex projects with use many
> different shared libraries, all with their own memory allocators.

Uhm... was there any feedback yet for that idea ?

----

Bye,
Roland

-- 
  __ .  . __
 (o.\ \/ /.o) rol...@nr...
  \__\/\/__/  MPEG specialist, C&&JAVA&&Sun&&Unix programmer
  /O /==\ O\  TEL +49 641 3992797
 (;O/ \/ \O;)

Re: [Valgrind-users] Core dump trying to run my application under valgrind

[Patrick J. L. <lop...@gm...>]

On Mon, May 13, 2013 at 6:46 AM, Gonzalo <gon...@gm...> wrote:

> ==8527== Process terminating with default action of signal 11 (SIGSEGV)
> ==8527==  Access not within mapped region at address 0xBEF3EDA4
> ==8527==    at 0x8091CD0: WriteFiasco(std::string const&) (fiasco_finder.h:28)
> ==8527==    by 0x43F810D: __static_initialization_and_destruction_0(int, int) (exceptions.cpp:2)
> ==8527==    by 0x43F817D: global constructors keyed to exceptions.cpp (exceptions.cpp:122)

...

> inline bool WriteFiasco(const std::string& fileName)
> {
>     static int counter = 0;
>     ++counter;
>
>     std::ofstream file;
>     file.open("FiascoFinder.txt", std::ios::out | std::ios::app);
>     file << "Starting to initialize file - number: [" << counter << "] filename: [" << fileName.c_str() << "]" << std::endl;
>     file.flush();
>     file.close();
>     return true;                                  <----- line 28
> }

Looks like a crash in the std::ofstream destructor.  You will need to
disassemble your code near the fault (0x8091CD0) to be sure.

The order in which static global constructors run is unspecified in
C++. I do not know whether this order can be affected by running under
Valgrind, but if so, it could explain your failure. Perhaps some
global static object in the C++ runtime has to be constructed before
std::ofstream will work at all. (I am certain this can happen if you
use, say, std::cout; I am actually a little surprised to see it for
std::ofstream.)

Anyway, it is possible that if you change this code not to use
std::ofstream, and just use read()/write() or fread()/fwrite(), then
you will work around the problem.

Then again this could just be a problem with Valgrind. A disassembly
near the fault is the first step toward finding out.

 - Pat

P.S. As an aside, this is one reason most C++ experts recommend
avoiding global objects when possible and using the "Meyers Singleton"
when not.

[Valgrind-users] Core dump trying to run my application under valgrind

[Gonzalo <gon...@gm...>]

Hi guys,

    I'm having hard times trying to use valgrind with my project. I
regularly use valgrind without problems but in this project I can't.

If I run the application without valgrind, it works ok, but running under
valgrind, it crashs
es.

This is the stack trace:

==8527== Process terminating with default action of signal 11 (SIGSEGV)
==8527==  Access not within mapped region at address 0xBEF3EDA4
*==8527==    at 0x8091CD0: WriteFiasco(std::string const&)
(fiasco_finder.h:28)*
==8527==    by 0x43F810D: __static_initialization_and_destruction_0(int,
int) (*exceptions.cpp:2*)
==8527==    by 0x43F817D: global constructors keyed to exceptions.cpp
(exceptions.cpp:122)
==8527==    by 0x43F82AC: ??? (in
/home/gonzalo/Perforce/gonzalo_6523/Server/branches/StaticOrderFiasco/Common/Debug/libCommon.so)
==8527==    by 0x4347197: ??? (in
/home/gonzalo/Perforce/gonzalo_6523/Server/branches/StaticOrderFiasco/Common/Debug/libCommon.so)
==8527==    by 0x400DF0B: call_init (dl-init.c:70)
==8527==    by 0x400E028: _dl_init (dl-init.c:134)
==8527==    by 0x400088E: ??? (in /lib/ld-2.11.3.so)
==8527==  If you believe this happened as a result of a stack
==8527==  overflow in your program's main thread (unlikely but
==8527==  possible), you can try to increase the size of the
==8527==  main thread stack using the --main-stacksize= flag.
==8527==  The main thread stack size used in this run was 8388608.



and this is the function fiasco_finder.h::WriteFiasco:

inline bool WriteFiasco(const std::string& fileName)
{
    static int counter = 0;
    ++counter;

    std::ofstream file;
    file.open("FiascoFinder.txt", std::ios::out | std::ios::app);
    file << "Starting to initialize file - number: [" << counter << "]
filename: [" << fileName.c_str() << "]" << std::endl;
    file.flush();
    file.close();
    return true;                                  *<----- line 28*
}

#define *FIASCO_FINDER* const bool g_psuedoUniqueName =
WriteFiasco(__FILE__);



and this is exceptions.cpp:2

#include "fiasco_finder.h"
*FIASCO_FINDER                             <--------- line 2*
#include "exceptions.h"

I don't really know what is happenning. Did someone have the same problem
or a similar one?. I'm kind of lost and out of ideas.

Thanks a lot!
Gonzalo

Re: [Valgrind-users] Problem using Valgrind with libc6-dbg

[Tom H. <to...@co...>]

On 13/05/13 13:59, Michael R. Hunter wrote:

> --20396--   Considering /lib/ld-2.15.so ..
> --20396--   .. CRC mismatch (computed 405891ab wanted 9a1d2f37)
> --20396--    object doesn't have a symbol table

This is your problem.

> Can anyone explain why Valgrind is not finding the debug symbols?

It is finding them, they just don't match the library they claim to be 
associated with.

Most likely the version of the debug package does not match the version 
of the main package.

Tom

-- 
Tom Hughes (to...@co...)
http://compton.nu/

[Valgrind-users] Problem using Valgrind with libc6-dbg

[Michael R. H. <ma...@us...>]

Hello

I am running Precise Puppy 5.4.2, which is built from Ubuntu Precise  
Pangolin 12.04.1+ binary DEB packages, hence has binary compatibility  
with Ubuntu and access to the vast Ubuntu package repository.

I have installed the Valgrind 3.7.0 package together with the  
libc6-dbg package from the Ubuntu package repository.

I ran the command: valgrind -v ls -l and I get:

==20396== Memcheck, a memory error detector
==20396== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al.
==20396== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info
==20396== Command: ls -l
==20396==
--20396-- Valgrind options:
--20396--    --suppressions=/usr/lib/valgrind/debian-libc6-dbg.supp
--20396--    -v
--20396-- Contents of /proc/version:
--20396--   Linux version 3.2.29 (root@puppypc6993) (gcc version 4.6.3  
(Ubuntu/Linaro 4.6.3-1ubuntu5) ) #1 SMP Thu Sep 13 20:33:02 GMT-8 2012
--20396-- Arch and hwcaps: X86, x86-sse1-sse2
--20396-- Page sizes: currently 4096, max supported 4096
--20396-- Valgrind library directory: /usr/lib/valgrind
--20396-- Reading syms from /lib/ld-2.15.so (0x4000000)
--20396--   Considering /lib/ld-2.15.so ..
--20396--   .. CRC mismatch (computed 405891ab wanted 9a1d2f37)
--20396--    object doesn't have a symbol table

valgrind:  Fatal error at startup: a function redirection
valgrind:  which is mandatory for this platform-tool combination
valgrind:  cannot be set up.  Details of the redirection are:
valgrind:
valgrind:  A must-be-redirected function
valgrind:  whose name matches the pattern:      strlen
valgrind:  in an object with soname matching:   ld-linux.so.2
valgrind:  was not found whilst processing
valgrind:  symbols from the object with soname: ld-linux.so.2
valgrind:
valgrind:  Possible fixes: (1, short term): install glibc's debuginfo
valgrind:  package on this machine.  (2, longer term): ask the packagers
valgrind:  for your Linux distribution to please in future ship a non-
valgrind:  stripped ld.so (or whatever the dynamic linker .so is called)
valgrind:  that exports the above-named function using the standard
valgrind:  calling conventions for this platform.  The package you need
valgrind:  to install for fix (1) is called
valgrind:
valgrind:    On Debian, Ubuntu:                 libc6-dbg
valgrind:    On SuSE, openSuSE, Fedora, RHEL:   glibc-debuginfo
valgrind:
valgrind:  Cannot continue -- exiting now.  Sorry.


The filepath to the debug symbols installed by the 'libc6-dbg' package:

/usr/lib/debug/lib/i386-linux-gnu

Can anyone explain why Valgrind is not finding the debug symbols?

TIA
Michael

Re: [Valgrind-users] Application crashes on MIPS cpu if we valgrind it

[John R. <jr...@bi...>]

> ==3269== Using Valgrind-3.8.1 and LibVEX; rerun with -h for copyright info 
> ==3269== valgrind: Unrecognised instruction at address 0x40c038.
> ==3269==    at 0x40C038: _dl_aux_init (in /tmp/wqf/Install/bin/test)
> ==3269==    by 0x400400: (below main) (in /tmp/wqf/Install/bin/test)

> ==4441== valgrind: Unrecognised instruction at address 0x4016fd8.
> ==4441==    at 0x4016FD8: ??? (in /lib/ld-2.11.1.so)
> ==4441==    by 0x4000E5C: ??? (in /lib/ld-2.11.1.so)

Your ld-2.11.1.so uses some instruction opcode(s) that valgrind (memcheck; VEX)
does not implement yet.  You have also encountered a usability bug in memcheck,
namely memcheck 3.8.1 does not print the actual instruction bytes, and therefore
it is hard for anyone to discover exactly what is necessary to make memcheck work
on your program.  The memcheck bug is https://bugs.kde.org/show_bug.cgi?id=309323.
Please add a comment there "I encountered this bug, also."  Copy+paste those
two error reports above [7 or 8 lines] into your new comment at bug 309323.

Meanwhile, https://bugs.kde.org/show_bug.cgi?id=309323#c5 says that the source
for memcheck has been changed to print the instruction bytes.  However, no release
has been made that incorporates the changes.  So, you could try checking out
the current source tree, then building and running your own local version of memcheck.
If "successful", then at least you will know what opcode is missing from memcheck/VEX.

That's the first half of what needs to be done.  Then you should look up that opcode
in the MIPS documentation, and file another bug against memcheck: "Please implement
opocde 0xYYYYYYYY ("foo") for MIPS-74K."  Give the information about where that
opcode appears: ld-2.11.1, routine _dl_aux_init, etc.

You can also find the opcode by using a disassembler and/or debugger.
Disassemble the 8 instructions (4 before, 4 after) which surround addresses
0x40c038 and 0x4016fd8.  The reported address is the actual address in
the static linked version.  The reported address probably is 0x4000000 higher
than in the shared library /lib/ld-2.11.1.

All that will not solve the problem, but it will identify precisely what the problem _is_.
Then you need to convince the valgrind(VEX) maintainers to implement the missing opcode,
or you need to re-write the code to avoid using that instruction.

-- 

Re: [Valgrind-users] Application crashes on MIPS cpu if we valgrind it

[WANG Q. <Qua...@al...>]

Sorry a wong cpu info was given, now the correct one.

#cat cpuinfo 
system type             : Broadlight Lilac SOC
processor               : 0
cpu model               : MIPS 74Kc V4.12
BogoMIPS                : 248.21
wait instruction        : yes
microsecond timers      : yes
tlb_entries             : 64
extra interrupt vector  : yes
hardware watchpoint     : yes, count: 4, address/irw mask: [0x0000, 0x0000, 0x0000, 0x0000]
ASEs implemented        : mips16 dsp
shadow register sets    : 1
core                    : 0
VCED exceptions         : not available
VCEI exceptions         : not available

-----Original Message-----
From: WANG Quanfu 
Sent: 2013年5月8日 14:42
To: 'Val...@li...'
Subject: Application crashes on MIPS cpu if we valgrind it

Hi,
When we try to use valgrind on our board(cpu-mips, os-linux, clib-glibc), but met with some problem.
Please check if you met such problem before? Any suggestions? 
Thanks!

CPU Info:
#cat /proc/cpuinfo 
Processor       : ARMv7 Processor rev 0 (v7l)
BogoMIPS        : 1297.61
Features        : swp half thumb fastmult edsp 
CPU implementer : 0x41
CPU architecture: 7
CPU variant     : 0x3
CPU part        : 0xc09
CPU revision    : 0

Hardware        : hsan
Revision        : 0000
Serial          : 0000000000000000

GCC Info:
mips_74k_softfp-target-linux-gnu-gcc --verbose Using built-in specs.
Target: mips-wrs-linux-gnu
Configured with: /scratch/joseph/wrs/4.4a/src/gcc-4.4-wrs/configure --build=i686-pc-linux-gnu --host=i686-pc-linux-gnu --target=mips-wrs-linux-gnu --enable-threads --disable-libmudflap --disable-libssp --disable-libstdcxx-pch --with-float=soft --with-mips-plt --with-arch-32=4kc --with-arch-64=5kf --with-gnu-as --with-gnu-ld --with-specs='%{save-temps: -fverbose-asm} %{O2:%{!fno-remove-local-statics: -fremove-local-statics}} %{O*:%{O|O0|O1|O2|Os:;:%{!fno-remove-local-statics: -fremove-local-statics}}}' --enable-languages=c,c++ --enable-shared --disable-lto --enable-symvers=gnu --enable-__cxa_atexit --with-pkgversion='Wind River Linux Sourcery G++ 4.4a-323' --wit...@wi... --disable-nls --prefix=/opt/windriver/wrlinux/mips --with-sysroot=/opt/windriver/wrlinux/mips/mips-wrs-linux-gnu/libc --with-build-sysroot=/scratch/joseph/wrs/4.4a/mips/install/mips-wrs-linux-gnu/libc --with-gmp=/scratch/joseph/wrs/4.4a/mips/obj/host-libs-4.4a-323-mips-wrs-linux-gnu-i686-pc-linux-gnu/usr --with-mpfr=/scratch/joseph/wrs/4.4a/mips/obj/host-libs-4.4a-323-mips-wrs-linux-gnu-i686-pc-linux-gnu/usr --with-ppl=/scratch/joseph/wrs/4.4a/mips/obj/host-libs-4.4a-323-mips-wrs-linux-gnu-i686-pc-linux-gnu/usr --with-host-libstdcxx='-static-libgcc -Wl,-Bstatic,-lstdc++,-Bdynamic -lm' --with-cloog=/scratch/joseph/wrs/4.4a/mips/obj/host-libs-4.4a-323-mips-wrs-linux-gnu-i686-pc-linux-gnu/usr --with-libelf=/scratch/joseph/wrs/4.4a/mips/obj/host-libs-4.4a-323-mips-wrs-linux-gnu-i686-pc-linux-gnu/usr --disable-libgomp --with-license=/scratch/joseph/wrs/4.4a/mips/obj/host-libs-4.4a-323-mips-wrs-linux-gnu-i686-pc-linux-gnu/usr --with-csl-license-version=20101201 --with-csl-license-feature=gcc_MIPS_Wind_River_Linux --enable-poison-system-directories --with-debug-prefix-map='/scratch/joseph/wrs/4.4a/mips/install=/opt/windriver/wrlinux/mips            /scratch/joseph/wrs/4.4a/src/gcc-4.4-wrs=/opt/windriver/wrlinux/mips/mips-wrs-linux-gnu/src/gcc /scratch/joseph/wrs/4.4a/mips/obj/gcc-4.4a-323-mips-wrs-linux-gnu-i686-pc-linux-gnu=/opt/windriver/wrlinux/mips/mips-wrs-linux-gnu/src/generated/gcc' --with-build-time-tools=/scratch/joseph/wrs/4.4a/mips/install/mips-wrs-linux-gnu/bin --with-build-time-tools=/scratch/joseph/wrs/4.4a/mips/install/mips-wrs-linux-gnu/bin
Thread model: posix
gcc version 4.4.1 (Wind River Linux Sourcery G++ 4.4a-323) 


Crash Log1(The application was linked as static):
 ./valgrind ./test
==3269== Memcheck, a memory error detector ==3269== Copyright (C) 2002-2012, and GNU GPL'd, by Julian Seward et al.
==3269== Using Valgrind-3.8.1 and LibVEX; rerun with -h for copyright info ==3269== Command: ./test ==3269== ==3269== valgrind: Unrecognised instruction at address 0x40c038.
==3269==    at 0x40C038: _dl_aux_init (in /tmp/wqf/Install/bin/test)
==3269==    by 0x400400: (below main) (in /tmp/wqf/Install/bin/test)
==3269== Your program just tried to execute an instruction that Valgrind ==3269== did not recognise.  There are two possible reasons for this.
==3269== 1. Your program has a bug and erroneously jumped to a non-code
==3269==    location.  If you are running Memcheck and you just saw a
==3269==    warning about a bad jump, it's probably your program's fault.
==3269== 2. The instruction is legitimate but Valgrind doesn't handle it,
==3269==    i.e. it's Valgrind's fault.  If you think this is the case or
==3269==    you are not sure, please let us know and we'll try to fix it.
==3269== Either way, Valgrind will now raise a SIGILL signal which will ==3269== probably kill your program.
==3269==
==3269== Process terminating with default action of signal 4 (SIGILL) ==3269==  Illegal opcode at address 0x40C038
==3269==    at 0x40C038: _dl_aux_init (in /tmp/wqf/Install/bin/test)
==3269==    by 0x400400: (below main) (in /tmp/wqf/Install/bin/test)
==3269==
==3269== HEAP SUMMARY:
==3269==     in use at exit: 0 bytes in 0 blocks
==3269==   total heap usage: 0 allocs, 0 frees, 0 bytes allocated
==3269==
==3269== All heap blocks were freed -- no leaks are possible ==3269== ==3269== For counts of detected and suppressed errors, rerun with: -v ==3269== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0) Illegal instruction



Crash Log2(The application was linked as dynamic):

./valgrind ../../../a.out
==4441== Memcheck, a memory error detector ==4441== Copyright (C) 2002-2012, and GNU GPL'd, by Julian Seward et al.
==4441== Using Valgrind-3.8.1 and LibVEX; rerun with -h for copyright info ==4441== Command: ../../../a.out ==4441== ==4441== valgrind: Unrecognised instruction at address 0x4016fd8.
==4441==    at 0x4016FD8: ??? (in /lib/ld-2.11.1.so)
==4441==    by 0x4000E5C: ??? (in /lib/ld-2.11.1.so)
==4441== Your program just tried to execute an instruction that Valgrind ==4441== did not recognise.  There are two possible reasons for this.
==4441== 1. Your program has a bug and erroneously jumped to a non-code
==4441==    location.  If you are running Memcheck and you just saw a
==4441==    warning about a bad jump, it's probably your program's fault.
==4441== 2. The instruction is legitimate but Valgrind doesn't handle it,
==4441==    i.e. it's Valgrind's fault.  If you think this is the case or
==4441==    you are not sure, please let us know and we'll try to fix it.
==4441== Either way, Valgrind will now raise a SIGILL signal which will ==4441== probably kill your program.
==4441==
==4441== Process terminating with default action of signal 4 (SIGILL) ==4441==  Illegal opcode at address 0x4016FD8
==4441==    at 0x4016FD8: ??? (in /lib/ld-2.11.1.so)
==4441==    by 0x4000E5C: ??? (in /lib/ld-2.11.1.so)
==4441==
==4441== HEAP SUMMARY:
==4441==     in use at exit: 0 bytes in 0 blocks
==4441==   total heap usage: 0 allocs, 0 frees, 0 bytes allocated
==4441==
==4441== All heap blocks were freed -- no leaks are possible ==4441== ==4441== For counts of detected and suppressed errors, rerun with: -v ==4441== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0) Illegal instruction

[Valgrind-users] Application crashes on MIPS cpu if we valgrind it

[WANG Q. <Qua...@al...>]

Hi,
When we try to use valgrind on our board(cpu-mips, os-linux, clib-glibc), but met with some problem.
Please check if you met such problem before? Any suggestions? 
Thanks!

CPU Info:
#cat /proc/cpuinfo 
Processor       : ARMv7 Processor rev 0 (v7l)
BogoMIPS        : 1297.61
Features        : swp half thumb fastmult edsp 
CPU implementer : 0x41
CPU architecture: 7
CPU variant     : 0x3
CPU part        : 0xc09
CPU revision    : 0

Hardware        : hsan
Revision        : 0000
Serial          : 0000000000000000

GCC Info:
mips_74k_softfp-target-linux-gnu-gcc --verbose
Using built-in specs.
Target: mips-wrs-linux-gnu
Configured with: /scratch/joseph/wrs/4.4a/src/gcc-4.4-wrs/configure --build=i686-pc-linux-gnu --host=i686-pc-linux-gnu --target=mips-wrs-linux-gnu --enable-threads --disable-libmudflap --disable-libssp --disable-libstdcxx-pch --with-float=soft --with-mips-plt --with-arch-32=4kc --with-arch-64=5kf --with-gnu-as --with-gnu-ld --with-specs='%{save-temps: -fverbose-asm} %{O2:%{!fno-remove-local-statics: -fremove-local-statics}} %{O*:%{O|O0|O1|O2|Os:;:%{!fno-remove-local-statics: -fremove-local-statics}}}' --enable-languages=c,c++ --enable-shared --disable-lto --enable-symvers=gnu --enable-__cxa_atexit --with-pkgversion='Wind River Linux Sourcery G++ 4.4a-323' --wit...@wi... --disable-nls --prefix=/opt/windriver/wrlinux/mips --with-sysroot=/opt/windriver/wrlinux/mips/mips-wrs-linux-gnu/libc --with-build-sysroot=/scratch/joseph/wrs/4.4a/mips/install/mips-wrs-linux-gnu/libc --with-gmp=/scratch/joseph/wrs/4.4a/mips/obj/host-libs-4.4a-323-mips-wrs-linux-gnu-i686-pc-linux-gnu/usr --with-mpfr=/scratch/joseph/wrs/4.4a/mips/obj/host-libs-4.4a-323-mips-wrs-linux-gnu-i686-pc-linux-gnu/usr --with-ppl=/scratch/joseph/wrs/4.4a/mips/obj/host-libs-4.4a-323-mips-wrs-linux-gnu-i686-pc-linux-gnu/usr --with-host-libstdcxx='-static-libgcc -Wl,-Bstatic,-lstdc++,-Bdynamic -lm' --with-cloog=/scratch/joseph/wrs/4.4a/mips/obj/host-libs-4.4a-323-mips-wrs-linux-gnu-i686-pc-linux-gnu/usr --with-libelf=/scratch/joseph/wrs/4.4a/mips/obj/host-libs-4.4a-323-mips-wrs-linux-gnu-i686-pc-linux-gnu/usr --disable-libgomp --with-license=/scratch/joseph/wrs/4.4a/mips/obj/host-libs-4.4a-323-mips-wrs-linux-gnu-i686-pc-linux-gnu/usr --with-csl-license-version=20101201 --with-csl-license-feature=gcc_MIPS_Wind_River_Linux --enable-poison-system-directories --with-debug-prefix-map='/scratch/joseph/wrs/4.4a/mips/install=/opt/windriver/wrlinux/mips            /scratch/joseph/wrs/4.4a/src/gcc-4.4-wrs=/opt/windriver/wrlinux/mips/mips-wrs-linux-gnu/src/gcc /scratch/joseph/wrs/4.4a/mips/obj/gcc-4.4a-323-mips-wrs-linux-gnu-i686-pc-linux-gnu=/opt/windriver/wrlinux/mips/mips-wrs-linux-gnu/src/generated/gcc' --with-build-time-tools=/scratch/joseph/wrs/4.4a/mips/install/mips-wrs-linux-gnu/bin --with-build-time-tools=/scratch/joseph/wrs/4.4a/mips/install/mips-wrs-linux-gnu/bin
Thread model: posix
gcc version 4.4.1 (Wind River Linux Sourcery G++ 4.4a-323) 


Crash Log1(The application was linked as static):
 ./valgrind ./test 
==3269== Memcheck, a memory error detector
==3269== Copyright (C) 2002-2012, and GNU GPL'd, by Julian Seward et al.
==3269== Using Valgrind-3.8.1 and LibVEX; rerun with -h for copyright info
==3269== Command: ./test
==3269== 
==3269== valgrind: Unrecognised instruction at address 0x40c038.
==3269==    at 0x40C038: _dl_aux_init (in /tmp/wqf/Install/bin/test)
==3269==    by 0x400400: (below main) (in /tmp/wqf/Install/bin/test)
==3269== Your program just tried to execute an instruction that Valgrind
==3269== did not recognise.  There are two possible reasons for this.
==3269== 1. Your program has a bug and erroneously jumped to a non-code
==3269==    location.  If you are running Memcheck and you just saw a
==3269==    warning about a bad jump, it's probably your program's fault.
==3269== 2. The instruction is legitimate but Valgrind doesn't handle it,
==3269==    i.e. it's Valgrind's fault.  If you think this is the case or
==3269==    you are not sure, please let us know and we'll try to fix it.
==3269== Either way, Valgrind will now raise a SIGILL signal which will
==3269== probably kill your program.
==3269== 
==3269== Process terminating with default action of signal 4 (SIGILL)
==3269==  Illegal opcode at address 0x40C038
==3269==    at 0x40C038: _dl_aux_init (in /tmp/wqf/Install/bin/test)
==3269==    by 0x400400: (below main) (in /tmp/wqf/Install/bin/test)
==3269== 
==3269== HEAP SUMMARY:
==3269==     in use at exit: 0 bytes in 0 blocks
==3269==   total heap usage: 0 allocs, 0 frees, 0 bytes allocated
==3269== 
==3269== All heap blocks were freed -- no leaks are possible
==3269== 
==3269== For counts of detected and suppressed errors, rerun with: -v
==3269== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
Illegal instruction



Crash Log2(The application was linked as dynamic):

./valgrind ../../../a.out 
==4441== Memcheck, a memory error detector
==4441== Copyright (C) 2002-2012, and GNU GPL'd, by Julian Seward et al.
==4441== Using Valgrind-3.8.1 and LibVEX; rerun with -h for copyright info
==4441== Command: ../../../a.out
==4441== 
==4441== valgrind: Unrecognised instruction at address 0x4016fd8.
==4441==    at 0x4016FD8: ??? (in /lib/ld-2.11.1.so)
==4441==    by 0x4000E5C: ??? (in /lib/ld-2.11.1.so)
==4441== Your program just tried to execute an instruction that Valgrind
==4441== did not recognise.  There are two possible reasons for this.
==4441== 1. Your program has a bug and erroneously jumped to a non-code
==4441==    location.  If you are running Memcheck and you just saw a
==4441==    warning about a bad jump, it's probably your program's fault.
==4441== 2. The instruction is legitimate but Valgrind doesn't handle it,
==4441==    i.e. it's Valgrind's fault.  If you think this is the case or
==4441==    you are not sure, please let us know and we'll try to fix it.
==4441== Either way, Valgrind will now raise a SIGILL signal which will
==4441== probably kill your program.
==4441== 
==4441== Process terminating with default action of signal 4 (SIGILL)
==4441==  Illegal opcode at address 0x4016FD8
==4441==    at 0x4016FD8: ??? (in /lib/ld-2.11.1.so)
==4441==    by 0x4000E5C: ??? (in /lib/ld-2.11.1.so)
==4441== 
==4441== HEAP SUMMARY:
==4441==     in use at exit: 0 bytes in 0 blocks
==4441==   total heap usage: 0 allocs, 0 frees, 0 bytes allocated
==4441== 
==4441== All heap blocks were freed -- no leaks are possible
==4441== 
==4441== For counts of detected and suppressed errors, rerun with: -v
==4441== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
Illegal instruction

Re: [Valgrind-users] Valgrind on custom stack, without libc and signal delivering

[Philippe W. <phi...@sk...>]

On Sun, 2013-05-05 at 18:20 +0400, Anton Kozlov wrote:

> 
> So, the question is, what mechanism can be used to make bare version
> act     like libc one? I've tried to do STACK_REGISTER, but it brought
> no success. 
Increase the size of the stack. If you put 0x10000 instead of 0x1000,
then both versions are working (with or without libc).

>From what I understood debugging with GDB+vgdb, the difference of
behaviour is due to the different way the char stack[STACK_SZ];
is mapped, and the value of the stack ptr when SIGCHLD is received.
It looks like the char stack array is partially located in a "file" rw
mapping, and partially in a "anon" mapping.
When the signal is received, if the sigframe will overlap with the
part of stack in the file mapping, then Valgrind will believe it
has to grow the stack. With the libc version, the sigframe can be
fully in the anon segment, not with the "bare" version.
(I think the sigframe is about 1800 bytes).

(I was amazed to see that one single array can be mapped in two
different segments). But the below seems to indicate that quite
clearly:

bare
**3740** stack: id=1, begin=0x80493A0, end=0x804A3A0
--3740:0:aspacem    1:      0004000000-0008047fff     64m
--3740:0:aspacem    2: file 0008048000-0008048fff    4096 r-xT- d=0xfd00 i=705272  o=0       (1)
--3740:0:aspacem    3: file 0008049000-0008049fff    4096 rw--- d=0xfd00 i=705272  o=0       (1)
--3740:0:aspacem    4: anon 000804a000-000804afff    4096 rw---                                 
--3740:0:aspacem    5: anon 000804b000-000804bfff    4096 rwx--
--3740:0:aspacem    6: RSVN 000804c000-000884afff 8384512 ----- SmLower
(gdb) p $sp /// when SIGCHLD rcvd
$1 = (void *) 0x804a368 <stack+4040>

with libc:
**3721** stack: id=1, begin=0x8049860, end=0x804A860
--3721:0:aspacem   18:      0004028000-0008047fff     64m
--3721:0:aspacem   19: file 0008048000-0008048fff    4096 r-xT- d=0xfd00 i=705463  o=0       (1)
--3721:0:aspacem   20: file 0008049000-0008049fff    4096 rw--- d=0xfd00 i=705463  o=0       (1)
--3721:0:aspacem   21: anon 000804a000-000804afff    4096 rw---                                 
--3721:0:aspacem   22: anon 000804b000-000804bfff    4096 rwx--
--3721:0:aspacem   23: RSVN 000804c000-000884afff 8384512 ----- SmLower
--3721:0:aspacem   24:      000884b000-0037ffffff    759m
(gdb) p $sp /// when SIGCHLD rcvd
$1 = (void *) 0x804a854 <stack+4084>

If you slightly extend the stack needed with the libc version, then you get the
same behaviour:
==3894== Can't extend stack to 0x8049fa0 during signal delivery for thread 1:
==3894==   no stack segment
(I have introduced a function
void bpause()
{
  char truc[400];
  pause();
}

and calls bpause instead of pause in main.

Philippe

[Valgrind-users] Valgrind on custom stack, without libc and signal delivering

[Anton K. <dra...@gm...>]

Hi!

I'm doing a port of some custom OS to usermode (x86/linux), just as
usermode linux, with one of goals to use valgrind on it. To make some
things simplier and explicit, it doesn't link with host's libc at now, but
have some minimal subset (of libc) builtin.

I've faced a problem when running on valgrind, stack is not original one
and signal arrived: " Can't extend stack during signal delivery for
thread", and it segfaults. What's strange, by simple run or gdb it perfrom
normally.

I reproduced it with small program. I also noticied, when program is linked
with libc, everything goes ok.

Example is attached. It's like so:
$ make # build two versions, with libc and bare one
$ make TARGET=valgrind_test vg # run bare version, it will sleeps until
killed with SIGCHLD (17). On signal it segfaults
$ gbd valgrind_test
(gdb) r
# on SIGCHILD it exits normally
$ make TARGET=valgrind_test_libc vg # run libc version, sleeps until
SIGCHLD and exit ok.

I also attach logs of my run. Valgrind with more debugging info was used
(patch attached)

So, the question is, what mechanism can be used to make bare version act
  like libc one? I've tried to do STACK_REGISTER, but it brought no
success.

I've tried valgrind 3.8.1 and svn r13380, all the same.

With best regards,
Anton

[Valgrind-users] Getting target address of LDle

[Mohammad AK Q. <mqu...@pu...>]

Hi everyone,

is there any way to get the address of LDle?

For example, I have: t15 = LDle:I8(t11)

The problem is that I am not sure how to write a helper function that can extract the value of t11.

If you can provide pseudo-code with your answer then that would be great

Thanks

Mohammad

[Valgrind-users] valgrind-3.8.x cannot build with glibc-2.17

[vijay n. <vi...@gm...>]

Hi,

I was trying to compile valgrind-3.8.1 with the latest glibc-2.17 and I was
getting the following error

"Valgrind requires glibc version 2.2 - 2.16".

I made some changes in configure script and I was able to compile it
successfully. Is valgrind-3.8.1 not recommended for glibc-2.17 ?

Below is the patch for the same.

--- configure.in    2012-09-19 00:47:32.000000000 +0530
+++ configure.in.a    2013-05-03 01:39:05.561641058 +0530
@@ -906,6 +906,14 @@
     DEFAULT_SUPP="glibc-2.34567-NPTL-helgrind.supp ${DEFAULT_SUPP}"
     DEFAULT_SUPP="glibc-2.X-drd.supp ${DEFAULT_SUPP}"
     ;;
+     2.17)
+    AC_MSG_RESULT(2.17 family)
+    AC_DEFINE([GLIBC_2_17], 1, [Define to 1 if you're using glibc 2.17.x])
+    DEFAULT_SUPP="glibc-2.X.supp ${DEFAULT_SUPP}"
+    DEFAULT_SUPP="glibc-2.34567-NPTL-helgrind.supp ${DEFAULT_SUPP}"
+    DEFAULT_SUPP="glibc-2.X-drd.supp ${DEFAULT_SUPP}"
+    ;;
+
      darwin)
     AC_MSG_RESULT(Darwin)
     AC_DEFINE([DARWIN_LIBC], 1, [Define to 1 if you're using Darwin])
@@ -919,7 +927,7 @@

      *)
     AC_MSG_RESULT([unsupported version ${GLIBC_VERSION}])
-    AC_MSG_ERROR([Valgrind requires glibc version 2.2 - 2.16])
+    AC_MSG_ERROR([Valgrind requires glibc version 2.2 - 2.17])
     AC_MSG_ERROR([or Darwin libc])
     ;;
 esac
--- configure    2012-09-19 00:49:23.000000000 +0530
+++ configure.a    2013-05-03 01:40:16.977641212 +0530
@@ -6604,6 +6604,17 @@
     DEFAULT_SUPP="glibc-2.34567-NPTL-helgrind.supp ${DEFAULT_SUPP}"
     DEFAULT_SUPP="glibc-2.X-drd.supp ${DEFAULT_SUPP}"
     ;;
+     2.17)
+    { $as_echo "$as_me:${as_lineno-$LINENO}: result: 2.17 family" >&5
+$as_echo "2.17 family" >&6; }
+
+$as_echo "#define GLIBC_2_17 1" >>confdefs.h
+
+    DEFAULT_SUPP="glibc-2.X.supp ${DEFAULT_SUPP}"
+    DEFAULT_SUPP="glibc-2.34567-NPTL-helgrind.supp ${DEFAULT_SUPP}"
+    DEFAULT_SUPP="glibc-2.X-drd.supp ${DEFAULT_SUPP}"
+    ;;
+
      darwin)
     { $as_echo "$as_me:${as_lineno-$LINENO}: result: Darwin" >&5
 $as_echo "Darwin" >&6; }

Re: [Valgrind-users] Fwd: "__malloc_initialize_hook is deprecated"&&co. warnings while building ast-open.2013-04-22 ...

[Sebastian F. <seb...@gm...>]

On Wed, Apr 24, 2013 at 11:10 PM, Roland Mainz <rol...@nr...> wrote:
> On Wed, Apr 24, 2013 at 10:14 PM, Roland Mainz <rol...@nr...> wrote:
>> On Wed, Apr 24, 2013 at 12:45 AM, John Reiser <jr...@bi...> wrote:
>>>> Does valgrind provide any replacements for glibc's
>>>> |__malloc_initialize_hook()| ? It seems this call and it's |*hook*()|
>>>> siblings are depreciated now (at least in SuSE >=12.3) ...
>>>
>>> There is no glibc replacement.  [And the reasoning is correct.]
>>> There is no valgrind replacement.
>>> You must change your basic approach.
>>>
>>> We went through this just 6 months ago.
>>> Check the archives of this mailing list:
>>>
>>>    [Valgrind-users] __malloc_hook
>>>    Amir Szekely  <ki...@gm...>
>>>    10/19/2012
>>>
>>> That thread contains code that works.
>>> [The modification to detect the first use is obvious.]
>>
>> Grumpf... I tried that... but the combination how the stuff I'd like
>> to instrument+debug is build+used makes that solution more or less
>> impossible (for example... the allocator system lives in a seperate
>> namespace, e.g. it has |malloc()|&&|free()| etc. but all symbols are
>> prefixed with |_ast|, e.g. |_ast_malloc()|, |_ast_free()| etc.).
>>
>> I tried to work around the issues with the API provided in
>> <valgrind/valgrind.h> ... but it seems this doesn't detect any
>> read-from-unallocated etc. or even the plain double-free situations
>> (patch below) ... erm... is the API around
>> |VALGRIND_MALLOCLIKE_BLOCK()| know to work in valgrind-3.8.1 ?
>>
>> -- snip --
>> --- src/lib/libast/vmalloc/vmbest.c      2012-06-28 22:12:14.000000000 +0200
>> +++ src/lib/libast/vmalloc/vmbest.c     2013-04-24 03:03:44.207373019 +0200
>> @@ -10,40 +10,42 @@
>>  *          http://www.eclipse.org/org/documents/epl-v10.html           *
>>  *         (with md5 checksum b35adb5213ca9657e911e9befb180842)         *
>>  *                                                                      *
>>  *              Information and Software Systems Research               *
>>  *                            AT&T Research                             *
>>  *                           Florham Park NJ                            *
>>  *                                                                      *
>>  *                 Glenn Fowler <gs...@re...>                  *
>>  *                  David Korn <dg...@re...>                   *
>>  *                   Phong Vo <kp...@re...>                    *
>>  *                                                                      *
>>  ***********************************************************************/
>>  #if defined(_UWIN) && defined(_BLD_ast)
>>
>>  void _STUB_vmbest(){}
>>
>>  #else
>>
>>  #include       "vmhdr.h"
>>
>> +#include <valgrind/valgrind.h>
>> +
>>  /*     Best-fit allocation method. This is based on a best-fit strategy
>>  **     using a splay tree for storage of lists of free blocks of the same
>>  **     size. Recent free blocks may be cached for fast reuse.
>>  **
>>  **     Written by Kiem-Phong Vo, kp...@re..., 01/16/94.
>>  */
>>
>>  #ifdef DEBUG
>>  static int     N_free;         /* # of free calls                      */
>>  static int     N_alloc;        /* # of alloc calls                     */
>>  static int     N_resize;       /* # of resize calls                    */
>>  static int     N_wild;         /* # allocated from the wild block      */
>>  static int     N_last;         /* # allocated from last free block     */
>>  static int     N_reclaim;      /* # of bestreclaim calls               */
>>  #endif /*DEBUG*/
>>
>>  #define COMPACT                8       /* factor to decide when to
>> compact     */
>>
>>  /* Check to see if a block is in the free tree */
>>  #if __STD_C
>> @@ -692,41 +694,44 @@
>>
>>                         if(VMWILD(vd,np))
>>                         {       SIZE(np) &= ~BITS;
>>                                 SELF(np) = np;
>>                                 ap = NEXT(np); /**/ASSERT(ISBUSY(SIZE(ap)));
>>                                 SETPFREE(SIZE(ap));
>>                                 vd->wild = np;
>>                         }
>>                         else    vd->free = np;
>>                 }
>>
>>                 SETBUSY(SIZE(tp));
>>         }
>>
>>  done:
>>         if(tp && !local && (vd->mode&VM_TRACE) && _Vmtrace &&
>> VMETHOD(vd) == VM_MTBEST)
>>                 (*_Vmtrace)(vm,NIL(Vmuchar_t*),(Vmuchar_t*)DATA(tp),orgsize,0);
>>
>>         CLRLOCK(vm,local); /**/ASSERT(_vmbestcheck(vd, NIL(Block_t*)) == 0);
>>
>> -       return tp ? DATA(tp) : NIL(Void_t*);
>> +       void *res= tp ? DATA(tp) : NIL(Void_t*);
>> +       if (!local)
>> +               VALGRIND_MALLOCLIKE_BLOCK(res, size, 0, 0);
>> +       return res;
>>  }
>>
>>  #if __STD_C
>>  static long bestaddr(Vmalloc_t* vm, Void_t* addr, int local )
>>  #else
>>  static long bestaddr(vm, addr, local)
>>  Vmalloc_t*     vm;     /* region allocating from       */
>>  Void_t*                addr;   /* address to check             */
>>  int            local;
>>  #endif
>>  {
>>         reg Seg_t*      seg;
>>         reg Block_t     *b, *endb;
>>         reg long        offset;
>>         reg Vmdata_t*   vd = vm->data;
>>
>>         /**/ASSERT(local ? (vd->lock == 1) : 1 );
>>         SETLOCK(vm, local);
>>
>>         offset = -1L; b = endb = NIL(Block_t*);
>> @@ -816,40 +821,43 @@
>>                         vd->free = bp;
>>                 else
>>                 {       /**/ASSERT(!vmonlist(CACHE(vd)[S_CACHE], bp) );
>>                         LINK(bp) = CACHE(vd)[S_CACHE];
>>                         CACHE(vd)[S_CACHE] = bp;
>>                 }
>>
>>                 /* coalesce on freeing large blocks to avoid fragmentation */
>>                 if(SIZE(bp) >= 2*vd->incr)
>>                 {       bestreclaim(vd,NIL(Block_t*),0);
>>                         if(vd->wild && SIZE(vd->wild) >= COMPACT*vd->incr)
>>                                 KPVCOMPACT(vm,bestcompact);
>>                 }
>>         }
>>
>>         if(!local && _Vmtrace && (vd->mode&VM_TRACE) && VMETHOD(vd) ==
>> VM_MTBEST )
>>                 (*_Vmtrace)(vm,(Vmuchar_t*)data,NIL(Vmuchar_t*), (s&~BITS), 0);
>>
>>         CLRLOCK(vm, local); /**/ASSERT(_vmbestcheck(vd, NIL(Block_t*)) == 0);
>>
>> +       if (!local)
>> +               VALGRIND_FREELIKE_BLOCK(data, 0);
>> +
>>         return 0;
>>  }
>>
>>  #if __STD_C
>>  static Void_t* bestresize(Vmalloc_t* vm, Void_t* data, reg size_t
>> size, int type, int local)
>>  #else
>>  static Void_t* bestresize(vm, data, size, type, local)
>>  Vmalloc_t*     vm;             /* region allocating from       */
>>  Void_t*                data;           /* old block of data            */
>>  reg size_t     size;           /* new size                     */
>>  int            type;           /* !=0 to move, <0 for not copy */
>>  int            local;
>>  #endif
>>  {
>>         reg Block_t     *rp, *np, *t;
>>         size_t          s, bs;
>>         size_t          oldz = 0,  orgsize = size;
>>         Void_t          *oldd = 0, *orgdata = data;
>>         Vmdata_t        *vd = vm->data;
>>
>> @@ -936,40 +944,46 @@
>>                         {       if(type&VM_RSCOPY)
>>                                         memcpy(data, oldd, bs);
>>
>>                         do_free: /* reclaim these right away */
>>                                 SETJUNK(SIZE(rp));
>>                                 LINK(rp) = CACHE(vd)[S_CACHE];
>>                                 CACHE(vd)[S_CACHE] = rp;
>>                                 bestreclaim(vd, NIL(Block_t*), S_CACHE);
>>                         }
>>                 }
>>         }
>>
>>         if(data && (type&VM_RSZERO) && (size = SIZE(BLOCK(data))&~BITS) > oldz )
>>                 memset((Void_t*)((Vmuchar_t*)data + oldz), 0, size-oldz);
>>
>>         if(!local && _Vmtrace && data && (vd->mode&VM_TRACE) &&
>> VMETHOD(vd) == VM_MTBEST)
>>                 (*_Vmtrace)(vm, (Vmuchar_t*)orgdata, (Vmuchar_t*)data,
>> orgsize, 0);
>>
>>         CLRLOCK(vm, local); /**/ASSERT(_vmbestcheck(vd, NIL(Block_t*)) == 0);
>>
>> +       if (!local)
>> +       {
>> +               VALGRIND_FREELIKE_BLOCK(orgdata, 0);
>> +               VALGRIND_MALLOCLIKE_BLOCK(data, size, 0, 0);
>> +       }
>> +
>>         return data;
>>  }
>>
>>  #if __STD_C
>>  static long bestsize(Vmalloc_t* vm, Void_t* addr, int local )
>>  #else
>>  static long bestsize(vm, addr, local)
>>  Vmalloc_t*     vm;     /* region allocating from       */
>>  Void_t*                addr;   /* address to check             */
>>  int            local;
>>  #endif
>>  {
>>         Seg_t           *seg;
>>         Block_t         *b, *endb;
>>         long            size;
>>         Vmdata_t        *vd = vm->data;
>>
>>         SETLOCK(vm, local);
>>
>>         size = -1L;
>> -- snip --
>
> ... aaand more digging: I found
> http://code.google.com/p/valgrind-variant/source/browse/trunk/valgrind/coregrind/m_replacemalloc/vg_replace_malloc.c#1175
> which seems to be from one of the valgrind forks... what about
> talking-up that idea and provide a command line option called
> --allocator-sym-redirect which works like passing down a small list of
> symbol mappings to instruct valgrind that it should monitor some extra
> allocators.
>
> Example:
> $ valgrind  "--allocator-sym-redirect=sh_malloc=malloc,sh_free=free,sh_calloc=calloc"
> ... # would instruct valgrind to take function |sh_malloc()| as an
> alternative |malloc(), |sh_free()| as alternative |free()| version
> etc. etc.
>
> The only issue is that if multiple allocators are active within a
> single process we may need some kind of "grouping" to explain valgrind
> that memory allocated by |sh_malloc()| can not be freed by |tcfree()|
> or |_ast_free()| ... maybe it could be done using '{'- and '}'-pairs,
> e.g. $ valgrind
> "--allocator-sym-redirect={sh_malloc=malloc,sh_free=free,sh_calloc=calloc},{_ast_malloc=malloc,_ast_free=free,_ast_calloc=calloc}"
> ... #

The idea of (finally!) providing such an option sounds like a very
good idea. Until now the only way to probe python and bash4 via
valgrind is to poke in the valgrind sources (which should never
happen).

I also think the idea to let valgrind detect mixing of different
allocators is a very valuable feature since this has been a source of
more and more bugs. Usually happens in complex projects with use many
different shared libraries, all with their own memory allocators.

Re: [Valgrind-users] Fwd: "__malloc_initialize_hook is deprecated"&&co. warnings while building ast-open.2013-04-22 ...

[Roland M. <rol...@nr...>]

On Wed, Apr 24, 2013 at 10:14 PM, Roland Mainz <rol...@nr...> wrote:
> On Wed, Apr 24, 2013 at 12:45 AM, John Reiser <jr...@bi...> wrote:
>>> Does valgrind provide any replacements for glibc's
>>> |__malloc_initialize_hook()| ? It seems this call and it's |*hook*()|
>>> siblings are depreciated now (at least in SuSE >=12.3) ...
>>
>> There is no glibc replacement.  [And the reasoning is correct.]
>> There is no valgrind replacement.
>> You must change your basic approach.
>>
>> We went through this just 6 months ago.
>> Check the archives of this mailing list:
>>
>>    [Valgrind-users] __malloc_hook
>>    Amir Szekely  <ki...@gm...>
>>    10/19/2012
>>
>> That thread contains code that works.
>> [The modification to detect the first use is obvious.]
>
> Grumpf... I tried that... but the combination how the stuff I'd like
> to instrument+debug is build+used makes that solution more or less
> impossible (for example... the allocator system lives in a seperate
> namespace, e.g. it has |malloc()|&&|free()| etc. but all symbols are
> prefixed with |_ast|, e.g. |_ast_malloc()|, |_ast_free()| etc.).
>
> I tried to work around the issues with the API provided in
> <valgrind/valgrind.h> ... but it seems this doesn't detect any
> read-from-unallocated etc. or even the plain double-free situations
> (patch below) ... erm... is the API around
> |VALGRIND_MALLOCLIKE_BLOCK()| know to work in valgrind-3.8.1 ?
>
> -- snip --
> --- src/lib/libast/vmalloc/vmbest.c      2012-06-28 22:12:14.000000000 +0200
> +++ src/lib/libast/vmalloc/vmbest.c     2013-04-24 03:03:44.207373019 +0200
> @@ -10,40 +10,42 @@
>  *          http://www.eclipse.org/org/documents/epl-v10.html           *
>  *         (with md5 checksum b35adb5213ca9657e911e9befb180842)         *
>  *                                                                      *
>  *              Information and Software Systems Research               *
>  *                            AT&T Research                             *
>  *                           Florham Park NJ                            *
>  *                                                                      *
>  *                 Glenn Fowler <gs...@re...>                  *
>  *                  David Korn <dg...@re...>                   *
>  *                   Phong Vo <kp...@re...>                    *
>  *                                                                      *
>  ***********************************************************************/
>  #if defined(_UWIN) && defined(_BLD_ast)
>
>  void _STUB_vmbest(){}
>
>  #else
>
>  #include       "vmhdr.h"
>
> +#include <valgrind/valgrind.h>
> +
>  /*     Best-fit allocation method. This is based on a best-fit strategy
>  **     using a splay tree for storage of lists of free blocks of the same
>  **     size. Recent free blocks may be cached for fast reuse.
>  **
>  **     Written by Kiem-Phong Vo, kp...@re..., 01/16/94.
>  */
>
>  #ifdef DEBUG
>  static int     N_free;         /* # of free calls                      */
>  static int     N_alloc;        /* # of alloc calls                     */
>  static int     N_resize;       /* # of resize calls                    */
>  static int     N_wild;         /* # allocated from the wild block      */
>  static int     N_last;         /* # allocated from last free block     */
>  static int     N_reclaim;      /* # of bestreclaim calls               */
>  #endif /*DEBUG*/
>
>  #define COMPACT                8       /* factor to decide when to
> compact     */
>
>  /* Check to see if a block is in the free tree */
>  #if __STD_C
> @@ -692,41 +694,44 @@
>
>                         if(VMWILD(vd,np))
>                         {       SIZE(np) &= ~BITS;
>                                 SELF(np) = np;
>                                 ap = NEXT(np); /**/ASSERT(ISBUSY(SIZE(ap)));
>                                 SETPFREE(SIZE(ap));
>                                 vd->wild = np;
>                         }
>                         else    vd->free = np;
>                 }
>
>                 SETBUSY(SIZE(tp));
>         }
>
>  done:
>         if(tp && !local && (vd->mode&VM_TRACE) && _Vmtrace &&
> VMETHOD(vd) == VM_MTBEST)
>                 (*_Vmtrace)(vm,NIL(Vmuchar_t*),(Vmuchar_t*)DATA(tp),orgsize,0);
>
>         CLRLOCK(vm,local); /**/ASSERT(_vmbestcheck(vd, NIL(Block_t*)) == 0);
>
> -       return tp ? DATA(tp) : NIL(Void_t*);
> +       void *res= tp ? DATA(tp) : NIL(Void_t*);
> +       if (!local)
> +               VALGRIND_MALLOCLIKE_BLOCK(res, size, 0, 0);
> +       return res;
>  }
>
>  #if __STD_C
>  static long bestaddr(Vmalloc_t* vm, Void_t* addr, int local )
>  #else
>  static long bestaddr(vm, addr, local)
>  Vmalloc_t*     vm;     /* region allocating from       */
>  Void_t*                addr;   /* address to check             */
>  int            local;
>  #endif
>  {
>         reg Seg_t*      seg;
>         reg Block_t     *b, *endb;
>         reg long        offset;
>         reg Vmdata_t*   vd = vm->data;
>
>         /**/ASSERT(local ? (vd->lock == 1) : 1 );
>         SETLOCK(vm, local);
>
>         offset = -1L; b = endb = NIL(Block_t*);
> @@ -816,40 +821,43 @@
>                         vd->free = bp;
>                 else
>                 {       /**/ASSERT(!vmonlist(CACHE(vd)[S_CACHE], bp) );
>                         LINK(bp) = CACHE(vd)[S_CACHE];
>                         CACHE(vd)[S_CACHE] = bp;
>                 }
>
>                 /* coalesce on freeing large blocks to avoid fragmentation */
>                 if(SIZE(bp) >= 2*vd->incr)
>                 {       bestreclaim(vd,NIL(Block_t*),0);
>                         if(vd->wild && SIZE(vd->wild) >= COMPACT*vd->incr)
>                                 KPVCOMPACT(vm,bestcompact);
>                 }
>         }
>
>         if(!local && _Vmtrace && (vd->mode&VM_TRACE) && VMETHOD(vd) ==
> VM_MTBEST )
>                 (*_Vmtrace)(vm,(Vmuchar_t*)data,NIL(Vmuchar_t*), (s&~BITS), 0);
>
>         CLRLOCK(vm, local); /**/ASSERT(_vmbestcheck(vd, NIL(Block_t*)) == 0);
>
> +       if (!local)
> +               VALGRIND_FREELIKE_BLOCK(data, 0);
> +
>         return 0;
>  }
>
>  #if __STD_C
>  static Void_t* bestresize(Vmalloc_t* vm, Void_t* data, reg size_t
> size, int type, int local)
>  #else
>  static Void_t* bestresize(vm, data, size, type, local)
>  Vmalloc_t*     vm;             /* region allocating from       */
>  Void_t*                data;           /* old block of data            */
>  reg size_t     size;           /* new size                     */
>  int            type;           /* !=0 to move, <0 for not copy */
>  int            local;
>  #endif
>  {
>         reg Block_t     *rp, *np, *t;
>         size_t          s, bs;
>         size_t          oldz = 0,  orgsize = size;
>         Void_t          *oldd = 0, *orgdata = data;
>         Vmdata_t        *vd = vm->data;
>
> @@ -936,40 +944,46 @@
>                         {       if(type&VM_RSCOPY)
>                                         memcpy(data, oldd, bs);
>
>                         do_free: /* reclaim these right away */
>                                 SETJUNK(SIZE(rp));
>                                 LINK(rp) = CACHE(vd)[S_CACHE];
>                                 CACHE(vd)[S_CACHE] = rp;
>                                 bestreclaim(vd, NIL(Block_t*), S_CACHE);
>                         }
>                 }
>         }
>
>         if(data && (type&VM_RSZERO) && (size = SIZE(BLOCK(data))&~BITS) > oldz )
>                 memset((Void_t*)((Vmuchar_t*)data + oldz), 0, size-oldz);
>
>         if(!local && _Vmtrace && data && (vd->mode&VM_TRACE) &&
> VMETHOD(vd) == VM_MTBEST)
>                 (*_Vmtrace)(vm, (Vmuchar_t*)orgdata, (Vmuchar_t*)data,
> orgsize, 0);
>
>         CLRLOCK(vm, local); /**/ASSERT(_vmbestcheck(vd, NIL(Block_t*)) == 0);
>
> +       if (!local)
> +       {
> +               VALGRIND_FREELIKE_BLOCK(orgdata, 0);
> +               VALGRIND_MALLOCLIKE_BLOCK(data, size, 0, 0);
> +       }
> +
>         return data;
>  }
>
>  #if __STD_C
>  static long bestsize(Vmalloc_t* vm, Void_t* addr, int local )
>  #else
>  static long bestsize(vm, addr, local)
>  Vmalloc_t*     vm;     /* region allocating from       */
>  Void_t*                addr;   /* address to check             */
>  int            local;
>  #endif
>  {
>         Seg_t           *seg;
>         Block_t         *b, *endb;
>         long            size;
>         Vmdata_t        *vd = vm->data;
>
>         SETLOCK(vm, local);
>
>         size = -1L;
> -- snip --

... aaand more digging: I found
http://code.google.com/p/valgrind-variant/source/browse/trunk/valgrind/coregrind/m_replacemalloc/vg_replace_malloc.c#1175
which seems to be from one of the valgrind forks... what about
talking-up that idea and provide a command line option called
--allocator-sym-redirect which works like passing down a small list of
symbol mappings to instruct valgrind that it should monitor some extra
allocators.

Example:
$ valgrind  "--allocator-sym-redirect=sh_malloc=malloc,sh_free=free,sh_calloc=calloc"
... # would instruct valgrind to take function |sh_malloc()| as an
alternative |malloc(), |sh_free()| as alternative |free()| version
etc. etc.

The only issue is that if multiple allocators are active within a
single process we may need some kind of "grouping" to explain valgrind
that memory allocated by |sh_malloc()| can not be freed by |tcfree()|
or |_ast_free()| ... maybe it could be done using '{'- and '}'-pairs,
e.g. $ valgrind
"--allocator-sym-redirect={sh_malloc=malloc,sh_free=free,sh_calloc=calloc},{_ast_malloc=malloc,_ast_free=free,_ast_calloc=calloc}"
... #

----

Bye,
Roland

-- 
  __ .  . __
 (o.\ \/ /.o) rol...@nr...
  \__\/\/__/  MPEG specialist, C&&JAVA&&Sun&&Unix programmer
  /O /==\ O\  TEL +49 641 3992797
 (;O/ \/ \O;)

Re: [Valgrind-users] Fwd: "__malloc_initialize_hook is deprecated"&&co. warnings while building ast-open.2013-04-22 ...

[Roland M. <rol...@nr...>]

On Wed, Apr 24, 2013 at 12:45 AM, John Reiser <jr...@bi...> wrote:
>> Does valgrind provide any replacements for glibc's
>> |__malloc_initialize_hook()| ? It seems this call and it's |*hook*()|
>> siblings are depreciated now (at least in SuSE >=12.3) ...
>
> There is no glibc replacement.  [And the reasoning is correct.]
> There is no valgrind replacement.
> You must change your basic approach.
>
> We went through this just 6 months ago.
> Check the archives of this mailing list:
>
>    [Valgrind-users] __malloc_hook
>    Amir Szekely  <ki...@gm...>
>    10/19/2012
>
> That thread contains code that works.
> [The modification to detect the first use is obvious.]

Grumpf... I tried that... but the combination how the stuff I'd like
to instrument+debug is build+used makes that solution more or less
impossible (for example... the allocator system lives in a seperate
namespace, e.g. it has |malloc()|&&|free()| etc. but all symbols are
prefixed with |_ast|, e.g. |_ast_malloc()|, |_ast_free()| etc.).

I tried to work around the issues with the API provided in
<valgrind/valgrind.h> ... but it seems this doesn't detect any
read-from-unallocated etc. or even the plain double-free situations
(patch below) ... erm... is the API around
|VALGRIND_MALLOCLIKE_BLOCK()| know to work in valgrind-3.8.1 ?

-- snip --
--- src/lib/libast/vmalloc/vmbest.c      2012-06-28 22:12:14.000000000 +0200
+++ src/lib/libast/vmalloc/vmbest.c     2013-04-24 03:03:44.207373019 +0200
@@ -10,40 +10,42 @@
 *          http://www.eclipse.org/org/documents/epl-v10.html           *
 *         (with md5 checksum b35adb5213ca9657e911e9befb180842)         *
 *                                                                      *
 *              Information and Software Systems Research               *
 *                            AT&T Research                             *
 *                           Florham Park NJ                            *
 *                                                                      *
 *                 Glenn Fowler <gs...@re...>                  *
 *                  David Korn <dg...@re...>                   *
 *                   Phong Vo <kp...@re...>                    *
 *                                                                      *
 ***********************************************************************/
 #if defined(_UWIN) && defined(_BLD_ast)

 void _STUB_vmbest(){}

 #else

 #include       "vmhdr.h"

+#include <valgrind/valgrind.h>
+
 /*     Best-fit allocation method. This is based on a best-fit strategy
 **     using a splay tree for storage of lists of free blocks of the same
 **     size. Recent free blocks may be cached for fast reuse.
 **
 **     Written by Kiem-Phong Vo, kp...@re..., 01/16/94.
 */

 #ifdef DEBUG
 static int     N_free;         /* # of free calls                      */
 static int     N_alloc;        /* # of alloc calls                     */
 static int     N_resize;       /* # of resize calls                    */
 static int     N_wild;         /* # allocated from the wild block      */
 static int     N_last;         /* # allocated from last free block     */
 static int     N_reclaim;      /* # of bestreclaim calls               */
 #endif /*DEBUG*/

 #define COMPACT                8       /* factor to decide when to
compact     */

 /* Check to see if a block is in the free tree */
 #if __STD_C
@@ -692,41 +694,44 @@

                        if(VMWILD(vd,np))
                        {       SIZE(np) &= ~BITS;
                                SELF(np) = np;
                                ap = NEXT(np); /**/ASSERT(ISBUSY(SIZE(ap)));
                                SETPFREE(SIZE(ap));
                                vd->wild = np;
                        }
                        else    vd->free = np;
                }

                SETBUSY(SIZE(tp));
        }

 done:
        if(tp && !local && (vd->mode&VM_TRACE) && _Vmtrace &&
VMETHOD(vd) == VM_MTBEST)
                (*_Vmtrace)(vm,NIL(Vmuchar_t*),(Vmuchar_t*)DATA(tp),orgsize,0);

        CLRLOCK(vm,local); /**/ASSERT(_vmbestcheck(vd, NIL(Block_t*)) == 0);

-       return tp ? DATA(tp) : NIL(Void_t*);
+       void *res= tp ? DATA(tp) : NIL(Void_t*);
+       if (!local)
+               VALGRIND_MALLOCLIKE_BLOCK(res, size, 0, 0);
+       return res;
 }

 #if __STD_C
 static long bestaddr(Vmalloc_t* vm, Void_t* addr, int local )
 #else
 static long bestaddr(vm, addr, local)
 Vmalloc_t*     vm;     /* region allocating from       */
 Void_t*                addr;   /* address to check             */
 int            local;
 #endif
 {
        reg Seg_t*      seg;
        reg Block_t     *b, *endb;
        reg long        offset;
        reg Vmdata_t*   vd = vm->data;

        /**/ASSERT(local ? (vd->lock == 1) : 1 );
        SETLOCK(vm, local);

        offset = -1L; b = endb = NIL(Block_t*);
@@ -816,40 +821,43 @@
                        vd->free = bp;
                else
                {       /**/ASSERT(!vmonlist(CACHE(vd)[S_CACHE], bp) );
                        LINK(bp) = CACHE(vd)[S_CACHE];
                        CACHE(vd)[S_CACHE] = bp;
                }

                /* coalesce on freeing large blocks to avoid fragmentation */
                if(SIZE(bp) >= 2*vd->incr)
                {       bestreclaim(vd,NIL(Block_t*),0);
                        if(vd->wild && SIZE(vd->wild) >= COMPACT*vd->incr)
                                KPVCOMPACT(vm,bestcompact);
                }
        }

        if(!local && _Vmtrace && (vd->mode&VM_TRACE) && VMETHOD(vd) ==
VM_MTBEST )
                (*_Vmtrace)(vm,(Vmuchar_t*)data,NIL(Vmuchar_t*), (s&~BITS), 0);

        CLRLOCK(vm, local); /**/ASSERT(_vmbestcheck(vd, NIL(Block_t*)) == 0);

+       if (!local)
+               VALGRIND_FREELIKE_BLOCK(data, 0);
+
        return 0;
 }

 #if __STD_C
 static Void_t* bestresize(Vmalloc_t* vm, Void_t* data, reg size_t
size, int type, int local)
 #else
 static Void_t* bestresize(vm, data, size, type, local)
 Vmalloc_t*     vm;             /* region allocating from       */
 Void_t*                data;           /* old block of data            */
 reg size_t     size;           /* new size                     */
 int            type;           /* !=0 to move, <0 for not copy */
 int            local;
 #endif
 {
        reg Block_t     *rp, *np, *t;
        size_t          s, bs;
        size_t          oldz = 0,  orgsize = size;
        Void_t          *oldd = 0, *orgdata = data;
        Vmdata_t        *vd = vm->data;

@@ -936,40 +944,46 @@
                        {       if(type&VM_RSCOPY)
                                        memcpy(data, oldd, bs);

                        do_free: /* reclaim these right away */
                                SETJUNK(SIZE(rp));
                                LINK(rp) = CACHE(vd)[S_CACHE];
                                CACHE(vd)[S_CACHE] = rp;
                                bestreclaim(vd, NIL(Block_t*), S_CACHE);
                        }
                }
        }

        if(data && (type&VM_RSZERO) && (size = SIZE(BLOCK(data))&~BITS) > oldz )
                memset((Void_t*)((Vmuchar_t*)data + oldz), 0, size-oldz);

        if(!local && _Vmtrace && data && (vd->mode&VM_TRACE) &&
VMETHOD(vd) == VM_MTBEST)
                (*_Vmtrace)(vm, (Vmuchar_t*)orgdata, (Vmuchar_t*)data,
orgsize, 0);

        CLRLOCK(vm, local); /**/ASSERT(_vmbestcheck(vd, NIL(Block_t*)) == 0);

+       if (!local)
+       {
+               VALGRIND_FREELIKE_BLOCK(orgdata, 0);
+               VALGRIND_MALLOCLIKE_BLOCK(data, size, 0, 0);
+       }
+
        return data;
 }

 #if __STD_C
 static long bestsize(Vmalloc_t* vm, Void_t* addr, int local )
 #else
 static long bestsize(vm, addr, local)
 Vmalloc_t*     vm;     /* region allocating from       */
 Void_t*                addr;   /* address to check             */
 int            local;
 #endif
 {
        Seg_t           *seg;
        Block_t         *b, *endb;
        long            size;
        Vmdata_t        *vd = vm->data;

        SETLOCK(vm, local);

        size = -1L;
-- snip --

----

Bye,
Roland

-- 
  __ .  . __
 (o.\ \/ /.o) rol...@nr...
  \__\/\/__/  MPEG specialist, C&&JAVA&&Sun&&Unix programmer
  /O /==\ O\  TEL +49 641 3992797
 (;O/ \/ \O;)

Re: [Valgrind-users] Performance profiling in VM using Valgrind

[Josef W. <Jos...@gm...>]

Am 24.04.2013 05:44, schrieb Liu James:
> Dear all,
>
> I used Valgrind(callgrind) to profile database cache performance, and
> the host machine is a VM guest with XEON CPU.
>
> Without instrumentation, the performance of dbms with cache-on is 20%
>   higher

What do you mean by higher? Better or worse performance?

  than cache-off. With callgrind, the call graph shows there is a
> 5x performance improvement.

For Callgrid/Cachegrind, the environment doesn't matter as it's a
simulation of a simple cache hierarchy, only takeing user level into
account. So it does not matter if it's running within a VM guest.

BUT:
I would assume that user-level simulation can not really capture
what's going on in a database, as there probably is much I/O and
the OS involved, and more important, the performance probably depends
on external latency/bandwidth contrains from network/drives etc.

> My point is whether Valgrind is suitable for profiling working in VM?
>   e.g, with IR conversion, the performance gap becomes larger, which
> does not reflect the real gap.   If you are familiar with profiling, I
> appreciate you could recommend some other profile tools.

Any profiler which does system-wide sampling should be fine (perf, 
oprofile, other Intel/AMD tools, ...). But the VM must support to
pass performance counters to guests, if that tool should work in the
guest. I think that perf running within a KVM guest should be able to
do this.
Interpreting the results is another issue. An idea would be to
check how much the performance depends on external constrains. A VM
should allow to throttle e.g. the network connection.

On the other hand - as John suggested - why not get rid of the VM for
profiling first?

More tools should work, and you also can compare results with/without
VM afterwards. The VM may play a role if a lot of TLB misses are
involved - even with hardware support (extended/nested page tables)
a TLB miss may be way slower in a VM guest.

Josef

>
>
> Thanks a lot!!!
>
>
> Best regards,
> James
>
>
> ------------------------------------------------------------------------------
> Try New Relic Now & We'll Send You this Cool Shirt
> New Relic is the only SaaS-based application performance monitoring service
> that delivers powerful full stack analytics. Optimize and monitor your
> browser, app, & servers with just a few lines of code. Try New Relic
> and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_apr
>
>
>
> _______________________________________________
> Valgrind-users mailing list
> Val...@li...
> https://lists.sourceforge.net/lists/listinfo/valgrind-users
>