valgrind-users — General discussion list for Valgrind users

Re: [Valgrind-users] massif & counts per type

[folkert <fo...@va...>]

Hi,

> > How can I obtain the number of mallocs per type in a time-frame using
> > massif? I'm NOT interested in the total in use, I would like to know how
> > often type x is allocated between t+1 and t+2.
> 
> To my knowledge, this data is not recorded by Massif. You could try to have a 
> look at heaptrack [1] instead.
> 
> [1]: https://invent.kde.org/sdk/heaptrack
> 
> Note though that on Linux, malloc does not retain any type information. As 
> such, even with heaptrack you cannot easily filter by type. But by leveraging 
> the flamegraph you can often still get a similar intuition based on the 
> callstacks. I have an idea to parse the callstack code lines to find type 
> information from the call to `new` but that's probably quite hairy to get 
> right in practice. Suggestions welcome on how to trace the type information!
> 
> And finally, with heaptrack it is also not yet easily doable to get a diff 
> between two time stamps. Also a feature I've long thought about implementing, 
> but never got around to...

This morning I came up with an other solution: I made a LD_PRELOAD
wrapper which counts every malloc-call. Decided that allocated-type is
not really required in my case, but to know which malloc did it would
work as well.
So in malloc() I do __builtin_return_address(0), hash the pointer and
use that as an index in array of counters.

Gives me:

pointer          count
...
000000000041db10 00000000002a427
00007ffff4a6c1b1 00000000007f1da

Going from 000000000041db10 to a symbol works fine, the shared library
(00007ffff4a6c1b1) is a bit troublesome though (disabled
randomize_va_space and using eu-addr2line).



Folkert van Heusden

-- 

[Valgrind-users] massif & counts per type

[folkert <fo...@va...>]

Hi,

How can I obtain the number of mallocs per type in a time-frame using
massif? I'm NOT interested in the total in use, I would like to know how
often type x is allocated between t+1 and t+2.


regards

Re: [Valgrind-users] correct self-modifying-code for Valgrind

[Duc N. <duc...@gm...>]

Thanks a lot Philippe!

regarding the code that performs self-modifying-code check:
*create_self_checks_as_needed*() (inside guest_generic_bb_to_IR.c). I am
trying to write another statement to just print a string out when a
self-modifying-code has been detected. To do that, I increase the allocated
spaces for self-modifying-check from 21 (i.e., three extents, 7 statements
each) to 24 (i.e, three extents, 8 statements each). The newly allocated
space is used to perform a mkIRExprCCall to the print function.

However, most of the existing support is for statements that create/modify
values e.g., IRStmt_WrTmp,.
I would like to have a mkIRExprCCall that simply calls a helper function
(similar to checksum functions) to print out a string when
self-modifying-code has been detected.
Yet, if such a statement (of the corresponding function) is not used in
the succeeding statement, the helper function is not triggered at run-time.

Would you please let me know if there is a way to integrate such a helper
function to print out a simple string indicating that a self-modifying-code
has been detected by Valgrind.

Thank you in advance for your help!

Best regards,

Duc

On Sun, Aug 30, 2020 at 4:40 PM Philippe Waroquiers <
phi...@sk...> wrote:

> Valgrind has a lot of heuristics to optimise the speed of the JIT-ted code.
> One of these heuristics is to chase jumps/calls to known destination.
>
> This is somewhat similar to inlining performed by the compiler,
> but performed by valgrind at runtime, when it encounters a new call/jump.
>
> In this case, the function f1 is inlined twice following this heuristic.
> So, the second inlining is using the modified function.
>
> If you disable chasing, then the code prints twice the same value:
>   valgrind --tool=none --smc-check=none --vex-guest-chase=no ...
> produces twice 4660 as output.
>
> Also, if you do a loop
>   for (int j = 0; j < 2; j++) {
>      f1();
>      .... here modify f1 code
>   }
>
> then valgrind inserts only once the code of f1, and it prints twice the
> same
> value, whatever the parameter --vex-guest-chase
>
> The code that does the self modifying code check is in the function
> needs_self_check
> in m_translate.c.  This function is called by VEX.
>
> Philippe
>
>
> On Fri, 2020-08-28 at 12:15 +0200, Duc Nguyen wrote:
> > Hello everyone,
> >
> > I am trying the self-modifying-code check of Valgrind but I am not sure
> if I understand the definition of  self-modifying-code in Valgrind
> correctly.
> >
> > I had prepared an example (see below) that has function f1 that is first
> executed in main, outputs something (number 4660). Afterward, two
> instructions of f1 are modified, and f1 is then executed one more time. It
> then outputs something (number 22068) that is different from the first
> time.
> >
> > When I run Valgrind with --smc-check=all and  --smc-check=none I do not
> see any difference in the outputs of Valgrind e.g., both times f1 produces
> different numbers (e.g., self-modifying-code successfully runs despite the
> --smc-check if turned on or off)
> >
> > Could someone please let me know if this behavior is expected from
> Valgrind?
> >
> > I further looked into the source code and found
> valgrind\VEX\priv\guest_generic_bb_to_IR.c that generates the code to
> check. However, I do not know where such a check is executed. It would be
> great if somebody knows where such a check takes place, and where we can
> modify the source code to just simply say e.g., self-modifying-code is
> found.
> >
> > Thank you very much in advance.
> >
> > Best regards,
> > Duc
> >
> >
> > =============================
> > Self-modifying-code example
> > -------
> >
> >
> >
> > #include <stdio.h>
> > #include <sys/mman.h>
> > #include <unistd.h>
> >
> > __asm__( ".text" );
> > __asm__( ".align 4096" );
> >
> > void f1( void )
> > {
> >   printf( "%d\n", 0x1234 );
> > }
> > void f2( void ){
> >   printf("this is just a dummy function");
> > }
> >
> > int main( void )
> > {
> >   int rc;
> >   int pagesize;
> >   char *p;
> >   int i;
> >
> >   printf( "f1=0x%08X.\n", f1 );
> >
> >   f1( );
> >
> >   pagesize = sysconf( _SC_PAGE_SIZE );
> >   printf( "pagesize=%d (0x%08X).\n", pagesize, pagesize );
> >   if( pagesize == -1 )
> >     return( 2 );
> >
> >   p = (char*) f1;
> >   rc = mprotect( p, pagesize, PROT_READ | PROT_WRITE | PROT_EXEC );
> >   printf( "rc=%d.\n", rc );
> >   if( rc != 0 )
> >     return( 2 );
> >   printf( "'mprotect()' succeeded.\n" );
> >
> >
> >   for( i = 0; i+1 < (size_t) f2- (size_t)f1; i++ ) {
> >      if( ((char*) f1)[ i ] == 0x34  && ((char*) f1)[ i+1 ] == 0x12 ) {
>
> >          ((char*) f1)[ i+1 ] =0x78;//here performs self-modifying-code
> >          ((char*) f1)[ i+1 ] =0x56;//here performs self-modifying-code
> >      }
> >    }
> >
> >   f1( );//here the output of f1 will be different from the first f1()
> call
> >
> >   printf( "Call succeeded.\n" );
> >   return( 0 );
> > }
> >
> >
> > _______________________________________________
> > Valgrind-users mailing list
> > Val...@li...
> > https://lists.sourceforge.net/lists/listinfo/valgrind-users
>
>

Re: [Valgrind-users] valgrind always fails with out of memory

[Mario E. <ma...@em...>]

On 08.09.20 12:25, Mario Emmenlauer wrote:
> 
> On 08.09.20 12:04, Mario Emmenlauer wrote:
>> The error I get most frequently is (full output attached in log.txt)
>> ==32==     Valgrind's memory management: out of memory:
>> ==32==        newSuperblock's request for 6864695621860790272 bytes failed.
>> ==32==          114,106,368 bytes have already been mmap-ed ANONYMOUS.
> 
> Argh! After sending the email, I went through the stack trace for
> the hundredth time, and spotted the use of "zlib". And indeed, when
> replacing my own zlib 1.2.11 with the system zlib 1.2.11, valgrind
> works as expected!
> 
> Does that make sense? Is zlib used by valgrind itself? And why could
> my debug build differ (so much) from the system zlib that it breaks
> valgrind? I double-checked and its the identical source code from
> Ubuntu, just missing two or three patches.

So it seems I can (partially) answer my own question: when valgrind
is used on an executable that links zlib built with -ggdb3, then it
does not work (due to aforementioned error). Keeping all other debug-
settings except -ggdb3 works still fine.

I have no clue as to _why_ this may happen, but I hope it can be
helpful to other people running into the same issue.

All the best,

    Mario Emmenlauer

[Valgrind-users] valgrind always fails with out of memory

[Mario E. <ma...@em...>]

Dear All,

many years ago, I've been using valgrind frequently and successfully,
admittedly without ever giving it much thought! Thanks for the awesome
tool.

Now I'm setting up a larger CI system and want automatic memcheck for
our tests. However, in the whole past year, I could not get a single
successful run. So I must be doing something very wrong. Help would be
greatly appreciated :-(


The error I get most frequently is (full output attached in log.txt)
==32==     Valgrind's memory management: out of memory:
==32==        newSuperblock's request for 6864695621860790272 bytes failed.
==32==          114,106,368 bytes have already been mmap-ed ANONYMOUS.


Here is what I tried so far:
 - Versions valgrind-3.13.0 from Ubuntu 18.04 and valgrind-3.16.1
   compiled from source
 - Executed valgrind in a docker container running Ubuntu 18.04 x86_64
   and Ubuntu 20.04 x86_64
 - Checked `ulimit -a` in Docker, there are no tight limits
 - Tried valgrind with some 50++ different executables, all lead to
   the same error message
 - Tried valgrind outside Docker, leads to the same error message
 - Checked `ulimit -a` outside Docker, there are no tight limits
 - Tried the tests work successfully when _not_ using valgrind


I have also tried valgrind on other executables than our debug builds,
and it seems to work there without problems. So maybe the errors are
related to how we create debug builds?

We make pretty standard debug builds (I assume), with flags
-ggdb3 -fno-omit-frame-pointer -O1 -m64 -march=nehalem -mtune=haswell.
Are some of these suspicious?

The host machines I have tried are relatively modern desktop computers
with 64GB of RAM, and modern Skylake or Ryzen processors. The OS is
typically Ubuntu 18.04 or 20.04. I have not set up any tight permission
restrictions like selinux (unless it would be the default for Ubuntu).

And ideas for what I can try are more than appreciated!

All the best,

    Mario Emmenlauer

Re: [Valgrind-users] valgrind always fails with out of memory

[Mario E. <ma...@em...>]

On 08.09.20 12:04, Mario Emmenlauer wrote:
> The error I get most frequently is (full output attached in log.txt)
> ==32==     Valgrind's memory management: out of memory:
> ==32==        newSuperblock's request for 6864695621860790272 bytes failed.
> ==32==          114,106,368 bytes have already been mmap-ed ANONYMOUS.

Argh! After sending the email, I went through the stack trace for
the hundredth time, and spotted the use of "zlib". And indeed, when
replacing my own zlib 1.2.11 with the system zlib 1.2.11, valgrind
works as expected!

Does that make sense? Is zlib used by valgrind itself? And why could
my debug build differ (so much) from the system zlib that it breaks
valgrind? I double-checked and its the identical source code from
Ubuntu, just missing two or three patches.

All the best,

    Mario Emmenlauer


--
BioDataAnalysis GmbH, Mario Emmenlauer      Tel. Buero: +49-89-74677203
Balanstr. 43                   mailto: memmenlauer * biodataanalysis.de
D-81669 München                          http://www.biodataanalysis.de/

Re: [Valgrind-users] correct self-modifying-code for Valgrind

[Philippe W. <phi...@sk...>]

Valgrind has a lot of heuristics to optimise the speed of the JIT-ted code.
One of these heuristics is to chase jumps/calls to known destination.

This is somewhat similar to inlining performed by the compiler,
but performed by valgrind at runtime, when it encounters a new call/jump.

In this case, the function f1 is inlined twice following this heuristic.
So, the second inlining is using the modified function.

If you disable chasing, then the code prints twice the same value:
  valgrind --tool=none --smc-check=none --vex-guest-chase=no ...
produces twice 4660 as output.

Also, if you do a loop 
  for (int j = 0; j < 2; j++) {
     f1();
     .... here modify f1 code
  }

then valgrind inserts only once the code of f1, and it prints twice the same
value, whatever the parameter --vex-guest-chase

The code that does the self modifying code check is in the function needs_self_check
in m_translate.c.  This function is called by VEX.

Philippe


On Fri, 2020-08-28 at 12:15 +0200, Duc Nguyen wrote:
> Hello everyone,
> 
> I am trying the self-modifying-code check of Valgrind but I am not sure if I understand the definition of  self-modifying-code in Valgrind correctly. 
> 
> I had prepared an example (see below) that has function f1 that is first executed in main, outputs something (number 4660). Afterward, two instructions of f1 are modified, and f1 is then executed one more time. It then outputs something (number 22068) that is different from the first time. 
> 
> When I run Valgrind with --smc-check=all and  --smc-check=none I do not see any difference in the outputs of Valgrind e.g., both times f1 produces different numbers (e.g., self-modifying-code successfully runs despite the --smc-check if turned on or off)
> 
> Could someone please let me know if this behavior is expected from Valgrind?
> 
> I further looked into the source code and found valgrind\VEX\priv\guest_generic_bb_to_IR.c that generates the code to check. However, I do not know where such a check is executed. It would be great if somebody knows where such a check takes place, and where we can modify the source code to just simply say e.g., self-modifying-code is found.
> 
> Thank you very much in advance.
> 
> Best regards,
> Duc
> 
> 
> =============================
> Self-modifying-code example
> -------
> 
> 
> 
> #include <stdio.h>
> #include <sys/mman.h>
> #include <unistd.h>
> 
> __asm__( ".text" );
> __asm__( ".align 4096" );
> 
> void f1( void )
> {
>   printf( "%d\n", 0x1234 );
> }
> void f2( void ){
>   printf("this is just a dummy function");
> }
> 
> int main( void )
> {
>   int rc;
>   int pagesize;
>   char *p;
>   int i;
> 
>   printf( "f1=0x%08X.\n", f1 );
> 
>   f1( );
> 
>   pagesize = sysconf( _SC_PAGE_SIZE );
>   printf( "pagesize=%d (0x%08X).\n", pagesize, pagesize );
>   if( pagesize == -1 )
>     return( 2 );
> 
>   p = (char*) f1;
>   rc = mprotect( p, pagesize, PROT_READ | PROT_WRITE | PROT_EXEC );
>   printf( "rc=%d.\n", rc );
>   if( rc != 0 )
>     return( 2 );
>   printf( "'mprotect()' succeeded.\n" );
> 
> 
>   for( i = 0; i+1 < (size_t) f2- (size_t)f1; i++ ) {
>      if( ((char*) f1)[ i ] == 0x34  && ((char*) f1)[ i+1 ] == 0x12 ) {         
>          ((char*) f1)[ i+1 ] =0x78;//here performs self-modifying-code
>          ((char*) f1)[ i+1 ] =0x56;//here performs self-modifying-code
>      }
>    }
>   
>   f1( );//here the output of f1 will be different from the first f1() call
> 
>   printf( "Call succeeded.\n" );
>   return( 0 );
> }    
> 
> 
> _______________________________________________
> Valgrind-users mailing list
> Val...@li...
> https://lists.sourceforge.net/lists/listinfo/valgrind-users

[Valgrind-users] correct self-modifying-code for Valgrind

[Duc N. <duc...@gm...>]

Hello everyone,

I am trying the self-modifying-code check of Valgrind but I am not sure if
I understand the definition of  self-modifying-code in Valgrind correctly.

I had prepared an example (see below) that has function f1 that is first
executed in main, outputs something (number 4660). Afterward, two
instructions of f1 are modified, and f1 is then executed one more time. It
then outputs something (number 22068) that is different from the first
time.

When I run Valgrind with --smc-check=all and  --smc-check=none I do not see
any difference in the outputs of Valgrind e.g., both times f1 produces
different numbers (e.g., self-modifying-code successfully runs despite the
--smc-check if turned on or off)

*Could someone please let me know if this behavior is expected from
Valgrind?*

I further looked into the source code and found
valgrind\VEX\priv\guest_generic_bb_to_IR.c that generates the code to
check. However, I do not know where such a check is executed. It would be
great if somebody knows where such a check takes place, and *where we can
modify the source code to just simply say e.g., self-modifying-code is
found*.

Thank you very much in advance.

Best regards,
Duc


=============================
Self-modifying-code example
-------





















































*#include <stdio.h>#include <sys/mman.h>#include <unistd.h>__asm__( ".text"
);__asm__( ".align 4096" );void f1( void ){  printf( "%d\n", 0x1234 );}void
f2( void ){  printf("this is just a dummy function");}int main( void ){
int rc;  int pagesize;  char *p;  int i;  printf( "f1=0x%08X.\n", f1 );
f1( );  pagesize = sysconf( _SC_PAGE_SIZE );  printf( "pagesize=%d
(0x%08X).\n", pagesize, pagesize );  if( pagesize == -1 )    return( 2 );
p = (char*) f1;  rc = mprotect( p, pagesize, PROT_READ | PROT_WRITE |
PROT_EXEC );  printf( "rc=%d.\n", rc );  if( rc != 0 )    return( 2 );
printf( "'mprotect()' succeeded.\n" );  for( i = 0; i+1 < (size_t) f2-
(size_t)f1; i++ ) {     if( ((char*) f1)[ i ] == 0x34  && ((char*) f1)[ i+1
] == 0x12 ) {                  ((char*) f1)[ i+1 ] =0x78;//here performs
self-modifying-code         ((char*) f1)[ i+1 ] =0x56;//here performs
self-modifying-code     }   }    f1( );//here the output of f1 will be
different from the first f1() call  printf( "Call succeeded.\n" );  return(
0 );}    *

Re: [Valgrind-users] compat_ioctl cmd does not match even if it shows same value

[Paul F. <pj...@wa...>]

> This is incorrect for C99 and indeed any compiler that supports "long long". On such systems, the integer constant 3222829167
> has type "long long", and it is absolutely guaranteed to preserve that value when cast to unsigned. Assuming 32-bit int, of course.

It is also incorrect for C90. See here https://en.cppreference.com/w/c/language/integer_constant. For C90 an integer constant
without suffix can be int, long int or unsigned long int. In this case is it would be unsigned long int.

A+
Paul

Re: [Valgrind-users] compat_ioctl cmd does not match even if it shows same value

[Patrick J. L. <lop...@gm...>]

On Sat, Aug 15, 2020 at 10:59 AM John Reiser <jr...@bi...> wrote:
>
> The fact that you write '0xc018786f' here, but '3222829167' in the code,
> shows that you are not sufficiently paranoid.  First, if the bit pattern
is important
> then you should write hex.  If you insist on decimal, then you should
write '3222829167u'
> to remind yourself and the compiler that the value is unsigned; otherwise
because
> 3222829167 exceeds INT_MAX then a compiler is allowed to interpret it as
-1072138129
> (or ANY VALUE WHATSOEVER!!!)

This is incorrect for C99 and indeed any compiler that supports "long
long". On such systems, the integer constant 3222829167 has type "long
long", and it is absolutely guaranteed to preserve that value when cast to
unsigned. Assuming 32-bit int, of course.

I agree it is poor style, but there is essentially zero chance it is
causing a problem here. Although 32-bit vs. 64-bit confusion elsewhere
certainly might.

 - Pat

Re: [Valgrind-users] compat_ioctl cmd does not match even if it shows same value

[John R. <jr...@bi...>]

> long mgr_compat_ioctl(struct file *pFile, unsigned int cmd, unsigned long arg)
> {
>      int err = -EFAULT;
>      //unsigned int openLink = kIoctlOpenEthernetLink;
>      unsigned int openLink = 3222829167;

"3222829167" is not a legal 32-bit value in source code.  You MUST use "3222829167u"
with a trailing 'u'.  Do not post or reply to this mailing list until you have changed
the source, re-compiled, re-linked, and re-run the test.

> 
>      printk("cmd 0x%x/%u, openLink 0x%x/%u\n", cmd, cmd, openLink, openLink);

//  [409204.415924/1] cmd 0xc018786f/3222829167, openLink 0xc018786f/3222829167

The documentation for ioctl(), as shown by running the command "man 2 ioctl",
says that the prototype is:

        int ioctl(int fd, unsigned long request, ...);

Notice that the second parameter has type "unsigned long", which is 64 bits.
In contrast, the definition of this function 'mgr_compat_ioctl' has a second
parameter type of "unsigned int", which is 32 bits.  Beware of this difference.
[Also note that the width of the return value differs: 'int' in the documentation,
'long' in the implementation; but this should not matter here.]

Try an experiment: change the implementation to:

    long mgr_compat_ioctl(struct file *pFile, unsigned long cmd, unsigned long arg)

where the second parameter has 'long' width instead of 'int'.  If necessary,
(for instance, in the "struct file_operations mgr_fops" table of function pointers)
then cast the type of the function pointer &mgr_compat_ioctl to hide the difference
in the type of the second parameter.  At the same time, change the printk format
conversions corresponding to 'cmd' to become "0x%lx/%lu" which has two 'l' modifiers.

The experiment might reveal that the second parameter to mgr_compat_ioctl is being passed
a 64-bit value whose upper 32 bits are not all zero.

> 
>      if (openLink == cmd) {
>          printk("match\n");
>      } else {
>          printk("no match: cmd 0x%x/%u, openLink 0x%x/%u\n", cmd, cmd, openLink, openLink);
>      }

Post the assembly-language code for the function mgr_compat_ioctl.  The safest way
is to use "(gdb) disassemble mgr_compat_ioctl" or "objdump --disaassemble",
but "gcc -S" usually is OK.  We need to see EXACTLY what instructions are generated.

-- 

Re: [Valgrind-users] compat_ioctl cmd does not match even if it shows same value

[Manomugdha B. <man...@ke...>]

Hi John,
Valgrind version: valgrind-3.14.0

compat_ioctl prototype:
/* File operations core */
struct file_operations  mgr_fops =
    {
    owner:
        THIS_MODULE,
    open:
        mgr_open_common,
    FILE_OPS_IOCTL(mgr_ioctl_common),
#ifdef CONFIG_COMPAT
    compat_ioctl:
        mgr_compat_ioctl,
#endif
    poll:
        mgr_poll_common,
    release:
        mgr_close_common
    };

long mgr_compat_ioctl(struct file *pFile, unsigned int cmd, unsigned long arg)
{
    int err = -EFAULT;
    //unsigned int openLink = kIoctlOpenEthernetLink;
    unsigned int openLink = 3222829167;

    printk("cmd 0x%x/%u, openLink 0x%x/%u\n", cmd, cmd, openLink, openLink);

    if (openLink == cmd) {
        printk("match\n");
    } else {
        printk("no match: cmd 0x%x/%u, openLink 0x%x/%u\n", cmd, cmd, openLink, openLink);
    }
}

Output is:
[409204.415924/1] cmd 0xc018786f/3222829167, openLink 0xc018786f/3222829167
[409204.415933/1] no match: cmd 0xc018786f/3222829167, openLink 0xc018786f/3222829167

Note1: it fails at MSB. If i do memcmp instead of "if" then it works.
Note2: this is happening only under valgrind. When I run it without valgrind then it works fine.
Note3: this is a kernel module and ioctl is called from user space application.

Regards,
Mano

-----Original Message-----
From: John Reiser <jr...@bi...> 
Sent: Saturday, August 15, 2020 11:28 PM
To: val...@li...
Subject: Re: [Valgrind-users] compat_ioctl cmd does not match even if it shows same value

[EXTERNAL]

> I have a 32 bit application (user space) which communicate with a 
> kernel module through compat_ioctl(). My system is
>
> # uname -a
>
> Linux chassis1-board1-port5 3.10 #1 SMP Fri Apr 24 02:31:48 PDT 2020 
> mips64 GNU/Linux

Which version of valgrind?

The source code of valgrind:
     commit 24b247aec5397cad5f6cfcf885f118e90fea8735 (HEAD -> master, origin/master, origin/HEAD)
     Date:   Sat Aug 15 16:54:14 2020 +0200
does not contain the string 'compat_ioctl' anywhere, so you must show us the prototype and the environment (32-bit or 64-bit) for compat_ioctl.

> Ioctl function is following:
>
> long mgr_compat_ioctl(struct file *pFile, unsigned int cmd, unsigned 
> long arg)
>
> inside this function definition, I am getting correct cmd value e.g. 0xc018786f.

The fact that you write '0xc018786f' here, but '3222829167' in the code, shows that you are not sufficiently paranoid.  First, if the bit pattern is important then you should write hex.  If you insist on decimal, then you should write '3222829167u'
to remind yourself and the compiler that the value is unsigned; otherwise because
3222829167 exceeds INT_MAX then a compiler is allowed to interpret it as -1072138129 (or ANY VALUE WHATSOEVER!!!) Then, it is very likely that your problem lies in a mismatch of width (64-bit vs 32-bit) and/or signedness at one or more interfaces (subroutine call or system call).
You say "printk shows same value" but you did not say what format conversion you specified to printk, therefore no one can determine what the actual value is.
This is likely to be important, because printk is implemented using "..." varargs, and on most 64-bit machines that means that most scalar actual arguments are promoted to 64-bit width upon the transition into the varargs mechanism, and it is UNSPECIFIED whether the conversion from 32-bit to 64-bit is sign-extended or zero-extended.
That makes it extremely important that each varargs access uses the desired width and signedness.

Due to the possibility of bugs, the only way to be sure of what is happening is to look at the assembly-language code, and examine registers and values while single-stepping actual execution.

--


_______________________________________________
Valgrind-users mailing list
Val...@li...
https://urldefense.com/v3/__https://lists.sourceforge.net/lists/listinfo/valgrind-users__;!!I5pVk4LIGAfnvw!w9ZQebKExIzQOBnXRp0AgAnSuApBAKyRQwK8-da-j4Y9NjzgGTYvJNpMt7LSIX7caCZHaZDi$

Re: [Valgrind-users] compat_ioctl cmd does not match even if it shows same value

[John R. <jr...@bi...>]

> I have a 32 bit application (user space) which communicate with a kernel module through compat_ioctl(). My system is
> 
> # uname -a
> 
> Linux chassis1-board1-port5 3.10 #1 SMP Fri Apr 24 02:31:48 PDT 2020 mips64 GNU/Linux

Which version of valgrind?

The source code of valgrind:
     commit 24b247aec5397cad5f6cfcf885f118e90fea8735 (HEAD -> master, origin/master, origin/HEAD)
     Date:   Sat Aug 15 16:54:14 2020 +0200
does not contain the string 'compat_ioctl' anywhere, so you must show us the prototype
and the environment (32-bit or 64-bit) for compat_ioctl.

> Ioctl function is following:
> 
> long mgr_compat_ioctl(struct file *pFile, unsigned int cmd, unsigned long arg)
> 
> inside this function definition, I am getting correct cmd value e.g. 0xc018786f.

The fact that you write '0xc018786f' here, but '3222829167' in the code,
shows that you are not sufficiently paranoid.  First, if the bit pattern is important
then you should write hex.  If you insist on decimal, then you should write '3222829167u'
to remind yourself and the compiler that the value is unsigned; otherwise because
3222829167 exceeds INT_MAX then a compiler is allowed to interpret it as -1072138129
(or ANY VALUE WHATSOEVER!!!)
Then, it is very likely that your problem lies in a mismatch of width (64-bit vs 32-bit)
and/or signedness at one or more interfaces (subroutine call or system call).
You say "printk shows same value" but you did not say what format conversion
you specified to printk, therefore no one can determine what the actual value is.
This is likely to be important, because printk is implemented using "..." varargs,
and on most 64-bit machines that means that most scalar actual arguments are promoted
to 64-bit width upon the transition into the varargs mechanism, and it is UNSPECIFIED
whether the conversion from 32-bit to 64-bit is sign-extended or zero-extended.
That makes it extremely important that each varargs access uses the desired width
and signedness.

Due to the possibility of bugs, the only way to be sure of what is happening
is to look at the assembly-language code, and examine registers and values
while single-stepping actual execution.

-- 

[Valgrind-users] compat_ioctl cmd does not match even if it shows same value

[Manomugdha B. <man...@ke...>]

Hi,
I have a 32 bit application (user space) which communicate with a kernel module through compat_ioctl(). My system is
# uname -a
Linux chassis1-board1-port5 3.10 #1 SMP Fri Apr 24 02:31:48 PDT 2020 mips64 GNU/Linux

Ioctl function is following:
long mgr_compat_ioctl(struct file *pFile, unsigned int cmd, unsigned long arg)

inside this function definition, I am getting correct cmd value e.g. 0xc018786f. the variable to which this cmd is to be matched has exactly same value of 0xc018786f (printk shows same value). But still if check fails.

Sample code is like this:
int mgr_ioctl_common (struct inode *pInode, struct file *pFile,
                             unsigned int cmd, unsigned long arg)
{
    int err = -EFAULT;
    int minor;
    void  *pUserStruct = (void *)arg;
    SkixpppMgrDevDesc *pDev = NULL;
    tKHandle retHandle;

    unsigned int openLink = 3222829167;
   if (openLink == cmd) { //printk shows value of cmd is 3222829167
              printk("matches\n");
   } else {
              printk("no matche\n");
  }
}
Why this if check fails here?

Regards,
Mano

Re: [Valgrind-users] compat_ioctl cmd does not match even if it shows same value

[Manomugdha B. <man...@ke...>]

(openLink - cmd) shows zero still if check fails.
Note: this is happening only under valgrind!

Regards,
Mano, IxNetwork
[Keysight-signature-block-5]<http://www.keysight.com/>

From: Manomugdha Biswas
Sent: Saturday, August 15, 2020 6:38 PM
To: val...@li...
Subject: compat_ioctl cmd does not match even if it shows same value

Hi,
I have a 32 bit application (user space) which communicate with a kernel module through compat_ioctl(). My system is
# uname -a
Linux chassis1-board1-port5 3.10 #1 SMP Fri Apr 24 02:31:48 PDT 2020 mips64 GNU/Linux

Ioctl function is following:
long mgr_compat_ioctl(struct file *pFile, unsigned int cmd, unsigned long arg)

inside this function definition, I am getting correct cmd value e.g. 0xc018786f. the variable to which this cmd is to be matched has exactly same value of 0xc018786f (printk shows same value). But still if check fails.

Sample code is like this:
int mgr_ioctl_common (struct inode *pInode, struct file *pFile,
                             unsigned int cmd, unsigned long arg)
{
    int err = -EFAULT;
    int minor;
    void  *pUserStruct = (void *)arg;
    SkixpppMgrDevDesc *pDev = NULL;
    tKHandle retHandle;

    unsigned int openLink = 3222829167;
   if (openLink == cmd) { //printk shows value of cmd is 3222829167
              printk("matches\n");
   } else {
              printk("no matche\n");
  }
}
Why this if check fails here?

Regards,
Mano

Re: [Valgrind-users] Use of valgrind with microthreading

[Philippe W. <phi...@sk...>]

On Fri, 2020-08-14 at 19:16 +0200, Mark Wielaard wrote:
> Hi Shachar,
> 
> On Fri, Aug 14, 2020 at 06:35:14PM +0300, Shachar Shemesh wrote:
> > I am writing my own C++ microthreading library. At the very first switch, I
> > get the following message from Valgrind:
> > 
> > ==15122== Warning: client switching stacks?  SP change: 0x1ffeffe788 -->
> > 0x75d0f80
> > ==15122==          to suppress, use: --max-stackframe=137298630664 or
> > greater
> > 
> > 
> > I understand that valgrind needs to know where the stack starts and ends. I
> > am wondering whether is any way I can tell it that information. Since I'm
> > writing the library, what I was thinking was compiling the library with a
> > special flag saying "you're running with valgrind", and then have my library
> > call a valgrind function that says "This is the new stack range, this is the
> > old one".
> >  
> > Is there such a thing? If so, how do I interface with it?
> 
> Yes, there is such a thing. The valgrind client request mechanism:
> https://valgrind.org/docs/manual/manual-core-adv.html#manual-core-adv.clientreq
> 
> It does have various VALGRIND_STACK_* macros to signal use of user
> threads/stacks. But note that the documentations says:
> Warning: Unfortunately, this client request is unreliable and best avoided.
> Unfortunately I don't know why that is, or what alternative mechanisms there are.

I also do not know what the 'unreliable' doc warning points at.
Searching in bugzilla, there are a few bugs that are related to such STACK requests:
Bug 202463 - VALGRIND_STACK_{REGISTER,DEREGISTER,CHANGE} are unreliable 
Bug 133154 - crash when using client requests to register/deregister stack

However, some years ago, I did various improvements to the stack handling code
in order to better support running valgrind under valgrind: the inner valgrind is
using such client requests to inform the outer valgrind about the inner stacks.

So, IMO, these requests should (could?) work reasonably well.

Philippe

     

Re: [Valgrind-users] Use of valgrind with microthreading

[Mark W. <ma...@kl...>]

Hi Shachar,

On Fri, Aug 14, 2020 at 06:35:14PM +0300, Shachar Shemesh wrote:
> I am writing my own C++ microthreading library. At the very first switch, I
> get the following message from Valgrind:
> 
> ==15122== Warning: client switching stacks?  SP change: 0x1ffeffe788 -->
> 0x75d0f80
> ==15122==          to suppress, use: --max-stackframe=137298630664 or
> greater
> 
> 
> I understand that valgrind needs to know where the stack starts and ends. I
> am wondering whether is any way I can tell it that information. Since I'm
> writing the library, what I was thinking was compiling the library with a
> special flag saying "you're running with valgrind", and then have my library
> call a valgrind function that says "This is the new stack range, this is the
> old one".
>  
> Is there such a thing? If so, how do I interface with it?

Yes, there is such a thing. The valgrind client request mechanism:
https://valgrind.org/docs/manual/manual-core-adv.html#manual-core-adv.clientreq

It does have various VALGRIND_STACK_* macros to signal use of user
threads/stacks. But note that the documentations says:
Warning: Unfortunately, this client request is unreliable and best avoided.
Unfortunately I don't know why that is, or what alternative mechanisms there are.

Cheers,

Mark

[Valgrind-users] Use of valgrind with microthreading

[Shachar S. <sh...@sh...>]

Hi all,


I am writing my own C++ microthreading library. At the very first 
switch, I get the following message from Valgrind:

==15122== Warning: client switching stacks?  SP change: 0x1ffeffe788 --> 
0x75d0f80
==15122==          to suppress, use: --max-stackframe=137298630664 or 
greater


I understand that valgrind needs to know where the stack starts and 
ends. I am wondering whether is any way I can tell it that information. 
Since I'm writing the library, what I was thinking was compiling the 
library with a special flag saying "you're running with valgrind", and 
then have my library call a valgrind function that says "This is the new 
stack range, this is the old one".


Is there such a thing? If so, how do I interface with it?


Thank you,

Shachar

Re: [Valgrind-users] Mismatched free() / delete / delete[] -- false positives

[<pj...@wa...>]

<div dir='auto'><div dir="auto">Maybe. Can you try --soname-synonyms=malloc=NONE ?</div><div><br><div class="elided-text">Paul</div></div></div>

Re: [Valgrind-users] Mismatched free() / delete / delete[] -- false positives

[Foelsche, P. <Pet...@me...>]

Could this be caused by some smart* using

-static-libstdc

To create one of the involved shared libraries?

Peter


From: Foelsche, Peter
Sent: Tuesday, August 4, 2020 10:28
To: val...@li...
Subject: [Valgrind-users] Mismatched free() / delete / delete[] -- false positives


*         Valgrind-3.13.0

*         CentOS release 6.5 (Final)

*         g++ (GCC) 5.3.0

*         no usage of tcmalloc
I checked by
cat /proc/<PID>/maps |sed -e 's/[a-zA-Z0-9_:-]*[ \t][ \t]*//g'|sort -u|xargs nm -A|grep '\<free\>'|grep -v '\<U\>'
output was
/lib64/ld-2.12.so:0000003bfba16090 W free
/lib64/libc-2.12.so:0000003bfc27b520 T free
/path/executable:000000000f9f3c50 W free

*         positives for both directions

o   free used for memory allocated with new

o   delete used for memory allocated with malloc

1)
==11679== Mismatched free() / delete / delete []
==11679==    at 0x127ECAFE: free (vg_replace_malloc.c:530)
==11679==    by 0x1CC871A1: void std::_Destroy<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*) (stl_construct.h:93)
==11679==    by 0x1CC82295: void std::_Destroy_aux<false>::__destroy<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*) (stl_construct.h:103)
...
==11679==  Address 0x2633aeb0 is 0 bytes inside a block of size 155 alloc'd
==11679==    at 0x127EDBE7: operator new(unsigned long) (vg_replace_malloc.c:334)
==11679==    by 0x54B81C3: void std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_construct<char*>(char*, char*, std::forwar
d_iterator_tag) (basic_string.tcc:223)

2)
==11679== Mismatched free() / delete / delete []
==11679==    at 0x127EC64E: operator delete(void*) (vg_replace_malloc.c:576)
==11679==    by 0x8172EC3: __gnu_cxx::new_allocator<Sdi::StateVar const*>::deallocate(Sdi::StateVar const**, unsigned long) (new_allocator.h:110)
==11679==    by 0x8172767: std::allocator_traits<std::allocator<Sdi::StateVar const*> >::deallocate(std::allocator<Sdi::StateVar const*>&, Sdi::StateVar const**, unsigned long) (alloc_traits.h:386)
...
==11679==  Address 0x2a5e08b0 is 0 bytes inside a block of size 32 alloc'd
==11679==    at 0x127ED104: malloc (vg_replace_malloc.c:299)
==11679==    by 0x206F33E7: operator new(unsigned long) (new_op.cc:50)
==11679==    by 0x1CCE9829: __gnu_cxx::new_allocator<Sdi::StateVar const*>::allocate(unsigned long, void const*) (new_allocator.h:104)


There are only a 7 of the 1) messages and 3 of the 2) despite there are many more such allocations.
As you can see, both places are inside STL and thus are likely to be correct.
I think vg_replace_malloc.c is coming from valgrind, isn't it?


Peter

Mogan : Hey Bruno. What are you? A 65?
Bruno : Model 63. Deluxe.
Kelp : 63? Talk about ancient.
Bruno : Your sister didn't seem to mind.

[Valgrind-users] Mismatched free() / delete / delete[] -- false positives

[Foelsche, P. <Pet...@me...>]

*         Valgrind-3.13.0

*         CentOS release 6.5 (Final)

*         g++ (GCC) 5.3.0

*         no usage of tcmalloc
I checked by
cat /proc/<PID>/maps |sed -e 's/[a-zA-Z0-9_:-]*[ \t][ \t]*//g'|sort -u|xargs nm -A|grep '\<free\>'|grep -v '\<U\>'
output was
/lib64/ld-2.12.so:0000003bfba16090 W free
/lib64/libc-2.12.so:0000003bfc27b520 T free
/path/executable:000000000f9f3c50 W free

*         positives for both directions

o   free used for memory allocated with new

o   delete used for memory allocated with malloc

1)
==11679== Mismatched free() / delete / delete []
==11679==    at 0x127ECAFE: free (vg_replace_malloc.c:530)
==11679==    by 0x1CC871A1: void std::_Destroy<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*) (stl_construct.h:93)
==11679==    by 0x1CC82295: void std::_Destroy_aux<false>::__destroy<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*) (stl_construct.h:103)
...
==11679==  Address 0x2633aeb0 is 0 bytes inside a block of size 155 alloc'd
==11679==    at 0x127EDBE7: operator new(unsigned long) (vg_replace_malloc.c:334)
==11679==    by 0x54B81C3: void std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_construct<char*>(char*, char*, std::forwar
d_iterator_tag) (basic_string.tcc:223)

2)
==11679== Mismatched free() / delete / delete []
==11679==    at 0x127EC64E: operator delete(void*) (vg_replace_malloc.c:576)
==11679==    by 0x8172EC3: __gnu_cxx::new_allocator<Sdi::StateVar const*>::deallocate(Sdi::StateVar const**, unsigned long) (new_allocator.h:110)
==11679==    by 0x8172767: std::allocator_traits<std::allocator<Sdi::StateVar const*> >::deallocate(std::allocator<Sdi::StateVar const*>&, Sdi::StateVar const**, unsigned long) (alloc_traits.h:386)
...
==11679==  Address 0x2a5e08b0 is 0 bytes inside a block of size 32 alloc'd
==11679==    at 0x127ED104: malloc (vg_replace_malloc.c:299)
==11679==    by 0x206F33E7: operator new(unsigned long) (new_op.cc:50)
==11679==    by 0x1CCE9829: __gnu_cxx::new_allocator<Sdi::StateVar const*>::allocate(unsigned long, void const*) (new_allocator.h:104)


There are only a 7 of the 1) messages and 3 of the 2) despite there are many more such allocations.
As you can see, both places are inside STL and thus are likely to be correct.
I think vg_replace_malloc.c is coming from valgrind, isn't it?


Peter

Mogan : Hey Bruno. What are you? A 65?
Bruno : Model 63. Deluxe.
Kelp : 63? Talk about ancient.
Bruno : Your sister didn't seem to mind.

[Valgrind-users] memory issue.

[Jay N. <nja...@gm...>]

Hi,

I have a program, which i am monitoring with linux top command(CentOS). The
virt memory remains the same. There is increase IN RES memory and SHR
memory. How can i analyze the rise in RES and shr memory with valgrind. How
does valgrind points out the increase in RES and SHR memory.

Please let me know.

Thanks,
Jayshankar

[Valgrind-users] Valgrind tool for dead stores.

[John B. <all...@gm...>]

I found this blog post interesting enough to share.
https://kristerw.blogspot.com/2020/02/watching-for-software-inefficiencies.html

It's post on writing a small valgrind tool for looking for dead stores.

Re: [Valgrind-users] Original Path

[John R. <jr...@bi...>]

On 7/20/20 23:14 UTC, Chithkala Dhulipati wrote:

> I wanted to use valgrind for profiling my programs. What I am looking for is a list of functions called and a path to those functions. When I previously used gprof for this, there was no option that could produce the original path where the functions where found. Is there a way I can achieve this?

The terms "path to those functions" and "original path where the functions were found"
are not well-defined.  Please provide actual literal examples of what you seek,
as well as an explanation of how the existing valgrind tracebacks are not sufficient.

[Valgrind-users] Original Path

[Chithkala D. <chithkaladhulipa@u.boisestate.edu>]

Hello!

I wanted to use valgrind for profiling my programs. What I am looking for
is a list of functions called and a path to those functions. When I
previously used gprof for this, there was no option that could produce the
original path where the functions where found. Is there a way I can achieve
this?

Thanks!
Chithkala