|
From: Tanmay M. <tan...@ho...> - 2012-07-06 09:52:45
|
Hi.
I'm getting an invalid read of size 8 warning with the code (check.c) below.Everything seems to be fine, and the warning about reading beyond the allocated source pointer comes only when I'm copying the source to a particular index of the destination.
My environment:- MacOSX Darwin 10.8.0 Darwin Kernel Version 10.8.0- valgrind-3.7.0- gcc version: i686-apple-darwin10-gcc-4.2.1 (GCC) 4.2.1 (Apple Inc. build 5666) (dot 3)
Any clues on why this could happen?
Thanks.- tan
check.c-------------------------------------------------------------------#include <stdio.h>#include <stdlib.h>#include <pthread.h>#include <assert.h>#include <time.h>#include <unistd.h>#include <string.h>
#define NUM_E 3
typedef struct mvk_lruc_kv { long l1; long l2; char c[129];} mvk_lruc_kv_t;
int main(void) { mvk_lruc_kv_t *p1 = malloc(NUM_E * sizeof(mvk_lruc_kv_t)); mvk_lruc_kv_t *p2 = malloc(30 * sizeof(mvk_lruc_kv_t));
printf("sizeof(mvk_lruc_kv_t)=%ld, %p, %p, %p, %p\n", sizeof(mvk_lruc_kv_t), p2, p2+1, p2+2, p1);
memset(p1, 0, sizeof(mvk_lruc_kv_t));
int i = 0; for(i=0; i<(30-NUM_E); i++) { printf("at index %d\n", i); memcpy(p2+i, p1, NUM_E*sizeof(mvk_lruc_kv_t)); printf("end at index %d\n", i); }
return i;}
------------------------------------------------------------ |
|
From: John R. <jr...@bi...> - 2012-07-06 15:15:45
|
> I'm getting an invalid read of size 8 warning with the code (check.c) below. > Everything seems to be fine, and the warning about reading beyond the allocated source pointer comes only when I'm copying the source to a particular index of the destination. > Please copy+paste the exact error message and traceback. The pc matters: your program, memcheck's interceptor, or un-intercepted libc instructions. Also, the addresses matter: the low-order 5 bits of each of the source, destination, and length. > My environment: > - MacOSX Darwin 10.8.0 Darwin Kernel Version 10.8.0 > - valgrind-3.7.0 > - gcc version: i686-apple-darwin10-gcc-4.2.1 (GCC) 4.2.1 (Apple Inc. build 5666) (dot 3) Thank you for reporting those versions! Also, which version of libc are you using? Re-run with "valgrind -v ./check". Check the suppressions file --1658-- Reading suppressions file: /usr/lib64/valgrind/default.supp to see which mem* reports are being suppressed, and by which version of the suppressions master file(s). Pay particular attention to all REDIR which refer to mem* functions: --1658-- REDIR: 0x363b88e690 (memcpy@@GLIBC_2.14) redirected to 0x4802700 (_vgnU_ifunc_wrapper) --1658-- REDIR: 0x363b944b40 (__memcpy_ssse3_back) redirected to 0x4a09ff0 (memcpy@@GLIBC_2.14) memcheck gives me no complaints when running your check.c on Fedora 17: Linux host.domain 3.4.3-1.fc17.x86_64 #1 SMP Mon Jun 18 19:53:17 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux Using Valgrind-3.7.0 gcc (GCC) 4.7.0 20120507 (Red Hat 4.7.0-5) glibc-2.15-37.fc17.x86_64 -- |
|
From: Tanmay M. <tan...@ho...> - 2012-07-06 17:59:10
|
Thanks John! The libc shared library does not seem to mention any version number. I'm on MacOs 10.6.8 on Intel. How do I get the version of libc?Here's what I get from valgrind -v: ---------------------------------------------------------==57528== Memcheck, a memory error detector==57528== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al.==57528== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info==57528== Command: ./check==57528== --57528-- Valgrind options:--57528-- -v--57528-- --dsymutil=yes--57528-- Contents of /proc/version:--57528-- can't open /proc/version--57528-- Arch and hwcaps: AMD64, amd64-sse3-cx16--57528-- Page sizes: currently 4096, max supported 4096--57528-- Valgrind library directory: /Users/tan/Tools/valgrind/install_dir/lib/valgrind--57528-- ./check (0x100000000)--57528-- reading syms from primary file (7 4)--57528-- run: /usr/bin/dsymutil "./check"warning: no debug symbols in executable (-arch x86_64)--57528-- dsyms= ./check.dSYM/Contents/Resources/DWARF/check--57528-- /usr/lib/dyld (0x7fff5fc00000)--57528-- reading syms from primary file (6 1186)--57528-- Reading suppressions file: /Users/tan/Tools/valgrind/install_dir/lib/valgrind/default.supp==57528== embedded gdbserver: reading from /var/folders/In/In0K1bIIGmOe2IoRTx-hcE+++TM/-Tmp-//vgdb-pipe-from-vgdb-to-57528-by-tan-on-???==57528== embedded gdbserver: writing to /var/folders/In/In0K1bIIGmOe2IoRTx-hcE+++TM/-Tmp-//vgdb-pipe-to-vgdb-from-57528-by-tan-on-???==57528== embedded gdbserver: shared mem /var/folders/In/In0K1bIIGmOe2IoRTx-hcE+++TM/-Tmp-//vgdb-pipe-shared-mem-vgdb-57528-by-tan-on-???==57528== ==57528== TO CONTROL THIS PROCESS USING vgdb (which you probably==57528== don't want to do, unless you know exactly what you're doing,==57528== or are doing some strange experiment):==57528== /Users/tan/Tools/valgrind/install_dir/lib/valgrind/../../bin/vgdb --pid=57528 ...command...==57528== ==57528== TO DEBUG THIS PROCESS USING GDB: start GDB like this==57528== /path/to/gdb ./check==57528== and then give GDB the following command==57528== target remote | /Users/tan/Tools/valgrind/install_dir/lib/valgrind/../../bin/vgdb --pid=57528==57528== --pid is optional if only one valgrind process is running==57528== --57528-- REDIR: 0x7fff5fc22fb0 (strcmp) redirected to 0x13804cb90 (???)--57528-- REDIR: 0x7fff5fc20693 (arc4random) redirected to 0x13804cc2e (???)--57528-- REDIR: 0x7fff5fc22e90 (strlen) redirected to 0x13804cb5f (???)--57528-- REDIR: 0x7fff5fc22ee0 (strcpy) redirected to 0x13804cbac (???)--57528-- REDIR: 0x7fff5fc2306f (strcat) redirected to 0x13804cb70 (???)--57528-- /Users/tan/Tools/valgrind/install_dir/lib/valgrind/vgpreload_core-amd64-darwin.so (0x100004000)--57528-- reading syms from primary file (3 135)--57528-- dSYM= /Users/tan/Tools/valgrind/install_dir/lib/valgrind/vgpreload_core-amd64-darwin.so.dSYM/Contents/Resources/DWARF/vgpreload_core-amd64-darwin.so--57528-- reading dwarf3 from dsyms file--57528-- /Users/tan/Tools/valgrind/install_dir/lib/valgrind/vgpreload_memcheck-amd64-darwin.so (0x10000f000)--57528-- reading syms from primary file (32 273)--57528-- dSYM= /Users/tan/Tools/valgrind/install_dir/lib/valgrind/vgpreload_memcheck-amd64-darwin.so.dSYM/Contents/Resources/DWARF/vgpreload_memcheck-amd64-darwin.so--57528-- reading dwarf3 from dsyms file--57528-- /usr/lib/libSystem.B.dylib (0x10001d000)--57528-- reading syms from primary file (4606 3793)--57528-- REDIR: 0x10001deb4 (memset) redirected to 0x100010c80 (memset)--57528-- REDIR: 0x10001fc5c (malloc) redirected to 0x1000105f7 (malloc)--57528-- REDIR: 0x100020bf0 (strlen) redirected to 0x100010af0 (strlen)--57528-- REDIR: 0x100020280 (strncmp) redirected to 0x100010be0 (strncmp)sizeof(mvk_lruc_kv_t)=152, 0x1002772f0, 0x100277388, 0x100277420, 0x1002770e0at index 0end at index 0at index 1==57528== Invalid read of size 8==57528== at 0x7FFFFFE00BAC: ???==57528== by 0x100000E4D: __inline_memcpy_chk (in ./check)==57528== by 0x100000DCF: main (in ./check)==57528== Address 0x1002772a8 is 0 bytes after a block of size 456 alloc'd==57528== at 0x100010679: malloc (vg_replace_malloc.c:266)==57528== by 0x100000CC5: main (in ./check)==57528== end at index 1at index 2end at index 2at index 3-------------------------- some more similar printf outputs pruned here --------------------------------at index 26end at index 26==57528== ==57528== HEAP SUMMARY:==57528== in use at exit: 9,200 bytes in 4 blocks==57528== total heap usage: 4 allocs, 0 frees, 9,200 bytes allocated==57528== ==57528== Searching for pointers to 4 not-freed blocks==57528== Checked 415,728 bytes==57528== ==57528== LEAK SUMMARY:==57528== definitely lost: 5,016 bytes in 2 blocks==57528== indirectly lost: 0 bytes in 0 blocks==57528== possibly lost: 0 bytes in 0 blocks==57528== still reachable: 4,096 bytes in 1 blocks==57528== suppressed: 88 bytes in 1 blocks==57528== Rerun with --leak-check=full to see details of leaked memory==57528== ==57528== ERROR SUMMARY: 13 errors from 1 contexts (suppressed: 0 from 0)==57528== ==57528== 13 errors in context 1 of 1:==57528== Invalid read of size 8==57528== at 0x7FFFFFE00BAC: ???==57528== by 0x100000E4D: __inline_memcpy_chk (in ./check)==57528== by 0x100000DCF: main (in ./check)==57528== Address 0x1002772a8 is 0 bytes after a block of size 456 alloc'd==57528== at 0x100010679: malloc (vg_replace_malloc.c:266)==57528== by 0x100000CC5: main (in ./check)==57528== --57528-- --57528-- used_suppression: 1 libSystem-keymgr-leak-at-exit==57528== ==57528== ERROR SUMMARY: 13 errors from 1 contexts (suppressed: 0 from 0) --------------------------------------------------------------------------------- > Date: Fri, 6 Jul 2012 08:16:33 -0700 > From: jr...@bi... > To: val...@li... > Subject: Re: [Valgrind-users] Strange warning for invalid read of size 8 in memcpy > > > I'm getting an invalid read of size 8 warning with the code (check.c) below. > > Everything seems to be fine, and the warning about reading beyond the allocated source pointer comes only when I'm copying the source to a particular index of the destination. > > > > Please copy+paste the exact error message and traceback. The pc matters: > your program, memcheck's interceptor, or un-intercepted libc instructions. > Also, the addresses matter: the low-order 5 bits of each of the source, > destination, and length. > > > My environment: > > - MacOSX Darwin 10.8.0 Darwin Kernel Version 10.8.0 > > - valgrind-3.7.0 > > - gcc version: i686-apple-darwin10-gcc-4.2.1 (GCC) 4.2.1 (Apple Inc. build 5666) (dot 3) > > Thank you for reporting those versions! Also, which version of libc are you using? > > Re-run with "valgrind -v ./check". Check the suppressions file > --1658-- Reading suppressions file: /usr/lib64/valgrind/default.supp > to see which mem* reports are being suppressed, and by which version of > the suppressions master file(s). > Pay particular attention to all REDIR which refer to mem* functions: > --1658-- REDIR: 0x363b88e690 (memcpy@@GLIBC_2.14) redirected to 0x4802700 (_vgnU_ifunc_wrapper) > --1658-- REDIR: 0x363b944b40 (__memcpy_ssse3_back) redirected to 0x4a09ff0 (memcpy@@GLIBC_2.14) > > memcheck gives me no complaints when running your check.c on Fedora 17: > Linux host.domain 3.4.3-1.fc17.x86_64 #1 SMP Mon Jun 18 19:53:17 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux > Using Valgrind-3.7.0 > gcc (GCC) 4.7.0 20120507 (Red Hat 4.7.0-5) > glibc-2.15-37.fc17.x86_64 > > -- > > ------------------------------------------------------------------------------ > Live Security Virtual Conference > Exclusive live event will cover all the ways today's security and > threat landscape has changed and how IT managers can respond. Discussions > will include endpoint security, mobile security and the latest in malware > threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ > _______________________________________________ > Valgrind-users mailing list > Val...@li... > https://lists.sourceforge.net/lists/listinfo/valgrind-users |
|
From: John R. <jr...@bi...> - 2012-07-06 18:44:46
|
(Note no REDIR of memcpy. Alas.) > sizeof(mvk_lruc_kv_t)=152, 0x1002772f0, 0x100277388, 0x100277420, 0x1002770e0 > at index 0 > end at index 0 > at index 1 > ==57528== Invalid read of size 8 > ==57528== at 0x7FFFFFE00BAC: ??? > ==57528== by 0x100000E4D: __inline_memcpy_chk (in ./check) > ==57528== by 0x100000DCF: main (in ./check) > ==57528== Address 0x1002772a8 is 0 bytes after a block of size 456 alloc'd > ==57528== at 0x100010679: malloc (vg_replace_malloc.c:266) > ==57528== by 0x100000CC5: main (in ./check) The compiler has decided to generate inline code for memcpy. This is hard for memcheck to recognize as memcpy. What is the code? Start with this, then adjust as appropriate: $ gdb ./check (gdb) x/15i 0x100000E4D - 0x18 -- |
|
From: Patrick J. L. <lop...@gm...> - 2012-07-06 22:38:04
|
On Fri, Jul 6, 2012 at 11:45 AM, John Reiser <jr...@bi...> wrote: > > The compiler has decided to generate inline code for memcpy. > This is hard for memcheck to recognize as memcpy. But memcheck does not have to recognize it at all if you use "--partial-loads-ok=yes". The problem is that the default memory model in memcheck does not reflect the way modern machines (and compilers) work. The inlined memcpy() is reading undefined data past the end of an allocated block, but it is doing so with an aligned load and it is not propagating the undefined data further. So the code is perfectly valid and memcheck's error report is simply wrong. The good news is that you can fix memcheck's memory model by specifying "--partial-loads-ok=yes". (As I have said before, I am of the opinion that this quite obviously should be the default.) The bad news is that (a) it is broken for SSE loads (https://bugs.kde.org/show_bug.cgi?id=294285) and (b) memcheck's validity bit propagation may be insufficiently precise to suppress the false positive. Both of these problems are fixable and would make Valgrind/memcheck much more useful on this sort of optimized code, which is clearly becoming increasingly common as compilers get smarter. - Pat |
|
From: John R. <jr...@bi...> - 2012-07-07 00:26:28
|
On 07/06/2012 03:37 PM, Patrick J. LoPresti wrote: > On Fri, Jul 6, 2012 at 11:45 AM, John Reiser <jr...@bi...> wrote: >> >> The compiler has decided to generate inline code for memcpy. >> This is hard for memcheck to recognize as memcpy. > > But memcheck does not have to recognize it at all if you use > "--partial-loads-ok=yes". > > The problem is that the default memory model in memcheck does not > reflect the way modern machines (and compilers) work. The inlined > memcpy() is reading undefined data past the end of an allocated block, > but it is doing so with an aligned load and it is not propagating the > undefined data further. So the code is perfectly valid and memcheck's > error report is simply wrong. > > The good news is that you can fix memcheck's memory model by > specifying "--partial-loads-ok=yes". (As I have said before, I am of > the opinion that this quite obviously should be the default.) The bad > news is that (a) it is broken for SSE loads > (https://bugs.kde.org/show_bug.cgi?id=294285) and (b) memcheck's > validity bit propagation may be insufficiently precise to suppress the > false positive. In general it may be true that --partial-loads-ok=yes is a better default. However in this case, something else looks like a bigger culprit, and the use of --partial-loads-ok=yes as default results in a substantial performance penalty, as well as less error checking by memcheck. The bigger culprit is that some portions of Mac OS X 10.8.0, in particular some portions of XCode 4.3.2 (4E2002), have changed the name from 'memcpy_chk' to '__memcpy_chk', and from 'inline_memcpy_chk' to '__inline_memcpy_chk'. [In the Debug style of XCode, the "inline" are actual out-of-line subroutines.] The use of the double underscore prefix prevents memcheck's usual redirection of those routines. The memcheck source code needs to be enhanced to recognize the new names as well as the old. Because __memcpy_chk and __inline_memcpy_chk are not intercepted as whole routines, then memcheck must emulate them instruction-by-instruction. This is slow, and omits the checks for overlapping ranges. (These OS X libsystem_c.dylib *_chk routines check for stack smashing, but not for overlap of source and destination.) -- |
|
From: Tanmay M. <tan...@ho...> - 2012-07-08 04:58:18
|
Thanks John and Pat. So what I understand is I can ignore the warning in this case.But, I still get the warning even with --partial-loads-ok=yes. Why so? - Tan Below is the output with --partial-loads-ok=yes:--------------------------------------------------------------------------------==67529== Memcheck, a memory error detector==67529== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al.==67529== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info==67529== Command: ./check==67529== --67529-- Valgrind options:--67529-- -v--67529-- --dsymutil=yes--67529-- --partial-loads-ok=yes--67529-- Contents of /proc/version:--67529-- can't open /proc/version--67529-- Arch and hwcaps: AMD64, amd64-sse3-cx16--67529-- Page sizes: currently 4096, max supported 4096--67529-- Valgrind library directory: /Users/tan/Tools/valgrind/install_dir/lib/valgrind--67529-- ./check (0x100000000)--67529-- reading syms from primary file (7 4)--67529-- dSYM= ./check.dSYM/Contents/Resources/DWARF/check--67529-- /usr/lib/dyld (0x7fff5fc00000)--67529-- reading syms from primary file (6 1186)--67529-- Reading suppressions file: /Users/tan/Tools/valgrind/install_dir/lib/valgrind/default.supp==67529== embedded gdbserver: reading from /var/folders/In/In0K1bIIGmOe2IoRTx-hcE+++TM/-Tmp-//vgdb-pipe-from-vgdb-to-67529-by-tan-on-???==67529== embedded gdbserver: writing to /var/folders/In/In0K1bIIGmOe2IoRTx-hcE+++TM/-Tmp-//vgdb-pipe-to-vgdb-from-67529-by-tan-on-???==67529== embedded gdbserver: shared mem /var/folders/In/In0K1bIIGmOe2IoRTx-hcE+++TM/-Tmp-//vgdb-pipe-shared-mem-vgdb-67529-by-tan-on-???==67529== ==67529== TO CONTROL THIS PROCESS USING vgdb (which you probably==67529== don't want to do, unless you know exactly what you're doing,==67529== or are doing some strange experiment):==67529== /Users/tan/Tools/valgrind/install_dir/lib/valgrind/../../bin/vgdb --pid=67529 ...command...==67529== ==67529== TO DEBUG THIS PROCESS USING GDB: start GDB like this==67529== /path/to/gdb ./check==67529== and then give GDB the following command==67529== target remote | /Users/tan/Tools/valgrind/install_dir/lib/valgrind/../../bin/vgdb --pid=67529==67529== --pid is optional if only one valgrind process is running==67529== --67529-- REDIR: 0x7fff5fc22fb0 (strcmp) redirected to 0x13804cb90 (???)--67529-- REDIR: 0x7fff5fc20693 (arc4random) redirected to 0x13804cc2e (???)--67529-- REDIR: 0x7fff5fc22e90 (strlen) redirected to 0x13804cb5f (???)--67529-- REDIR: 0x7fff5fc22ee0 (strcpy) redirected to 0x13804cbac (???)--67529-- REDIR: 0x7fff5fc2306f (strcat) redirected to 0x13804cb70 (???)--67529-- /Users/tan/Tools/valgrind/install_dir/lib/valgrind/vgpreload_core-amd64-darwin.so (0x100004000)--67529-- reading syms from primary file (3 135)--67529-- dSYM= /Users/tan/Tools/valgrind/install_dir/lib/valgrind/vgpreload_core-amd64-darwin.so.dSYM/Contents/Resources/DWARF/vgpreload_core-amd64-darwin.so--67529-- reading dwarf3 from dsyms file--67529-- /Users/tan/Tools/valgrind/install_dir/lib/valgrind/vgpreload_memcheck-amd64-darwin.so (0x10000f000)--67529-- reading syms from primary file (32 273)--67529-- dSYM= /Users/tan/Tools/valgrind/install_dir/lib/valgrind/vgpreload_memcheck-amd64-darwin.so.dSYM/Contents/Resources/DWARF/vgpreload_memcheck-amd64-darwin.so--67529-- reading dwarf3 from dsyms file--67529-- /usr/lib/libSystem.B.dylib (0x10001d000)--67529-- reading syms from primary file (4606 3793)--67529-- REDIR: 0x10001deb4 (memset) redirected to 0x100010c80 (memset)--67529-- REDIR: 0x10001fc5c (malloc) redirected to 0x1000105f7 (malloc)--67529-- REDIR: 0x100020bf0 (strlen) redirected to 0x100010af0 (strlen)--67529-- REDIR: 0x100020280 (strncmp) redirected to 0x100010be0 (strncmp)sizeof(mvk_lruc_kv_t)=152, 0x1002772f0, 0x100277388, 0x100277420, 0x1002770e0at index 0end at index 0at index 1==67529== Invalid read of size 8==67529== at 0x7FFFFFE00BAC: ???==67529== by 0x100000E4D: __inline_memcpy_chk (in ./check)==67529== by 0x100000DCF: main (in ./check)==67529== Address 0x1002772a8 is 0 bytes after a block of size 456 alloc'd==67529== at 0x100010679: malloc (vg_replace_malloc.c:266)==67529== by 0x100000CC5: main (in ./check)==67529== end at index 1.....-------------------------------------------------------------------------------- > Date: Fri, 6 Jul 2012 17:27:29 -0700 > From: jr...@bi... > To: val...@li... > Subject: Re: [Valgrind-users] Strange warning for invalid read of size 8 in memcpy > > On 07/06/2012 03:37 PM, Patrick J. LoPresti wrote: > > On Fri, Jul 6, 2012 at 11:45 AM, John Reiser <jr...@bi...> wrote: > >> > >> The compiler has decided to generate inline code for memcpy. > >> This is hard for memcheck to recognize as memcpy. > > > > But memcheck does not have to recognize it at all if you use > > "--partial-loads-ok=yes". > > > > The problem is that the default memory model in memcheck does not > > reflect the way modern machines (and compilers) work. The inlined > > memcpy() is reading undefined data past the end of an allocated block, > > but it is doing so with an aligned load and it is not propagating the > > undefined data further. So the code is perfectly valid and memcheck's > > error report is simply wrong. > > > > The good news is that you can fix memcheck's memory model by > > specifying "--partial-loads-ok=yes". (As I have said before, I am of > > the opinion that this quite obviously should be the default.) The bad > > news is that (a) it is broken for SSE loads > > (https://bugs.kde.org/show_bug.cgi?id=294285) and (b) memcheck's > > validity bit propagation may be insufficiently precise to suppress the > > false positive. > > In general it may be true that --partial-loads-ok=yes is a better default. > > However in this case, something else looks like a bigger culprit, and the > use of --partial-loads-ok=yes as default results in a substantial performance > penalty, as well as less error checking by memcheck. > > The bigger culprit is that some portions of Mac OS X 10.8.0, in particular > some portions of XCode 4.3.2 (4E2002), have changed the name from 'memcpy_chk' > to '__memcpy_chk', and from 'inline_memcpy_chk' to '__inline_memcpy_chk'. > [In the Debug style of XCode, the "inline" are actual out-of-line > subroutines.] The use of the double underscore prefix prevents > memcheck's usual redirection of those routines. The memcheck source > code needs to be enhanced to recognize the new names as well as the old. > > Because __memcpy_chk and __inline_memcpy_chk are not intercepted as whole > routines, then memcheck must emulate them instruction-by-instruction. > This is slow, and omits the checks for overlapping ranges. (These OS X > libsystem_c.dylib *_chk routines check for stack smashing, but not for > overlap of source and destination.) > > -- > > > ------------------------------------------------------------------------------ > Live Security Virtual Conference > Exclusive live event will cover all the ways today's security and > threat landscape has changed and how IT managers can respond. Discussions > will include endpoint security, mobile security and the latest in malware > threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ > _______________________________________________ > Valgrind-users mailing list > Val...@li... > https://lists.sourceforge.net/lists/listinfo/valgrind-users |
|
From: John R. <jr...@bi...> - 2012-07-08 23:51:57
|
> But, I still get the warning even with --partial-loads-ok=yes. Why so? I cannot reproduce that (in particular, I get no complaint from memcheck) with a freshly-built valgrind-3.7.0 in the environment: Darwin host.local 11.4.0 Darwin Kernel Version 11.4.0: Mon Apr 9 19:32:15 PDT 2012; root:xnu-1699.26.8~1/RELEASE_X86_64 x86_64 i686-apple-darwin10-gcc-4.2.1 (GCC) 4.2.1 (Apple Inc. build 5646) About This Mac > More Info... > System Report... > Software > Developer 3.2 (10A432) /Developer 4.3.2 (4E2002) /Applications/Xcode.app ln -s /Developer/SDKs/MacOSX10.6.sdk/usr/include /usr/include CC='gcc -L/Developer/SDKs/MacOSX10.6.sdk/usr/lib' $ valgrind -v ./check ## note no '--partial-loads-ok=yes' ==24112== Memcheck, a memory error detector ==24112== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al. ==24112== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info ==24112== Command: ./check ==24112== --24112-- Valgrind options: --24112-- -v --24112-- Contents of /proc/version: --24112-- can't open /proc/version --24112-- Arch and hwcaps: AMD64, amd64-sse3-cx16 --24112-- Page sizes: currently 4096, max supported 4096 --24112-- Valgrind library directory: /Users/jreiser/local/lib/valgrind --24112-- ./check (0x100000000) --24112-- reading syms from primary file (7 20) --24112-- dSYM directory is missing; consider using --dsymutil=yes --24112-- /usr/lib/dyld (0x7fff5fc00000) --24112-- reading syms from primary file (6 1179) --24112-- Reading suppressions file: /Users/jreiser/local/lib/valgrind/default.supp ==24112== embedded gdbserver: reading from /var/folders/0y/vg59c4zd2p593j05mr_dcqch0000gn/T//vgdb-pipe-from-vgdb-to-24112-by-jreiser-on-??? ==24112== embedded gdbserver: writing to /var/folders/0y/vg59c4zd2p593j05mr_dcqch0000gn/T//vgdb-pipe-to-vgdb-from-24112-by-jreiser-on-??? ==24112== embedded gdbserver: shared mem /var/folders/0y/vg59c4zd2p593j05mr_dcqch0000gn/T//vgdb-pipe-shared-mem-vgdb-24112-by-jreiser-on-??? ==24112== ==24112== TO CONTROL THIS PROCESS USING vgdb (which you probably ==24112== don't want to do, unless you know exactly what you're doing, ==24112== or are doing some strange experiment): ==24112== /Users/jreiser/local/lib/valgrind/../../bin/vgdb --pid=24112 ...command... ==24112== ==24112== TO DEBUG THIS PROCESS USING GDB: start GDB like this ==24112== /path/to/gdb ./check ==24112== and then give GDB the following command ==24112== target remote | /Users/jreiser/local/lib/valgrind/../../bin/vgdb --pid=24112 ==24112== --pid is optional if only one valgrind process is running ==24112== --24112-- REDIR: 0x7fff5fc1dfc0 (strcmp) redirected to 0x13804cba0 (???) --24112-- REDIR: 0x7fff5fc1b161 (arc4random) redirected to 0x13804cc3e (???) --24112-- REDIR: 0x7fff5fc20a50 (strlen) redirected to 0x13804cb6f (???) --24112-- REDIR: 0x7fff5fc1c850 (strcpy) redirected to 0x13804cbbc (???) --24112-- REDIR: 0x7fff5fc19f17 (strcat) redirected to 0x13804cb80 (???) --24112-- /Users/jreiser/local/lib/valgrind/vgpreload_core-amd64-darwin.so (0x100004000) --24112-- reading syms from primary file (3 135) --24112-- dSYM= /Users/jreiser/local/lib/valgrind/vgpreload_core-amd64-darwin.so.dSYM/Contents/Resources/DWARF/vgpreload_core-amd64-darwin.so --24112-- reading dwarf3 from dsyms file --24112-- /Users/jreiser/local/lib/valgrind/vgpreload_memcheck-amd64-darwin.so (0x10000f000) --24112-- reading syms from primary file (32 273) --24112-- dSYM= /Users/jreiser/local/lib/valgrind/vgpreload_memcheck-amd64-darwin.so.dSYM/Contents/Resources/DWARF/vgpreload_memcheck-amd64-darwin.so --24112-- reading dwarf3 from dsyms file --24112-- /usr/lib/libSystem.B.dylib (0x10001d000) --24112-- reading syms from primary file (529 439) --24112-- /usr/lib/system/libcache.dylib (0x100058000) --24112-- reading syms from primary file (28 32) --24112-- /usr/lib/system/libcommonCrypto.dylib (0x100062000) --24112-- reading syms from primary file (96 358) --24112-- /usr/lib/system/libcompiler_rt.dylib (0x1000b2000) --24112-- reading syms from primary file (291 6) --24112-- /usr/lib/system/libcopyfile.dylib (0x1000bf000) --24112-- reading syms from primary file (7 26) --24112-- /usr/lib/system/libdispatch.dylib (0x1000cc000) --24112-- reading syms from primary file (114 431) --24112-- /usr/lib/system/libdnsinfo.dylib (0x1000eb000) --24112-- reading syms from primary file (5 8) --24112-- /usr/lib/system/libdyld.dylib (0x1000f0000) --24112-- reading syms from primary file (68 92) --24112-- /usr/lib/system/libkeymgr.dylib (0x1000fa000) --24112-- reading syms from primary file (12 4) --24112-- /usr/lib/system/liblaunch.dylib (0x1000fe000) --24112-- reading syms from primary file (117 1) --24112-- /usr/lib/system/libmacho.dylib (0x10010f000) --24112-- reading syms from primary file (86 5) --24112-- /usr/lib/system/libquarantine.dylib (0x100121000) --24112-- reading syms from primary file (51 7) --24112-- /usr/lib/system/libremovefile.dylib (0x100128000) --24112-- reading syms from primary file (14 4) --24112-- /usr/lib/system/libsystem_blocks.dylib (0x10012e000) --24112-- reading syms from primary file (27 27) --24112-- /usr/lib/system/libsystem_c.dylib (0x100134000) --24112-- reading syms from primary file (1800 1823) ==24112== Preferring higher priority redirection: --24112-- old: 0x10015bcbd (memcpy$VARIANT$sse42) R-> (2018.0) 0x1000111d0 memcpy$VARIANT$sse42 --24112-- new: 0x10015bcbd (memcpy$VARIANT$sse42) R-> (2018.1) 0x100010ed0 memmove$VARIANT$sse42 ==24112== Preferring higher priority redirection: --24112-- old: 0x10015bebd (memcpy$VARIANT$sse3x) R-> (2018.0) 0x100011410 memcpy$VARIANT$sse3x --24112-- new: 0x10015bebd (memcpy$VARIANT$sse3x) R-> (2018.1) 0x100010d10 memmove$VARIANT$sse3x --24112-- /usr/lib/system/libsystem_dnssd.dylib (0x100250000) --24112-- reading syms from primary file (64 33) --24112-- /usr/lib/system/libsystem_info.dylib (0x10025e000) --24112-- reading syms from primary file (689 510) --24112-- /usr/lib/system/libsystem_kernel.dylib (0x1002af000) --24112-- reading syms from primary file (767 72) --24112-- /usr/lib/system/libsystem_network.dylib (0x1002df000) --24112-- reading syms from primary file (23 100) --24112-- /usr/lib/system/libsystem_notify.dylib (0x1002eb000) --24112-- reading syms from primary file (115 55) --24112-- /usr/lib/system/libsystem_sandbox.dylib (0x1002fb000) --24112-- reading syms from primary file (41 7) --24112-- /usr/lib/system/libunc.dylib (0x100301000) --24112-- reading syms from primary file (30 4) --24112-- /usr/lib/system/libunwind.dylib (0x100307000) --24112-- reading syms from primary file (102 67) --24112-- /usr/lib/system/libxpc.dylib (0x100314000) --24112-- reading syms from primary file (523 218) --24112-- REDIR: 0x1001d55e0 (memset) redirected to 0x100010c90 (memset) --24112-- REDIR: 0x1001d64e0 (strlen) redirected to 0x100010b00 (strlen) --24112-- REDIR: 0x1001d5178 (malloc) redirected to 0x100010607 (malloc) --24112-- REDIR: 0x1001d512d (calloc) redirected to 0x1000102ff (calloc) --24112-- REDIR: 0x10015bcbd (memcpy$VARIANT$sse42) redirected to 0x100010ed0 (memmove$VARIANT$sse42) --24112-- REDIR: 0x1001d5820 (strcmp) redirected to 0x100010c50 (strcmp) --24112-- REDIR: 0x1001d63f0 (strncmp) redirected to 0x100010bf0 (strncmp) sizeof(mvk_lruc_kv_t)=152, 0x1003462e0, 0x100346378, 0x100346410, 0x1003460d0 at index 0 end at index 0 at index 1 end at index 1 at index 2 end at index 2 at index 3 end at index 3 at index 4 end at index 4 at index 5 end at index 5 at index 6 end at index 6 at index 7 end at index 7 at index 8 end at index 8 at index 9 end at index 9 at index 10 end at index 10 at index 11 end at index 11 at index 12 end at index 12 at index 13 end at index 13 at index 14 end at index 14 at index 15 end at index 15 at index 16 end at index 16 at index 17 end at index 17 at index 18 end at index 18 at index 19 end at index 19 at index 20 end at index 20 at index 21 end at index 21 at index 22 end at index 22 at index 23 end at index 23 at index 24 end at index 24 at index 25 end at index 25 at index 26 end at index 26 ==24112== ==24112== HEAP SUMMARY: ==24112== in use at exit: 11,123 bytes in 35 blocks ==24112== total heap usage: 35 allocs, 0 frees, 11,123 bytes allocated ==24112== ==24112== Searching for pointers to 35 not-freed blocks ==24112== Checked 486,912 bytes ==24112== ==24112== LEAK SUMMARY: ==24112== definitely lost: 5,016 bytes in 2 blocks ==24112== indirectly lost: 0 bytes in 0 blocks ==24112== possibly lost: 0 bytes in 0 blocks ==24112== still reachable: 6,107 bytes in 33 blocks ==24112== suppressed: 0 bytes in 0 blocks ==24112== Rerun with --leak-check=full to see details of leaked memory ==24112== ==24112== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 1 from 1) --24112-- --24112-- used_suppression: 1 OSX107:blah ==24112== ==24112== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 1 from 1) $ -- |
|
From: Tanmay M. <tan...@ho...> - 2012-07-09 04:37:19
|
Hmm...I think there's a difference in the environment. For me:About This Mac > More Info... > System Report... > Software > Developer: Version: 3.2 (10M2518) Location: /Developer Applications: Xcode: 3.2.6 (1761) Interface Builder: 3.2.6 (851) Instruments: 2.7 (3017) Dashcode: 3.0.2 (333) SDKs: Mac OS X: 10.5: (9L31a) 10.6: (10M2518) and for you memcpy is redirected differently:> --24112-- REDIR: 0x10015bcbd (memcpy$VARIANT$sse42) redirected to 0x100010ed0 (memmove$VARIANT$sse42) In any case... pardon my questions... I'm still a newbie in this.But can you please explain the discussions between you and Pat? In particular...- Why would memcpy need to read 8 bytes more?- And why would the warning depend on the destination? That is, why am I getting this only in the 2nd iteration and not any other time? Thanks.- Tan > Date: Sun, 8 Jul 2012 16:52:57 -0700 > From: jr...@bi... > To: val...@li... > Subject: Re: [Valgrind-users] Strange warning for invalid read of size 8 in memcpy > > > But, I still get the warning even with --partial-loads-ok=yes. Why so? > > I cannot reproduce that (in particular, I get no complaint from memcheck) > with a freshly-built valgrind-3.7.0 in the environment: > Darwin host.local 11.4.0 Darwin Kernel Version 11.4.0: Mon Apr 9 19:32:15 PDT 2012; root:xnu-1699.26.8~1/RELEASE_X86_64 x86_64 > i686-apple-darwin10-gcc-4.2.1 (GCC) 4.2.1 (Apple Inc. build 5646) > About This Mac > More Info... > System Report... > Software > Developer > 3.2 (10A432) /Developer > 4.3.2 (4E2002) /Applications/Xcode.app > ln -s /Developer/SDKs/MacOSX10.6.sdk/usr/include /usr/include > CC='gcc -L/Developer/SDKs/MacOSX10.6.sdk/usr/lib' > > $ valgrind -v ./check ## note no '--partial-loads-ok=yes' > ==24112== Memcheck, a memory error detector > ==24112== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al. > ==24112== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info > ==24112== Command: ./check > ==24112== > --24112-- Valgrind options: > --24112-- -v > --24112-- Contents of /proc/version: > --24112-- can't open /proc/version > --24112-- Arch and hwcaps: AMD64, amd64-sse3-cx16 > --24112-- Page sizes: currently 4096, max supported 4096 > --24112-- Valgrind library directory: /Users/jreiser/local/lib/valgrind > --24112-- ./check (0x100000000) > --24112-- reading syms from primary file (7 20) > --24112-- dSYM directory is missing; consider using --dsymutil=yes > --24112-- /usr/lib/dyld (0x7fff5fc00000) > --24112-- reading syms from primary file (6 1179) > --24112-- Reading suppressions file: /Users/jreiser/local/lib/valgrind/default.supp > ==24112== embedded gdbserver: reading from /var/folders/0y/vg59c4zd2p593j05mr_dcqch0000gn/T//vgdb-pipe-from-vgdb-to-24112-by-jreiser-on-??? > ==24112== embedded gdbserver: writing to /var/folders/0y/vg59c4zd2p593j05mr_dcqch0000gn/T//vgdb-pipe-to-vgdb-from-24112-by-jreiser-on-??? > ==24112== embedded gdbserver: shared mem /var/folders/0y/vg59c4zd2p593j05mr_dcqch0000gn/T//vgdb-pipe-shared-mem-vgdb-24112-by-jreiser-on-??? > ==24112== > ==24112== TO CONTROL THIS PROCESS USING vgdb (which you probably > ==24112== don't want to do, unless you know exactly what you're doing, > ==24112== or are doing some strange experiment): > ==24112== /Users/jreiser/local/lib/valgrind/../../bin/vgdb --pid=24112 ...command... > ==24112== > ==24112== TO DEBUG THIS PROCESS USING GDB: start GDB like this > ==24112== /path/to/gdb ./check > ==24112== and then give GDB the following command > ==24112== target remote | /Users/jreiser/local/lib/valgrind/../../bin/vgdb --pid=24112 > ==24112== --pid is optional if only one valgrind process is running > ==24112== > --24112-- REDIR: 0x7fff5fc1dfc0 (strcmp) redirected to 0x13804cba0 (???) > --24112-- REDIR: 0x7fff5fc1b161 (arc4random) redirected to 0x13804cc3e (???) > --24112-- REDIR: 0x7fff5fc20a50 (strlen) redirected to 0x13804cb6f (???) > --24112-- REDIR: 0x7fff5fc1c850 (strcpy) redirected to 0x13804cbbc (???) > --24112-- REDIR: 0x7fff5fc19f17 (strcat) redirected to 0x13804cb80 (???) > --24112-- /Users/jreiser/local/lib/valgrind/vgpreload_core-amd64-darwin.so (0x100004000) > --24112-- reading syms from primary file (3 135) > --24112-- dSYM= /Users/jreiser/local/lib/valgrind/vgpreload_core-amd64-darwin.so.dSYM/Contents/Resources/DWARF/vgpreload_core-amd64-darwin.so > --24112-- reading dwarf3 from dsyms file > --24112-- /Users/jreiser/local/lib/valgrind/vgpreload_memcheck-amd64-darwin.so (0x10000f000) > --24112-- reading syms from primary file (32 273) > --24112-- dSYM= /Users/jreiser/local/lib/valgrind/vgpreload_memcheck-amd64-darwin.so.dSYM/Contents/Resources/DWARF/vgpreload_memcheck-amd64-darwin.so > --24112-- reading dwarf3 from dsyms file > --24112-- /usr/lib/libSystem.B.dylib (0x10001d000) > --24112-- reading syms from primary file (529 439) > --24112-- /usr/lib/system/libcache.dylib (0x100058000) > --24112-- reading syms from primary file (28 32) > --24112-- /usr/lib/system/libcommonCrypto.dylib (0x100062000) > --24112-- reading syms from primary file (96 358) > --24112-- /usr/lib/system/libcompiler_rt.dylib (0x1000b2000) > --24112-- reading syms from primary file (291 6) > --24112-- /usr/lib/system/libcopyfile.dylib (0x1000bf000) > --24112-- reading syms from primary file (7 26) > --24112-- /usr/lib/system/libdispatch.dylib (0x1000cc000) > --24112-- reading syms from primary file (114 431) > --24112-- /usr/lib/system/libdnsinfo.dylib (0x1000eb000) > --24112-- reading syms from primary file (5 8) > --24112-- /usr/lib/system/libdyld.dylib (0x1000f0000) > --24112-- reading syms from primary file (68 92) > --24112-- /usr/lib/system/libkeymgr.dylib (0x1000fa000) > --24112-- reading syms from primary file (12 4) > --24112-- /usr/lib/system/liblaunch.dylib (0x1000fe000) > --24112-- reading syms from primary file (117 1) > --24112-- /usr/lib/system/libmacho.dylib (0x10010f000) > --24112-- reading syms from primary file (86 5) > --24112-- /usr/lib/system/libquarantine.dylib (0x100121000) > --24112-- reading syms from primary file (51 7) > --24112-- /usr/lib/system/libremovefile.dylib (0x100128000) > --24112-- reading syms from primary file (14 4) > --24112-- /usr/lib/system/libsystem_blocks.dylib (0x10012e000) > --24112-- reading syms from primary file (27 27) > --24112-- /usr/lib/system/libsystem_c.dylib (0x100134000) > --24112-- reading syms from primary file (1800 1823) > ==24112== Preferring higher priority redirection: > --24112-- old: 0x10015bcbd (memcpy$VARIANT$sse42) R-> (2018.0) 0x1000111d0 memcpy$VARIANT$sse42 > --24112-- new: 0x10015bcbd (memcpy$VARIANT$sse42) R-> (2018.1) 0x100010ed0 memmove$VARIANT$sse42 > ==24112== Preferring higher priority redirection: > --24112-- old: 0x10015bebd (memcpy$VARIANT$sse3x) R-> (2018.0) 0x100011410 memcpy$VARIANT$sse3x > --24112-- new: 0x10015bebd (memcpy$VARIANT$sse3x) R-> (2018.1) 0x100010d10 memmove$VARIANT$sse3x > --24112-- /usr/lib/system/libsystem_dnssd.dylib (0x100250000) > --24112-- reading syms from primary file (64 33) > --24112-- /usr/lib/system/libsystem_info.dylib (0x10025e000) > --24112-- reading syms from primary file (689 510) > --24112-- /usr/lib/system/libsystem_kernel.dylib (0x1002af000) > --24112-- reading syms from primary file (767 72) > --24112-- /usr/lib/system/libsystem_network.dylib (0x1002df000) > --24112-- reading syms from primary file (23 100) > --24112-- /usr/lib/system/libsystem_notify.dylib (0x1002eb000) > --24112-- reading syms from primary file (115 55) > --24112-- /usr/lib/system/libsystem_sandbox.dylib (0x1002fb000) > --24112-- reading syms from primary file (41 7) > --24112-- /usr/lib/system/libunc.dylib (0x100301000) > --24112-- reading syms from primary file (30 4) > --24112-- /usr/lib/system/libunwind.dylib (0x100307000) > --24112-- reading syms from primary file (102 67) > --24112-- /usr/lib/system/libxpc.dylib (0x100314000) > --24112-- reading syms from primary file (523 218) > --24112-- REDIR: 0x1001d55e0 (memset) redirected to 0x100010c90 (memset) > --24112-- REDIR: 0x1001d64e0 (strlen) redirected to 0x100010b00 (strlen) > --24112-- REDIR: 0x1001d5178 (malloc) redirected to 0x100010607 (malloc) > --24112-- REDIR: 0x1001d512d (calloc) redirected to 0x1000102ff (calloc) > --24112-- REDIR: 0x10015bcbd (memcpy$VARIANT$sse42) redirected to 0x100010ed0 (memmove$VARIANT$sse42) > --24112-- REDIR: 0x1001d5820 (strcmp) redirected to 0x100010c50 (strcmp) > --24112-- REDIR: 0x1001d63f0 (strncmp) redirected to 0x100010bf0 (strncmp) > sizeof(mvk_lruc_kv_t)=152, 0x1003462e0, 0x100346378, 0x100346410, 0x1003460d0 > at index 0 > end at index 0 > at index 1 |
|
From: John R. <jr...@bi...> - 2012-07-09 05:59:45
|
> I think there's a difference in the environment. > > For me: > About This Mac > More Info... > System Report... > Software > Developer: > Version:3.2 (10M2518) > Xcode:3.2.6 (1761) > SDKs: > Mac OS X: > 10.5:(9L31a) > 10.6:(10M2518) > > and for you memcpy is redirected differently: >> --24112-- REDIR: 0x10015bcbd (memcpy$VARIANT$sse42) redirected to 0x100010ed0 (memmove$VARIANT$sse42) > In any case... pardon my questions... I'm still a newbie in this. > But can you please explain the discussions between you and Pat? Pat points out that sometimes valgrind does not understand various "idioms" (arcane sequences of instructions) used by the compiler and/or runtime system. In particular, there is a long and convoluted history of "word at a time" idioms (instead of simple "character at a time" code) for computing strlen() and strchr(). Such idioms are part of the reason for valgrind's REDIR mechanism. When it works (and usually it does), then the REDIR mechanism makes it possible for memcheck to ignore the hard-to-understand code, and skip the instruction-by-instruction interpretation, of many str*() and mem*() routines. When REDIR does not work (and the bugs have been numerous) then confusion reigns. SSE4.1 introduced some new instructions which enable strlen/strchr to process 16 bytes at a time. Tracking these is even harder than the word-at-a-time idioms. Because one of the instructions always fetches 16 bytes, then naively it looks like that instruction often fetches beyond the end of an allocated block. But correct usage eventually ignores the bytes that are "over fetched", so there is no actual problem at the level of the whole idiom. Understanding the idiom involves understanding a 16-byte fetch. See below for why this can be tricky. The optimizations for memcpy do not involve SSE4.1, and the occurrence of over-fetching by memcpy is very much less than for strlen/strchr. I suspect that memcpy can over-fetch by at most 3 bytes (using a 4-byte fetch), so memcpy should not draw any complaints involving 8-byte fetches. In general --partial-loads-ok=yes silences memcheck even though memcheck would otherwise complain about overfetching. I suspect it's not the real problem in your case. > In particular... > - Why would memcpy need to read 8 bytes more? Although your memcheck run claims that memcpy is reading 8 bytes more, it isn't clear whether memcpy actually is reading 8 bytes more. The complaint could be part of a manifestation of a bug (or a couple of bugs) in memcheck. We've already pointed out that your run has no REDIR of memcpy when it should, so that is a problem of some kind. The REDIR machinery has failed. Part of the failure could be due to the actual symbolic name(s) that the compiler and runtime system have used for "memcpy". We need to see the actual code that is being executed. In particular, you haven't shown the code which surrounds 0x7FFFFFE00BAC from your initial report: ==57528== Invalid read of size 8 ==57528== at 0x7FFFFFE00BAC: ??? ==57528== by 0x100000E4D: __inline_memcpy_chk (in ./check) Please produce a disassembly of __inline_memcpy_chk that surrounds address 0x100000E4D. Please produce a disassembly of the routine that __inline_memcpy_chk calls (probably surrounding 0x7FFFFFE00BAC). Valgrind has a history that is more than ten years long. In the beginning, the only memory transfers that valgrind tracked were 1,2,4, or 8 bytes wide. Then came SSE and SSE2, which have some 16-byte transfers. Until valgrind implemented the internal machinery to track 16-byte transfers, there was an interim use of two 8-byte transfers. Keeping track of all of the associations between two 8-byte transfers is harder than keeping track of all of the associations of one 16-byte transfer. This might be involved in the specific complaint that you get. > - And why would the warning depend on the destination? That is, why am I getting this only in the 2nd iteration and not any other time? The first iteration might be special because the dynamic linkage is resolved on the first reference from any particular CALL, but not on subsequent references from the same CALL. The third and following references might actually be producing the same error, but because the traceback is the same as the error produced by the second call, then memcheck might process the "same error" by just incrementing a count associated with the traceback, instead of printing the full message each time. Because my effort failed to reproduce the bad behavior that you see, then the response to a bug report may well be "upgrade your XCode/Developer". Unless there is a good reason for insisting on fixing the interaction of valgrind with old versions of Apple code, then "upgrade to current version" (especially when the upgrade costs no money) is reasonable. [By the way, please trim more aggressively when responding. Most of what you quoted wasn't relevant to your response.] -- |
|
From: Tanmay M. <tan...@ho...> - 2012-07-10 03:04:08
|
Thanks John!I will probably upgrade my XCode soon. > > Because my effort failed to reproduce the bad behavior that you see, > then the response to a bug report may well be "upgrade your XCode/Developer". > Unless there is a good reason for insisting on fixing the interaction of > valgrind with old versions of Apple code, then "upgrade to current version" > (especially when the upgrade costs no money) is reasonable. |
|
From: Patrick J. L. <lop...@gm...> - 2012-07-10 14:44:01
|
On Sat, Jul 7, 2012 at 9:58 PM, Tanmay Mohapatra <tan...@ho...> wrote: > Thanks John and Pat. > > So what I understand is I can ignore the warning in this case. > But, I still get the warning even with --partial-loads-ok=yes. Why so? Because this is an SSE-optimized routine. The evidence is here: > ==67529== Invalid read of size 8 > ==67529== at 0x7FFFFFE00BAC: ??? > ==67529== by 0x100000E4D: __inline_memcpy_chk (in ./check) > ==67529== by 0x100000DCF: main (in ./check) > ==67529== Address 0x1002772a8 is 0 bytes after a block of size 456 alloc'd > ==67529== at 0x100010679: malloc (vg_replace_malloc.c:266) > ==67529== by 0x100000CC5: main (in ./check) > ==67529== "Address ... is 0 bytes after a block of size 456" means the 8-byte load was aligned (because 456 is a multiple of 8), but the _entire_ aligned word was invalid (because the first invalid byte was at offset 0). Since the entire aligned load (not just part of it) was invalid, "--partial-loads-ok=yes" did not have any effect. This is either a bug in the emitted code (very unlikely), or it means the compiler is generating SSE instructions for memcpy. This is pretty normal for a modern optimizing compiler on x86_64. As John mentions in another message, Valgrind implements 16-byte SSE loads and stores as a pair of 8-byte loads and stores, making it difficult to implement --partial-loads-ok for SSE instructions. This is simply a bug in Valgrind which I reported some time ago (https://bugs.kde.org/show_bug.cgi?id=294285). You could do me and yourself a favor by voting for it. :-) (I actually intend to fix it myself eventually, if I can ever find the time...) - Pat |
|
From: Tanmay M. <tan...@ho...> - 2012-07-12 05:14:51
|
voted! > > This is simply a bug in Valgrind which I reported some time ago > (https://bugs.kde.org/show_bug.cgi?id=294285). You could do me and > yourself a favor by voting for it. :-) > > (I actually intend to fix it myself eventually, if I can ever find the time...) > > - Pat |
|
From: Patrick J. L. <pa...@pa...> - 2012-07-10 15:08:32
|
On Sun, Jul 8, 2012 at 11:00 PM, John Reiser <jr...@bi...> wrote: > In particular, there is a long and convoluted > history of "word at a time" idioms (instead of simple "character at a time" > code) for computing strlen() and strchr(). Also memcpy(), memcmp(), etc. To elaborate a bit: An aligned load cannot partially fault, because every byte in an aligned word is on the same 4K page. So the "idiom" for these string and memory operations is to perform aligned loads as long as the first byte is within the allocated block, then throw away any "extra" bytes read past the end. > Such idioms are part of the > reason for valgrind's REDIR mechanism. When it works (and usually it does), > then the REDIR mechanism makes it possible for memcheck to ignore the > hard-to-understand code, and skip the instruction-by-instruction interpretation, > of many str*() and mem*() routines. When REDIR does not work (and the > bugs have been numerous) then confusion reigns. When compilers inline these functions, the REDIR mechanism cannot work. With the Intel C compiler that I use, every call to strlen() (among others) is open-coded using this idiom. This makes Valgrind unusable for me as it would require literally thousands of ever-changing suppression rules. GCC's optimizer will catch up eventually. This problem will only get worse, which is why I keep pushing to see it fixed properly. > SSE4.1 introduced some new instructions which enable strlen/strchr to > process 16 bytes at a time. Tracking these is even harder than the > word-at-a-time idioms. Actually it is the same thing once we implement 16-byte helper functions. But it is certainly broken today. > The optimizations for memcpy do not involve SSE4.1, and the occurrence > of over-fetching by memcpy is very much less than for strlen/strchr. I believe this is incorrect. The error message Tanmay reported is symptomatic of the second half of a 16-byte SSE load reading past the end of an allocated block. > I suspect that memcpy can over-fetch by at most 3 bytes (using a 4-byte > fetch), so memcpy should not draw any complaints involving 8-byte fetches. I believe this is also incorrect. On a 64-bit system, optimized memcpy() will either use 16-byte SSE instructions or 8-byte ordinary loads/stores. So the maximum possible over-fetch is either 7 bytes or 15 bytes. > In general --partial-loads-ok=yes silences memcheck even though memcheck > would otherwise complain about overfetching. This was true in Valgrind 3.7.0, but not entirely correct today. Silencing memcheck completely was actually a bug that I fixed (https://bugs.kde.org/show_bug.cgi?id=294523). In the fixed implementation, the extra bytes from beyond the end of the allocated block are simply marked "invalid". This silences the warning only when it should be silenced; i.e. when the extra bytes are not used for anything. In other words, the current implementation for --partial-loads-ok=yes models exactly the semantics of real machines: It is OK to do an aligned load past the end of the block, provided you do not use the invalid data for anything. If we just fix it for 16-byte loads -- and fix the validity bit propagation for, e.g., pmovmskb -- then all of these problems will go away. (Aside: In another response, you said --partial-loads-ok=yes is "slow". If you meant "slow compared to REDIR", then of course I agree. If you meant "slow compared to --partial-loads-ok=no", I would be curious to know your reasoning. The partial_loads_ok flag has no effect on the fast path; it is not even examined except in the case that would otherwise emit an error. Surely marking a few bytes invalid is actually _faster_ than emitting an error?) > I suspect it's not the real problem in your case. And I suspect it is. Of course, I could be wrong. There is only one way to find out... > We need to see the actual code > that is being executed. In particular, you haven't shown the code which > surrounds 0x7FFFFFE00BAC from your initial report: > ==57528== Invalid read of size 8 > ==57528== at 0x7FFFFFE00BAC: ??? > ==57528== by 0x100000E4D: __inline_memcpy_chk (in ./check) > Please produce a disassembly of __inline_memcpy_chk that surrounds > address 0x100000E4D. Please produce a disassembly of the routine > that __inline_memcpy_chk calls (probably surrounding 0x7FFFFFE00BAC). I agree completely. Seeing the disassembly is the only way to know what is really going on. - Pat |
|
From: Julian S. <js...@ac...> - 2012-07-10 15:54:05
|
> In other words, the current implementation for --partial-loads-ok=yes > models exactly the semantics of real machines: It is OK to do an > aligned load past the end of the block, provided you do not use the > invalid data for anything. If we just fix it for 16-byte loads -- and > fix the validity bit propagation for, e.g., pmovmskb -- then all of > these problems will go away. Alas, we will also have to do 32 byte loads. Maybe not right now, but as soon as Intel's ever-growing instruction set sprouts 256-bit PCMPxSTRx instructions, we will do. The pmovmskb definedness thing should be fixed sooner, and is relatively easy to do so -- add a new IR op that steers the data. It already causes problems not just in strlen code but also in SSE2 based graphics code, at least the stuff that is in Firefox. J |
|
From: Tanmay M. <tan...@ho...> - 2012-07-12 05:13:08
|
Thanks Pat.Here's the output of "otool -v -t check"Is this what is required?Do let me know if I need to disassemble in some other way. -----------------------------------------------------------check:(__TEXT,__text) sectionstart:0000000100000c78 pushq $0x000000000100000c7a movq %rsp,%rbp0000000100000c7d andq $0xf0,%rsp0000000100000c81 movq 0x08(%rbp),%rdi0000000100000c85 leaq 0x10(%rbp),%rsi0000000100000c89 movl %edi,%edx0000000100000c8b addl $0x01,%edx0000000100000c8e shll $0x03,%edx0000000100000c91 addq %rsi,%rdx0000000100000c94 movq %rdx,%rcx0000000100000c97 jmp 0x100000c9d0000000100000c99 addq $0x08,%rcx0000000100000c9d cmpq $0x00,(%rcx)0000000100000ca1 jne 0x100000c990000000100000ca3 addq $0x08,%rcx0000000100000ca7 callq 0x100000cb40000000100000cac movl %eax,%edi0000000100000cae callq 0x100000e5c0000000100000cb3 hlt_main:0000000100000cb4 pushq %rbp0000000100000cb5 movq %rsp,%rbp0000000100000cb8 subq $0x20,%rsp0000000100000cbc movl $0x000001c8,%edi0000000100000cc1 callq 0x100000e620000000100000cc6 movq %rax,0xf0(%rbp)0000000100000cca movl $0x000011d0,%edi0000000100000ccf callq 0x100000e620000000100000cd4 movq %rax,0xe8(%rbp)0000000100000cd8 movq 0xe8(%rbp),%rdx0000000100000cdc addq $0x00000130,%rdx0000000100000ce3 movq 0xe8(%rbp),%rcx0000000100000ce7 addq $0x00000098,%rcx0000000100000cee movq 0xf0(%rbp),%rax0000000100000cf2 movq 0xe8(%rbp),%rsi0000000100000cf6 movq %rax,%r90000000100000cf9 movq %rdx,%r80000000100000cfc movq %rsi,%rdx0000000100000cff movl $0x00000098,%esi0000000100000d04 leaq 0x00000165(%rip),%rdi0000000100000d0b movl $0x00000000,%eax0000000100000d10 callq 0x100000e680000000100000d15 movq $0xffffffff,%rax0000000100000d1c cmpq $0xff,%rax0000000100000d20 je 0x100000d3e0000000100000d22 movq $0xffffffff,%rcx0000000100000d29 movq 0xf0(%rbp),%rdi0000000100000d2d movl $0x00000098,%edx0000000100000d32 movl $0x00000000,%esi0000000100000d37 callq 0x100000e560000000100000d3c jmp 0x100000d510000000100000d3e movq 0xf0(%rbp),%rdi0000000100000d42 movl $0x00000098,%edx0000000100000d47 movl $0x00000000,%esi0000000100000d4c callq 0x100000df60000000100000d51 movl $0x00000000,0xfc(%rbp)0000000100000d58 movl $0x00000000,0xfc(%rbp)0000000100000d5f jmp 0x100000de70000000100000d64 movl 0xfc(%rbp),%esi0000000100000d67 leaq 0x0000012d(%rip),%rdi0000000100000d6e movl $0x00000000,%eax0000000100000d73 callq 0x100000e680000000100000d78 movq $0xffffffff,%rax0000000100000d7f cmpq $0xff,%rax0000000100000d83 je 0x100000daf0000000100000d85 movq $0xffffffff,%rcx0000000100000d8c movl 0xfc(%rbp),%eax0000000100000d8f cdqe0000000100000d91 imulq $0x00000098,%rax,%rax0000000100000d98 movq %rax,%rdi0000000100000d9b addq 0xe8(%rbp),%rdi0000000100000d9f movq 0xf0(%rbp),%rsi0000000100000da3 movl $0x000001c8,%edx0000000100000da8 callq 0x100000e500000000100000dad jmp 0x100000dd00000000100000daf movl 0xfc(%rbp),%eax0000000100000db2 cdqe0000000100000db4 imulq $0x00000098,%rax,%rax0000000100000dbb movq %rax,%rdi0000000100000dbe addq 0xe8(%rbp),%rdi0000000100000dc2 movq 0xf0(%rbp),%rsi0000000100000dc6 movl $0x000001c8,%edx0000000100000dcb callq 0x100000e220000000100000dd0 movl 0xfc(%rbp),%esi0000000100000dd3 leaq 0x000000ce(%rip),%rdi0000000100000dda movl $0x00000000,%eax0000000100000ddf callq 0x100000e680000000100000de4 incl 0xfc(%rbp)0000000100000de7 cmpl $0x1a,0xfc(%rbp)0000000100000deb jle 0x200000d640000000100000df1 movl 0xfc(%rbp),%eax0000000100000df4 leave0000000100000df5 ret___inline_memset_chk:0000000100000df6 pushq %rbp0000000100000df7 movq %rsp,%rbp0000000100000dfa subq $0x20,%rsp0000000100000dfe movq %rdi,0xf8(%rbp)0000000100000e02 movl %esi,0xf4(%rbp)0000000100000e05 movq %rdx,0xe8(%rbp)0000000100000e09 movq $0xffffffff,%rcx0000000100000e10 movq 0xe8(%rbp),%rdx0000000100000e14 movl 0xf4(%rbp),%esi0000000100000e17 movq 0xf8(%rbp),%rdi0000000100000e1b callq 0x100000e560000000100000e20 leave0000000100000e21 ret___inline_memcpy_chk:0000000100000e22 pushq %rbp0000000100000e23 movq %rsp,%rbp0000000100000e26 subq $0x20,%rsp0000000100000e2a movq %rdi,0xf8(%rbp)0000000100000e2e movq %rsi,0xf0(%rbp)0000000100000e32 movq %rdx,0xe8(%rbp)0000000100000e36 movq $0xffffffff,%rcx0000000100000e3d movq 0xe8(%rbp),%rdx0000000100000e41 movq 0xf0(%rbp),%rsi0000000100000e45 movq 0xf8(%rbp),%rdi0000000100000e49 callq 0x100000e500000000100000e4e leave0000000100000e4f ret------------------------------------------------------- > > I suspect it's not the real problem in your case. > > And I suspect it is. Of course, I could be wrong. There is only one > way to find out... > > > We need to see the actual code > > that is being executed. In particular, you haven't shown the code which > > surrounds 0x7FFFFFE00BAC from your initial report: > > ==57528== Invalid read of size 8 > > ==57528== at 0x7FFFFFE00BAC: ??? > > ==57528== by 0x100000E4D: __inline_memcpy_chk (in ./check) > > Please produce a disassembly of __inline_memcpy_chk that surrounds > > address 0x100000E4D. Please produce a disassembly of the routine > > that __inline_memcpy_chk calls (probably surrounding 0x7FFFFFE00BAC). > > I agree completely. Seeing the disassembly is the only way to know > what is really going on. > > - Pat |
|
From: Patrick J. L. <pa...@pa...> - 2012-07-12 22:01:23
|
On Wed, Jul 11, 2012 at 10:13 PM, Tanmay Mohapatra <tan...@ho...> wrote: > Thanks Pat. > Here's the output of "otool -v -t check" > Is this what is required? > Do let me know if I need to disassemble in some other way. Looks like the offending code is in a shared library (?) Maybe try "gdb ./check", and then "x/10i 0x7FFFFFE00BAC"? (Or "x/50i 0x7FFFFFE00B10" or something? Unfortunately we do not know the actual function being called, so it is hard to get the context. Disassembling at 0x7FFFFFE00BAC will show us the instruction but not the context; disassembling from somewhat earlier will show us the context but might be "out of sync" with the actual instruction stream) - Pat >> >> > We need to see the actual code >> > that is being executed. In particular, you haven't shown the code which >> > surrounds 0x7FFFFFE00BAC from your initial report: >> > ==57528== Invalid read of size 8 >> > ==57528== at 0x7FFFFFE00BAC: ??? >> > ==57528== by 0x100000E4D: __inline_memcpy_chk (in ./check) >> > Please produce a disassembly of __inline_memcpy_chk that surrounds >> > address 0x100000E4D. Please produce a disassembly of the routine >> > that __inline_memcpy_chk calls (probably surrounding 0x7FFFFFE00BAC). |