|
From: Josef W. <Jos...@gm...> - 2003-08-12 09:15:13
|
Hi, On Tuesday 12 August 2003 02:32, Steve G wrote: > After a few rounds of comments, I think you can accelerate > the maturity of Massif by distributing it with valgrind so > that people have easier access to it. You basically have to Nick, I would appreciate it if you could decide to make separate packages of your skins. This way, I would not be the only one, and issues regarding separate distribution (e.g. skin interface problems etc.) are more easily detected for sure. The installed header files work fine so far. Greetings, Josef |
|
From: Nicholas N. <nj...@ca...> - 2003-08-12 17:30:57
|
On Mon, 11 Aug 2003, Steve G wrote: > After a few rounds of comments, I think you can accelerate > the maturity of Massif by distributing it with valgrind so > that people have easier access to it. You basically have to > be a subscriber to this mail list just to know about these > extra skins unless Julian has linked your home page to the > valgrind web site. > > I understand the caution with rolling it out, but to get > thorough testing...people have to have an easy way to find > it. Absolutely, but we haven't reached that stage yet... it's only been released for 3 days :) I've had feedback saying that Massif isn't as novel/interesting as I claimed. Parasoft has InUse which I think comes with Insure++ which does what looks like similar profiling. There's another commercial program called GlowCode that might be similar. (It can be hard to tell with commercial products, the descriptions are often so vague.) Mpatrol has an mprof-like aspect that gives allocation stats; Mozilla has something called BloatBlame that looks similar. Does anyone else know of other such tools? Thanks. N |
|
From: Dan K. <da...@ke...> - 2003-08-12 17:57:58
|
Nicholas Nethercote wrote: > I've had feedback saying that Massif isn't as novel/interesting as I > claimed. Parasoft has InUse which I think comes with Insure++ which does > what looks like similar profiling. There's another commercial program > called GlowCode that might be similar. (It can be hard to tell with > commercial products, the descriptions are often so vague.) Mpatrol has an > mprof-like aspect that gives allocation stats; Mozilla has something > called BloatBlame that looks similar. Does anyone else know of other such > tools? It'd be great if you could link to all of 'em from your doc, and explain the differences. Also, your doc doesn't explain the benefit of doing this within valgrind as opposed to just doing it in a native environment; it should mention that. Thanks! - Dan -- Dan Kegel http://www.kegel.com http://counter.li.org/cgi-bin/runscript/display-person.cgi?user=78045 |
|
From: Nicholas N. <nj...@ca...> - 2003-09-29 15:00:07
|
Hi all, I've written another Valgrind skin. Valgrind's default skin, Memcheck, can find a lot of memory errors. In particular, it can find small overruns/underruns of heap blocks thanks to the redzones with which it pads heap blocks. But if you overrun by a larger amount, your access may unluckily fall into a different heap block, which Memcheck can't detect. Annelid is a skin designed to detect these cases. It tracks the bounds of every heap block, and pointers to heap blocks are associated with a particular block. If a pointer is used to access memory outside of its assigned block, a warning is given. Like Memcheck, it also warns if a freed heap block is accessed, but it can detect long-delayed accesses to freed blocks that Memcheck can't. Also, because Annelid tracks pointers, it can also find some type errors, such as if you add or multiply two heap pointers together. I hope to eventually take advantage of debug information to extend the bounds-checking to static arrays and structs, stack frames, and possibly stack-allocated arrays and structs. This would increase the range of errors that Annelid can detect that Memcheck cannot. Thanks to Jeremy Fitzhardinge for the original idea and advice. You can download a Valgrind bundle containing it from www.cl.cam.ac.uk/~njn25/valgrind.html. This is the first release, so apply the usual disclaimers about alpha software, etc. And there aren't any docs, although that shouldn't be much of a problem as it should be pretty straightforward to Valgrind users. N |
|
From: Steve G <lin...@ya...> - 2003-09-29 16:09:19
|
>I've written another Valgrind skin.
Great. Always glad to see innovation. :)
I seem to have found a problem. Since I'm familiar with xinetd, I
starting trying it on xinetd-2.3.12. I get the following message:
Invalid NEG: pointer arg
==7441== at 0x4024B9B3: memset (in /lib/libc-2.3.2.so)
==7441== by 0x805D722: Smorefds (siosup.c:877)
==7441== by 0x804F7FE: setup_file_descriptors (init.c:94)
==7441== by 0x804FBEA: init_daemon (init.c:317)
The code in question is:
old_size = __sio_n_descriptors * sizeof( mapd_s ) ;
new_size = n_fds * sizeof( mapd_s ) ;
new_size += old_size;
is_static = ( mmap_descriptors == NULL ) ;
p = sioexpand( (char *)mmap_descriptors, old_size, new_size,
is_static ) ;
if ( p == NULL )
return( SIO_ERR ) ;
memset(p+old_size, 0, new_size-old_size);
The values are:
new_size: 128
old_size:0
p: 0x40e1a018
This all looks normal.
I also see:
Invalid NEG: pointer arg
==7767== at 0x4024BF2F: memcpy (in /lib/libc-2.3.2.so)
==7767== by 0x40249FF3: __GI___strdup (in /lib/libc-2.3.2.so)
==7767== by 0x40264F9B: tzset_internal (in /lib/libc-2.3.2.so)
==7767== by 0x40265B96: __tz_convert (in /lib/libc-2.3.2.so)
Which is glibc function as is this:
Invalid NEG: pointer arg
==7767== at 0x4024BF2F: memcpy (in /lib/libc-2.3.2.so)
==7767== by 0x40236074: _IO_getdelim (in /lib/libc-2.3.2.so)
==7767== by 0x40234B24: __getline (in /lib/libc-2.3.2.so)
==7767== by 0x402BB6F8: nss_parse_file (in /lib/libc-2.3.2.so)
It always seems to be related to memcpy or memset. I am using Red
Hat 9.0, fully updated, on K6-2 processor and compiling with gcc
3.2.2 and the -O2 setting which is normal.
-Steve Grubb
__________________________________
Do you Yahoo!?
The New Yahoo! Shopping - with improved product search
http://shopping.yahoo.com
|
|
From: Philippe E. <ph...@wa...> - 2003-09-29 16:49:19
|
Nicholas Nethercote wrote:
> Hi all,
hi,
>
> I've written another Valgrind skin.
Look like great, a minor build problem, make clean
remove default.supp but it's not rebuilded
> Valgrind's default skin, Memcheck, can find a lot of memory errors. In
> particular, it can find small overruns/underruns of heap blocks thanks to
> the redzones with which it pads heap blocks. But if you overrun by a
> larger amount, your access may unluckily fall into a different heap block,
> which Memcheck can't detect.
>
> Annelid is a skin designed to detect these cases. It tracks the bounds of
> every heap block, and pointers to heap blocks are associated with a
> particular block. If a pointer is used to access memory outside of its
> assigned block, a warning is given. Like Memcheck, it also warns if a
> freed heap block is accessed, but it can detect long-delayed accesses to
> freed blocks that Memcheck can't. Also, because Annelid tracks pointers,
> it can also find some type errors, such as if you add or multiply two heap
> pointers together.
I tried it but I don't get any error (or I don't understand
how it works) for the following program:
#include <stdlib.h>
#include <stdio.h>
int main(void)
{
char * p1 = malloc(1000);
char * p2 = malloc(1000);
printf("%p %p %p\n", p1, p2, &p1[p2 - p1]);
p1[p2 - p1 + 1] = 0; /* expect error here */
free(p1);
free(p2);
return 0;
}
I tried to run it through various way
$ valgrind a.out
$ valgrind --skin=annelid a.out
$ valgrind --skin=annelid --trace-malloc=yes a.out
what I'm missing ?
I see also a lot of error:
Invalid NEG: pointer arg
==13600== at 0x402D6E27: memcpy (../sysdeps/generic/memcpy.c:43)
/* Copy just a few bytes to make DSTP aligned. */
len -= (-dstp) % OPSIZ;
I guess I need the right suppression file.
> I hope to eventually take advantage of debug information to extend the
> bounds-checking to static arrays and structs, stack frames, and possibly
> stack-allocated arrays and structs. This would increase the range of
> errors that Annelid can detect that Memcheck cannot.
It'll be great but what about adding these features directly
to MemCheck ?
regards,
Philippe Elie
|
|
From: Erik C. <er...@ar...> - 2003-09-29 18:38:02
|
On Mon, Sep 29, 2003 at 06:53:12PM +0000, Philippe Elie wrote:
> int main(void)
> {
> char * p1 = malloc(1000);
> char * p2 = malloc(1000);
> printf("%p %p %p\n", p1, p2, &p1[p2 - p1]);
>
> p1[p2 - p1 + 1] = 0; /* expect error here */
This isn't actually legal C code, and nor is the line above it.
It's section 6.3.6 in the ANSI standard, some other number in the
ISO standard. I'm looking at the Herbert Schildt book "The Annotated
C Standard".
You are only allowed to subtract pointers if they are pointers
into the same array object.
So it's a bit dodgy, though of course it is legal gcc, just not
legal C. And it could be legal some-other-language-that-you-compile-
and-feed-to-valgrind.
--
Erik Corry er...@ar...
A: Because it messes up the order in which people normally read text.
Q: Why is top-replying such a bad thing?
A: Top-replying.
Q: What is the most annoying thing in email?
|
|
From: Philippe E. <ph...@wa...> - 2003-09-29 19:11:29
|
Erik Corry wrote:
> On Mon, Sep 29, 2003 at 06:53:12PM +0000, Philippe Elie wrote:
>
>>int main(void)
>>{
>> char * p1 = malloc(1000);
>> char * p2 = malloc(1000);
>> printf("%p %p %p\n", p1, p2, &p1[p2 - p1]);
>>
>> p1[p2 - p1 + 1] = 0; /* expect error here */
>
>
> This isn't actually legal C code, and nor is the line above it.
>
> It's section 6.3.6 in the ANSI standard, some other number in the
> ISO standard. I'm looking at the Herbert Schildt book "The Annotated
> C Standard".
>
> You are only allowed to subtract pointers if they are pointers
> into the same array object.
I know and ? If you write only legal C valgrind will never
see any error :) In C you can't overrun array nor use memory
after free etc... The same problem occur with:
char * p1 = malloc(1000);
char * p += 1024; /* a bad index calculation in the prog */
p{p - p1] = 0;
this one is not catched too by annelid but I think it
should and yes it's definitely not legal C but who care
for a tools which check programmer error ?
> So it's a bit dodgy, though of course it is legal gcc, just not
> legal C. And it could be legal some-other-language-that-you-compile-
> and-feed-to-valgrind.
that's a real problem, is there any existing languages which use
malloc and allow weirds things like overrunning a heap block in
such way you access another block ?
Phil,
regards
|
|
From: Nicholas N. <nj...@ca...> - 2003-09-29 21:27:17
|
On Mon, 29 Sep 2003, Erik Corry wrote:
> > p1[p2 - p1 + 1] = 0; /* expect error here */
>
> This isn't actually legal C code, and nor is the line above it.
> [snip]
> You are only allowed to subtract pointers if they are pointers
> into the same array object.
>
> So it's a bit dodgy, though of course it is legal gcc, just not
> legal C. And it could be legal some-other-language-that-you-compile-
> and-feed-to-valgrind.
Interesting... were they allowing for the possibility of a non-flat
address space, or something like that? If you cast the pointers to
integers, did the subtraction, then used the result as the offset, would
that be legal C?
Either way, in practice we have to deal with it, especially since glibc
does it in strcpy() (see below), and it pops up in lots of other places.
N
/* Copy SRC to DEST. */
char *
strcpy (dest, src)
char *dest;
const char *src;
{
reg_char c;
char *__unbounded s = (char *__unbounded) CHECK_BOUNDS_LOW (src);
const ptrdiff_t off = CHECK_BOUNDS_LOW (dest) - s - 1;
size_t n;
do
{
c = *s++;
s[off] = c;
}
while (c != '\0');
n = s - src;
(void) CHECK_BOUNDS_HIGH (src + n);
(void) CHECK_BOUNDS_HIGH (dest + n);
return dest;
}
|
|
From: Erik C. <er...@ar...> - 2003-09-29 22:20:33
|
On Mon, Sep 29, 2003 at 10:27:15PM +0100, Nicholas Nethercote wrote:
> On Mon, 29 Sep 2003, Erik Corry wrote:
>
> > > p1[p2 - p1 + 1] = 0; /* expect error here */
> >
> > This isn't actually legal C code, and nor is the line above it.
> > [snip]
> > You are only allowed to subtract pointers if they are pointers
> > into the same array object.
> >
> > So it's a bit dodgy, though of course it is legal gcc, just not
> > legal C. And it could be legal some-other-language-that-you-compile-
> > and-feed-to-valgrind.
>
> Interesting... were they allowing for the possibility of a non-flat
> address space, or something like that?
Yes, segmentation.
> If you cast the pointers to
> integers, did the subtraction, then used the result as the offset, would
> that be legal C?
As far as I can tell the pointer->integer cast is allowed but you
can't use an offset that goes over the end of the object by more
than 1, so the answer is no.
> Either way, in practice we have to deal with it, especially since glibc
> does it in strcpy() (see below), and it pops up in lots of other places.
A bit unlucky if strcpy is always unchecked just because it tries to
be too clever.
Here's an idea:
* Each integer/pointer is associated with a possibly empty set
of malloc-areas.
* Obviously each pointer is initially associated with exactly
one malloc-area.
* If you generate a new pointer/integer by math between two operands
with different malloc-area sets then the new integer gets the union
of the sets.
* Unary ops don't change the malloc area associated (neg handling)
* As an optimisation, if you subtract two pointers and they each have
only one malloc-area associated and it's the same malloc-area then
you get an integer with no malloc-areas associated.
So we get:
> /* Copy SRC to DEST. */
> char *
> strcpy (dest, src)
dest is pointing to malloc area A
src is pointing to malloc area B
> char *dest;
> const char *src;
> {
> reg_char c;
> char *__unbounded s = (char *__unbounded) CHECK_BOUNDS_LOW (src);
s gets malloc-area from src so it has malloc area B
> const ptrdiff_t off = CHECK_BOUNDS_LOW (dest) - s - 1;
off is an op between dest and s so it has malloc areas A and B. The
-1 doesn't change its malloc area.
> size_t n;
>
> do
> {
> c = *s++;
OK iff s doesn't go outside B
> s[off] = c;
s + off has unified malloc areas (A, B), so this is OK iff (s+off) stays
within the union of A and B.
So you still catch the case where you overrun the source or destination
buffers (due to missing null or too short buffer), but most trickery
is allowed. This would also allow the notorious xor-double-linked-list
trick.
As an alternative you can have 3 types:
* Integer offset, I
* Pointer associated with memory area X, P(X)
* Integer that tranlates between memory areas Y and Z, T(Y,Z)
I op I -> I // Normal offset math
P(X) - P(X) -> I // Normal allowed pointer math
-P(X) -> P(X) // Unary neg
P(X) + P(X) -> P(X) // Together with above allows 'decoupled' sub
P(Y) - P(Z) -> I(Y,Z) // Subtract gets magic translation integer
P(Y) op P(Z) -> I // Other ops just get an integer
P(X) op I(X,Z) -> P(Z) // Magically moves to other area
P(X) op I(Y,X) -> P(Y) // Magically moves to other area
This one is more restrictive (you can add more rules to make it less
so), but also covers the cases we've seen until now (except xor linked
lists).
--
Erik Corry er...@ar...
A: Because it messes up the order in which people normally read text.
Q: Why is top-replying such a bad thing?
A: Top-replying.
Q: What is the most annoying thing in email?
|
|
From: Peter \Firefly\ L. <fi...@di...> - 2003-09-30 09:04:07
|
On Tue, 30 Sep 2003, Erik Corry wrote: > > Interesting... were they allowing for the possibility of a non-flat > > address space, or something like that? > > Yes, segmentation. And separate address spaces (as on 8051) and capability based systems (sort of like a segment per object). -Peter |
|
From: Nicholas N. <nj...@ca...> - 2003-09-30 11:56:10
|
On Tue, 30 Sep 2003, Erik Corry wrote:
> As an alternative you can have 3 types:
>
> * Integer offset, I
> * Pointer associated with memory area X, P(X)
> * Integer that tranlates between memory areas Y and Z, T(Y,Z)
>
> I op I -> I // Normal offset math
> P(X) - P(X) -> I // Normal allowed pointer math
> -P(X) -> P(X) // Unary neg
> P(X) + P(X) -> P(X) // Together with above allows 'decoupled' sub
> P(Y) - P(Z) -> I(Y,Z) // Subtract gets magic translation integer
> P(Y) op P(Z) -> I // Other ops just get an integer
> P(X) op I(X,Z) -> P(Z) // Magically moves to other area
> P(X) op I(Y,X) -> P(Y) // Magically moves to other area
>
> This one is more restrictive (you can add more rules to make it less
> so), but also covers the cases we've seen until now (except xor linked
> lists).
This is the Right Way to do it.
I tried the simpler approach of forming cliques whenever two pointers were
subtracted, ie. on p1-p2, join the two pointers' segments such that a
pointer derived from p1 can access p2's block, and vice versa. Problem
was that the cliques grew enormous, sometimes 1000's of entries, and
slowed things down terribly. I guess it's because certain pointers were
involved in lots of subtractions.
As for your operations above, it's a bit more complicated than that.
Here's what types I think are necessary:
* known non-pointer: n
* known pointer-difference: nXY
* unknown: ?
* known pointer: pX
(apologies for the different notation).
Here's the rules for addition:
-----------------------
+ | n nCD ? pY
-----------------------
n | n nCD ? pY
nAB | nAB (1) ? (2)
? | ? ? ? ?
pX | pX (2) ? n!
-----------------------
(1) nMN + nNO --> nMO
nMN + nNM --> n
otherwise --> n
(2) nMN + pM --> pN
pM + nMN --> pN
otherwise --> n
'!' means "report type error"
As a refinement of this, any '?' result that is definitely not a pointer,
as judged by a range test (eg. x < 0x01000000 || x > 0xff000000), gets
converted to 'n'.
Subtraction:
-----------------------
- | n nCD ? pY
-----------------------
n | n n ? n+
nAB | nAB (1) ? n
? | ? ? ?* ?*
pX | pX (2) ?* pYX
-----------------------
(1) nMN - nMO --> nNO
nMN - nNM --> n
otherwise --> n
(2) pX - nYX --> pY
otherwise --> n
Again, there's a range test on '?' results, except for those marked with a
'*'. Why not those? Because we can't convert what might be a pointer
difference into a nonptr. If you take a known pointer 'p' and an unknown
pointer 'u' and do this:
diff = u - p
if you ever mark 'diff' as a nonptr due to a range check, then Annelid
would think this:
p[diff]
is accessing p's block, when it's really accessing u's block, and give a
false positive. (This exact case has happened to me before. Believe me,
the typing of all these operations is significantly trickier to get right
than you would first think; I have been surprised several times by such
things while implementing Annelid.)
Other operations (&, |, ^, etc) would treat the types 'n' and 'nAB' as
equivalent, but results would always be downgraded to 'n' (eg. nAB & n --> n).
(The exact rules for every different op vary, however -- you can't just
lump them all together, although this is what you might expect.)
I think this can work.
N
|
|
From: Jeremy F. <je...@go...> - 2003-10-01 17:40:04
|
On Mon, 2003-09-29 at 18:20, Erik Corry wrote: > * Each integer/pointer is associated with a possibly empty set > of malloc-areas. > * Obviously each pointer is initially associated with exactly > one malloc-area. > * If you generate a new pointer/integer by math between two operands > with different malloc-area sets then the new integer gets the union > of the sets. > * Unary ops don't change the malloc area associated (neg handling) > * As an optimisation, if you subtract two pointers and they each have > only one malloc-area associated and it's the same malloc-area then > you get an integer with no malloc-areas associated. Yep, this is what I suggested. The tricky thing is dealing with pointers to things which aren't malloced, like things in data/bss. If you have a statically-initialized pointer in data, then annelid can't tell it is a pointer. J |
|
From: Nicholas N. <nj...@ca...> - 2003-10-01 17:55:29
|
On Wed, 1 Oct 2003, Jeremy Fitzhardinge wrote: > > * If you generate a new pointer/integer by math between two operands > > with different malloc-area sets then the new integer gets the union > > of the sets. > > Yep, this is what I suggested. But the sets become huge (eg. 1000s of entries), which slows things down horrifically. Besides, when you do the pointer subtraction, that "connection" between them is really a property of the ptrdiff, not a property of the segments, so putting the two segments into the same set isn't a true reflection of what's happening. These two reasons are why I think having a dedicated ptrdiff type is a better solution. Although I haven't implemented it yet, so I'm yet to see how it works in practice. N |
|
From: Erik C. <er...@ar...> - 2003-10-01 19:17:18
|
On Wed, Oct 01, 2003 at 06:54:20PM +0100, Nicholas Nethercote wrote: > On Wed, 1 Oct 2003, Jeremy Fitzhardinge wrote: > > > > * If you generate a new pointer/integer by math between two operands > > > with different malloc-area sets then the new integer gets the union > > > of the sets. > > > > Yep, this is what I suggested. > > But the sets become huge (eg. 1000s of entries), which slows things down > horrifically. If a pointer is dereferenced then you can reduce that set to one single segment, namely the segment it points at when it is dereferenced. > Besides, when you do the pointer subtraction, that > "connection" between them is really a property of the ptrdiff, not a > property of the segments, so putting the two segments into the same set > isn't a true reflection of what's happening. > > These two reasons are why I think having a dedicated ptrdiff type is a > better solution. For the record, that is what I was suggesting in the "simple" suggestion snipped above. It's the new integer (ptrdiff) that is assigned the union of the sets, not all pointers into the sets. > Although I haven't implemented it yet, so I'm yet to see > how it works in practice. The idea you explained the other day is a more complex (accurate) version of the "just taking the union" idea above. The question is whether it gives too many false alarms. -- Erik Corry er...@ar... A: Because it messes up the order in which people normally read text. Q: Why is top-replying such a bad thing? A: Top-replying. Q: What is the most annoying thing in email? |
|
From: Jeremy F. <je...@go...> - 2003-10-01 17:40:04
|
On Mon, 2003-09-29 at 17:27, Nicholas Nethercote wrote: > Interesting... were they allowing for the possibility of a non-flat > address space, or something like that? If you cast the pointers to > integers, did the subtraction, then used the result as the offset, would > that be legal C? Not really. There are some (very strange) machines which do annelid-like things internally, so the C standard allows for them. But it doesn't matter what C does or doesn't allow, since Valgrind works on machine-code from any language. > Either way, in practice we have to deal with it, especially since glibc > does it in strcpy() (see below), and it pops up in lots of other places. Yep. J |
|
From: Nicholas N. <nj...@ca...> - 2003-09-29 17:14:59
|
On Mon, 29 Sep 2003, Philippe Elie wrote:
> Look like great, a minor build problem, make clean
> remove default.supp but it's not rebuilded
Hmm, that seems to be a problem with Valgrind in general. I'll
investigate.
> I tried it but I don't get any error (or I don't understand
> how it works) for the following program:
>
> #include <stdlib.h>
> #include <stdio.h>
>
> int main(void)
> {
> char * p1 = malloc(1000);
> char * p2 = malloc(1000);
> printf("%p %p %p\n", p1, p2, &p1[p2 - p1]);
>
> p1[p2 - p1 + 1] = 0; /* expect error here */
>
> free(p1);
> free(p2);
>
> return 0;
> }
Indeed, that's one case it cannot find. Say you have a pointer P to a
block B. If you add an integer to P, the resulting pointer P' is still
considered to point to B. That's the basic idea of how Annelid works.
But if the integer came from a pointer difference, when you add that
integer to P, the resulting pointer P' really now points not to a
different block.
The way I deal with that currently is this: when subtracting a heap
pointer from another heap pointer, I don't consider the result R an
"integer", in the sense that adding R to a pointer doesn't result in
another pointer worth checking. So the result of adding the difference to
a heap pointer is something that I don't check. If that makes sense.
A pointer-pointer subtraction like this effectively links the two blocks;
I experimented with an approach that handles this and would find your
problem above, but I reverted to my simpler system because it was faster.
But I could try again if you think it's important.
> I see also a lot of error:
>
> Invalid NEG: pointer arg
> ==13600== at 0x402D6E27: memcpy (../sysdeps/generic/memcpy.c:43)
>
> /* Copy just a few bytes to make DSTP aligned. */
> len -= (-dstp) % OPSIZ;
>
> I guess I need the right suppression file.
Hmm, see my other reply to Steve G. Thanks for giving me the exact line
it occurred on. I won't pretend to understand what that code is doing,
but I'll change things to let pointer negation take place without warning
since it seems to crop up reasonably often.
[Steve G, don't worry about digging through the asm now, thanks]
> > I hope to eventually take advantage of debug information to extend the
> > bounds-checking to static arrays and structs, stack frames, and possibly
> > stack-allocated arrays and structs. This would increase the range of
> > errors that Annelid can detect that Memcheck cannot.
>
> It'll be great but what about adding these features directly
> to MemCheck ?
Memcheck uses a fundamentally different technique, one that just doesn't
work for static and stack errors like this. As for combining Memcheck and
Annelid into one, well, that would run programs 60--100x slower and more
than triple memory footprint, which I don't think would be such a good
idea.
Thanks for the feedback.
N
|
|
From: Nicholas N. <nj...@ca...> - 2003-09-29 17:29:19
|
On Mon, 29 Sep 2003, Nicholas Nethercote wrote: > > I tried it but I don't get any error (or I don't understand > > how it works) for the following program: > > > > p1[p2 - p1 + 1] = 0; /* expect error here */ > > Indeed, that's one case it cannot find. Actually, let me clarify: that code is quite legitimate, you're clearly accessing within p2's bounds, and in a safe way. And this use of pointer differences comes up reasonably often, so Annelid shouldn't complain about it. It should complain, however, if you do: p1[p2 - p1 + 1001] = 0; Currently Annelid doesn't detect this, but the approach I mentioned which handles pointer differencing would detect such an overrun. (Basically, the approach involves tying the two "segments" together, such that p1+i can be used to access p2's block, and vice versa.) N |
|
From: Philippe E. <ph...@wa...> - 2003-09-29 18:17:51
|
Nicholas Nethercote wrote:
> On Mon, 29 Sep 2003, Philippe Elie wrote:
>> char * p1 = malloc(1000);
>> char * p2 = malloc(1000);
>> p1[p2 - p1 + 1] = 0; /* expect error here */
> Indeed, that's one case it cannot find. Say you have a pointer P to a
> block B. If you add an integer to P, the resulting pointer P' is still
> considered to point to B. That's the basic idea of how Annelid works.
>
> But if the integer came from a pointer difference, when you add that
> integer to P, the resulting pointer P' really now points not to a
> different block.
>
> The way I deal with that currently is this: when subtracting a heap
> pointer from another heap pointer, I don't consider the result R an
> "integer", in the sense that adding R to a pointer doesn't result in
> another pointer worth checking. So the result of adding the difference to
> a heap pointer is something that I don't check. If that makes sense.
It make sense but the distinction seems a bit artificial,
people using annelid will be suprised than
void do_it(char* p1, int diff)
{
p1[diff] = 0;
}
do_it(p1, p2 - p1);
do_it(1024);
generate only one error but gdb will show the parameter
passed to do_it() get exactly the same value.
> A pointer-pointer subtraction like this effectively links the two blocks;
> I experimented with an approach that handles this and would find your
> problem above, but I reverted to my simpler system because it was faster.
> But I could try again if you think it's important.
It can catch typo/thinko when doing pointer substraction. I
think it's usefull especially for C programmers who (ab)use
a lot of pointer arithmetic.
On a different subject I used a few years ago a compiler
doing bound checking by instrumenting code, one feature
was the check of all pointer arithmetic to catch error
earlier than at memory read/write.
i.e.
char * p = malloc(1000);
p += 1024;
the addition is not valid in C and was catched. Annelid seems
able to do some pointer arithmetic check, have you plan to
catch this sort of things ?
If it's mainly a problem a execution speed what about adding
a --strict-mode=yes which do more validation on the executed
code ?
regards,
Phil
|
|
From: Nicholas N. <nj...@ca...> - 2003-09-29 21:21:56
|
On Mon, 29 Sep 2003, Philippe Elie wrote:
> On a different subject I used a few years ago a compiler
> doing bound checking by instrumenting code, one feature
> was the check of all pointer arithmetic to catch error
> earlier than at memory read/write.
>
> i.e.
>
> char * p = malloc(1000);
> p += 1024;
>
> the addition is not valid in C and was catched. Annelid seems
> able to do some pointer arithmetic check, have you plan to
> catch this sort of things ?
Hmm, I hadn't thought of doing that. It's not very hard to fit in to
Annelid, and I just tried it.
It doesn't work, unfortunately, because going one past the end of an array
is quite common, eg:
#include <stdlib.h>
int main ( void )
{
char* a = malloc(10);
char* p;
for (p = a; p < a+10; p++)
printf("%c\n", *p);
return 0;
}
The last time around the loop, p is one past the end of a's block. This
example is contrived, but in practice it seems to be very common -- when I
added the check to Annelid, it caused 44 incorrectly diagnosed errors when
checking 'date'. So it doesn't look feasible.
I think it's a bit like Memcheck allowing you to copy around uninitialised
memory, as long as you don't actually use it in a way that affects the
program's outcome -- you have to allow it, because it actually happens a
lot.
But thanks for the suggestion, I hadn't thought of this and it's useful to
know that it doesn't work.
N
|
|
From: Erik C. <er...@ar...> - 2003-09-29 22:23:05
|
On Mon, Sep 29, 2003 at 10:21:54PM +0100, Nicholas Nethercote wrote: > It doesn't work, unfortunately, because going one past the end of an array > is quite common, eg: For what it's worth, going one past the end is a special case in the C standard and is allowed. All other values aren't allowed. But my guess is that it isn't worth much, because you will undoubtedly discover other cases if you allow this one. -- Erik Corry er...@ar... A: Because it messes up the order in which people normally read text. Q: Why is top-replying such a bad thing? A: Top-replying. Q: What is the most annoying thing in email? |
|
From: Steve G <lin...@ya...> - 2003-09-30 00:33:57
|
>> It doesn't work, unfortunately, because going one past the end >> of an array is quite common, eg: > >For what it's worth, going one past the end is a special case >in the C standard and is allowed What if the loop increments by 2? The pointer can point 100 past the end legally. It just can't read or write to memory. :) Looking at the code in Nicholas's example, the printf part shouldn't be called...so nothing accessed memory. I guess the trick must be that the pointers can point to anything...when a read or write occurs, then you should start the error checking instead of when doing the math. For example, the negative check would be very useful for me. Some functions return -1 for error...even though they were declared returning a pointer. In that case, the negative check would help if its ever dereferenced. Just checking the pointer's value to see if it was the -1 error code shouldn't generate an error. -Steve Grubb __________________________________ Do you Yahoo!? The New Yahoo! Shopping - with improved product search http://shopping.yahoo.com |
|
From: Olly B. <ol...@su...> - 2003-09-30 08:15:29
|
On Mon, Sep 29, 2003 at 05:33:50PM -0700, Steve G wrote:
> >For what it's worth, going one past the end is a special case
> >in the C standard and is allowed
>
> What if the loop increments by 2?
Then if the pointer ends up more than one past the end of the block, the
program will invoke undefined behaviour according to the C standard.
If this worries you, allocate the block with the size rounded up to an
even number...
Note that undefined behaviour can be anything, including what you
expect (i.e. that it works as if the array were larger). That it works
on a particular compiler/platform doesn't mean it's actually
well-defined ISO C.
> The pointer can point 100 past the end legally. It just can't
> read or write to memory. :)
If it points 100 past the end, all bets are off. On a typical modern
machine with a flat memory space, it'll probably work as you hope.
But it's not maximally portable - for example, on a DOS machine in (say)
small memory model, it's likely to fail if you try it with a block of
memory ending less than 100 bytes from the end of the segment.
I can't easily check but I suspect the pointer would wrap and compare
less than the end value, so the loop won't terminate correctly, but
instead continue over into the start of the segment.
Cheers,
Olly
|
|
From: Erik C. <er...@ar...> - 2003-09-30 08:43:05
|
On Mon, Sep 29, 2003 at 05:33:50PM -0700, Steve G wrote: > >> It doesn't work, unfortunately, because going one past the end > > >> of an array is quite common, eg: > > > >For what it's worth, going one past the end is a special case > >in the C standard and is allowed > > What if the loop increments by 2? Then you are invoking undefined behaviour unless you put a conditional break at the end of the loop to avoid the pointer going out of range. Section 6.3.6 in the ANSI standard. > The pointer can point 100 past the end legally. It's not legal in C. In machine code, of course, only the ABI can rule something illegal and of course the SysV x86 ABI that Linux conforms to doesn't rule stray pointers illegal. If someone wanted to make a skin that checked a program for Boehm-GC-compatibility then it might be an error though. Interesting project. Alternatively you might be able to make a skin that gave more accurate info to the Boehm GC by giving the GC a magic way to find out whether something was an integer or a pointer. > It just can't read or write to memory. :) The one-past-the-end rule applies to creating the pointer, not to using it. You aren't alllowed to use it of course. > Looking at the code in Nicholas's example, the printf part > shouldn't be called...so nothing accessed memory. I guess the > trick must be that the pointers can point to anything...when a > read or write occurs, then you should start the error checking > instead of when doing the math. Yes, I think that's the only sensible way for Annelid to work, -- Erik Corry er...@ar... A: Because it messes up the order in which people normally read text. Q: Why is top-replying such a bad thing? A: Top-replying. Q: What is the most annoying thing in email? |
|
From: Jeremy F. <je...@go...> - 2003-10-01 17:40:03
|
On Mon, 2003-09-29 at 18:14, Nicholas Nethercote wrote: > > /* Copy just a few bytes to make DSTP aligned. */ > > len -= (-dstp) % OPSIZ; > > > > I guess I need the right suppression file. > > Hmm, see my other reply to Steve G. Thanks for giving me the exact line > it occurred on. I won't pretend to understand what that code is doing, It actually does make sense. This code wants to move the pointer to an aligned address, so it does the first part of the operation on small units until the pointer is aligned, then does the rest with word-sized operations. In other words, just make negation drop the pointerness and carry on. J |