|
From: Brendan D. <bre...@pe...> - 2006-02-20 09:30:35
|
First off, my heartfelt thanks to Julian and all the folks who have put time towards working on Valgrind --- this tool has saved my day more times than I can count (aah, the perils of working with other people's code when you don't have access to the source...). I have two questions --- hopefully folks can point me in the correct direction to find some answers. First, Valgrind (I'm running 3.1.0 on a dual dual core opteron box in pure 32bit mode) really doesn't seem to like static (scalar) variables. When I run code which uses static variables, memcheck complains about invalid reads of size 4 (not surprising since the variables in question were uint32_t's). The error messages went away when I got rid of the static variables, which is consistent with what I've seen elsewhere. Is this related to issues with static arrays? I don't know the C++ ABI well enough, but I'd expect that static scalars are allocated in the data section (together with global static arrays), as opposed to the stack which is used for local variables, so I'm pretty certain that these issues aren't related. Second question is a bit more troublesome: I get different behavior when executing a program under Valgrind (specifically, memcheck) than I do when just executing from the command line. In this case, I get a SIGSEGV which refers to a null pointer being dereferenced. That's fine --- I get why the null pointer deref should cause a segfault, the problem is that this doesn't happen when not running under Valgrind. The program continues executing normally, which is not the behavior one would expect when deref'ing a null pointer. Any thoughts? Thanks in advance for your help, Brendan. |
|
From: Nicholas N. <nj...@cs...> - 2006-02-20 21:02:25
|
On Mon, 20 Feb 2006, Brendan Drew wrote: > First, Valgrind (I'm running 3.1.0 on a dual dual core opteron box in pure > 32bit mode) really doesn't seem to like static (scalar) variables. When I run > code which uses static variables, memcheck complains about invalid reads of > size 4 (not surprising since the variables in question were uint32_t's). The > error messages went away when I got rid of the static variables, which is > consistent with what I've seen elsewhere. Is this related to issues with > static arrays? I don't know the C++ ABI well enough, but I'd expect that > static scalars are allocated in the data section (together with global static > arrays), as opposed to the stack which is used for local variables, so I'm > pretty certain that these issues aren't related. You need to give us more specific information. Do you have a small test case that exhibits the behaviour? What does Valgrind's output look like? > Second question is a bit more troublesome: I get different behavior when > executing a program under Valgrind (specifically, memcheck) than I do when > just executing from the command line. In this case, I get a SIGSEGV which > refers to a null pointer being dereferenced. That's fine --- I get why the > null pointer deref should cause a segfault, the problem is that this doesn't > happen when not running under Valgrind. The program continues executing > normally, which is not the behavior one would expect when deref'ing a null > pointer. Any thoughts? Sometimes bugs can crash a program when running under Valgrind that don't crash it normally, and vice versa. They're still bugs that should be fixed, though. Nick |
|
From: Brendan D. <bre...@pe...> - 2006-02-28 23:46:09
|
Nicholas Nethercote wrote: > On Mon, 20 Feb 2006, Brendan Drew wrote: > >> First, Valgrind (I'm running 3.1.0 on a dual dual core opteron box in >> pure 32bit mode) really doesn't seem to like static (scalar) >> variables. When I run code which uses static variables, memcheck >> complains about invalid reads of size 4 (not surprising since the >> variables in question were uint32_t's). The error messages went away >> when I got rid of the static variables, which is consistent with what >> I've seen elsewhere. Is this related to issues with static arrays? I >> don't know the C++ ABI well enough, but I'd expect that static >> scalars are allocated in the data section (together with global >> static arrays), as opposed to the stack which is used for local >> variables, so I'm pretty certain that these issues aren't related. > > You need to give us more specific information. Do you have a small > test case that exhibits the behaviour? What does Valgrind's output > look like? > I've looked at this some more and I can't seem to contrive a simple example. Here's what I've observed: in some cases, when using a static variable, valgrind complains about reads / writes to those variables upon entering their scope. To me this implies that something is going on and it is related to static variables. Given that I've got a program where Valgrind does what one would expect with the static variables (i.e. ignore them completely), I've run out of good ideas. Is it plausible that memory involved in the mechanism which controls reads / writes to static variables (maybe via some magical register like $sp or $fp in MIPS hardware) may have been previously trashed, thereby confusing Valgrind? Alternately, could it be that the portion of Valgrind which deals with static variables was trashed? >> Second question is a bit more troublesome: I get different behavior >> when executing a program under Valgrind (specifically, memcheck) than >> I do when just executing from the command line. In this case, I get a >> SIGSEGV which refers to a null pointer being dereferenced. That's >> fine --- I get why the null pointer deref should cause a segfault, >> the problem is that this doesn't happen when not running under >> Valgrind. The program continues executing normally, which is not the >> behavior one would expect when deref'ing a null pointer. Any thoughts? > > Sometimes bugs can crash a program when running under Valgrind that > don't crash it normally, and vice versa. They're still bugs that > should be fixed, though. > > Nick Trust me, I want these bugs fixed -- the problem is that I neither wrote nor have access to the relevant code. What confuses me is why the code execution would be any different --- as I've understood (and please, educate me if I'm wrong), Valgrind uses a virtualization approach rather than an instrumentation approach. Which means (in theory) I should see basically identical behavior out of programs which are supposed to display deterministic behavior. Are there subtle issues I'm not aware of which would make this not the case? Thanks, Brendan. -- Brendan Drew Research Scientist PercepTek, Inc. 12395 North Mead Way Littleton, CO 80125 Tel: 720-344-1037 x 126 Fax: 720-344-2360 |
|
From: Joe M. <joe...@me...> - 2006-03-01 16:08:50
|
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Brendan Drew wrote: > I've looked at this some more and I can't seem to contrive a simple > example. Here's what I've observed: in some cases, when using a static > variable, valgrind complains about reads / writes to those variables > upon entering their scope. To me this implies that something is going on > and it is related to static variables. Given that I've got a program Is your code threaded? It's possible a second thread is entering the function while the first thread is still initializing the statics - in that circumstance I could maybe see how valgrind would complain that the second thread is doing uninitialized memory reads. (In fact, I'm not at all sure what happens in that case - would the second thread reinitialize all the statics or try to use them partially initialized? Depends on the compiler implementation, I suppose.) > Trust me, I want these bugs fixed -- the problem is that I neither wrote > nor have access to the relevant code. What confuses me is why the code > execution would be any different --- as I've understood (and please, > educate me if I'm wrong), Valgrind uses a virtualization approach rather > than an instrumentation approach. Which means (in theory) I should see > basically identical behavior out of programs which are supposed to > display deterministic behavior. Are there subtle issues I'm not aware of > which would make this not the case? Valgrind adds red zones around each allocated block of memory to catch out-of-bounds errors, so an uninitialized pointer will point to entirely different memory under valgrind. In your case, the pointer was probably going somewhere harmless (as far as crashes go) in the regular setup, but pointing into a red zone while running under Valgrind. Also, this probably isn't happening here, but it's valgrind.h includes macros which are no-ops when running normally, but trigger behaviour in valgrind when running under it. (You can use this to make valgrind work with custom memory allocators, for example.) One of the macros returns whether or not the program was running under valgrind, so in theory the program could be running an entirely different code path depending on the return value. If your code was compiled in debug mode and the original authors used valgrind, it may use these macros. Joe -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2.1 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFEBcb+q5NnQEnPdLcRAk3rAJ9OabynZOwNA/k1PemurfW2dffeYACbB8CX wxIgs3pEN9Cg2lVyaqjMM60= =AEkj -----END PGP SIGNATURE----- |
|
From: Brendan D. <bre...@pe...> - 2006-03-01 21:08:10
|
Joe Mason wrote: >-----BEGIN PGP SIGNED MESSAGE----- >Hash: SHA1 > >Brendan Drew wrote: > > > >>I've looked at this some more and I can't seem to contrive a simple >>example. Here's what I've observed: in some cases, when using a static >>variable, valgrind complains about reads / writes to those variables >>upon entering their scope. To me this implies that something is going on >>and it is related to static variables. Given that I've got a program >> >> > >Is your code threaded? It's possible a second thread is entering the >function while the first thread is still initializing the statics - in >that circumstance I could maybe see how valgrind would complain that the >second thread is doing uninitialized memory reads. (In fact, I'm not at >all sure what happens in that case - would the second thread >reinitialize all the statics or try to use them partially initialized? >Depends on the compiler implementation, I suppose.) > > > Some portions are, yes. Although I'll admit right now that I'm surprised if this were the case. The variables are initialized as they are declared. I believe (and please, correct me if I'm wrong) that static variables which are initialized in place (e.g. static int foo = 0; ) result in the compiler emitting code which has the static variable allocated and initialized in the data section (i.e. initialized as the program is read into memory). >>Trust me, I want these bugs fixed -- the problem is that I neither wrote >>nor have access to the relevant code. What confuses me is why the code >>execution would be any different --- as I've understood (and please, >>educate me if I'm wrong), Valgrind uses a virtualization approach rather >>than an instrumentation approach. Which means (in theory) I should see >>basically identical behavior out of programs which are supposed to >>display deterministic behavior. Are there subtle issues I'm not aware of >>which would make this not the case? >> >> > >Valgrind adds red zones around each allocated block of memory to catch >out-of-bounds errors, so an uninitialized pointer will point to entirely >different memory under valgrind. In your case, the pointer was probably >going somewhere harmless (as far as crashes go) in the regular setup, >but pointing into a red zone while running under Valgrind. > > Okay -- this makes a bit more sense now. Having looked at their code, I've "isolated" (if such a word is appropriate to describe the diagnosis of systematic failures which look more like terminal cancer than independent faults) several potential issues (e.g: their image representations don't all agree on whether they use 1 based or 0 based indexing. I'm willing to wager that they've mixed and matched throughout, which would explain a great deal about why the heap and stack end up trashed beyond recognition). >Also, this probably isn't happening here, but it's valgrind.h includes >macros which are no-ops when running normally, but trigger behaviour in >valgrind when running under it. (You can use this to make valgrind work >with custom memory allocators, for example.) One of the macros returns >whether or not the program was running under valgrind, so in theory the >program could be running an entirely different code path depending on >the return value. If your code was compiled in debug mode and the >original authors used valgrind, it may use these macros. > > > I very much doubt that these folks have valgrind installed on their machines. I'll find out tomorrow when they get here and I help them install all the bits and pieces necessary to get their test harness up and running. Thanks for the heads up on this though. >Joe > >-----BEGIN PGP SIGNATURE----- >Version: GnuPG v1.4.2.1 (MingW32) >Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org > >iD8DBQFEBcb+q5NnQEnPdLcRAk3rAJ9OabynZOwNA/k1PemurfW2dffeYACbB8CX >wxIgs3pEN9Cg2lVyaqjMM60= >=AEkj >-----END PGP SIGNATURE----- > > Thanks for your help! -- Brendan Drew Research Scientist PercepTek, Inc. Tel: 720-344-1037 x 126 Fax: 720-344-2360 12395 North Mead Way Littleton, CO 80125 |
|
From: Joe M. <joe...@me...> - 2006-03-01 21:15:26
|
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Brendan Drew wrote: >> Some portions are, yes. Although I'll admit right now that I'm surprised >> if this were the case. The variables are initialized as they are >> declared. I believe (and please, correct me if I'm wrong) that static >> variables which are initialized in place (e.g. static int foo = 0; ) >> result in the compiler emitting code which has the static variable >> allocated and initialized in the data section (i.e. initialized as the >> program is read into memory). The C++ FAQ says, "static local objects are constructed the first time control flows over their declaration" (http://www.parashift.com/c++-faq-lite/ctors.html#faq-10.12). This is important because it allows the programmer to specify exactly the initialization order. (This only really matters if the initializer is a function call, or otherwise depends on other variables which need initialization.) I assume the compiler's allowed to initialize statics in the data section only if they're initialized with a compile-time constant. Joe -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2.1 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFEBg7jq5NnQEnPdLcRAsTBAJ4lY0W1z1rkEd69Sv5sQ0dJg5mTowCfVjWG yvWKNabIH3D6RN/oGFGYlNw= =WIXa -----END PGP SIGNATURE----- |