Re: [Valgrind-users] Are valgrind leak-check results valid if target allocates a block larger than 4G?

[Bob H. <rsh...@gm...>]

On Mar 31, 2011, at 4:25 PM, Tom Hughes wrote:
> I've looked back at your original post and the main problem with it is that
> you didn't really provide any information (like the text of the error reports
> from valgrind) that might let us help you.

Indeed, though I did paraphrase the reports in an attempt to keep the message
shorter.  I originally wrote it much longer, decided the specific error text
would be TMI, and shortened that.  After getting no reponses, one of the things
I did do in my second post was include the complete error text.

How 'bout we forget the first post, which everyone ignored anyway, and
concentrate on the second one.

> Instead you seemed to have jumped to a conclusion that valgrind is somehow
> broken and start speculating that maybe it can't handle large chunks of
> memory.

True, that is the conclusion I jumped to.  Limited info in the manual regarding
"address range perms" was strongly suggestive that something exceptional had
happened inside valgrind, something that it wasn't written to handle.

> The simple answer is that, assuming you don't actually run of of memory or
> address space, valgrind really couldn't care less how small or large the
> chunks of memory you use are. If you were to run of space then you would get
> an error.

Thanks, it is helpful to know valgrind will report such an error. I'm
assuming you
mean that valgrind would report an error in that case, that a call to
malloc or realloc
has failed.

And it is helpful to know that valgrind doesn't care about the size of my
memory blocks.  Though I can imagine how valgrind is implemented, I don't know
the specifics.  One can imagine that valgrind itself must be allocating *some*
memory for its own use.  Theoretically this could push it past some allocation
limit.  One can also imagine that valgrind has some shortcoming like tracking
block sizes in a 32-bit variable. I'm not saying this is true, I'm just saying
that if one doesn't know the internals and sees the information I saw, those
are valid possible sources of the problem.

So one question I have though is what "you" means in your last sentence.  Does
it mean that my target program has run out of memory, or the combination of
valgrind and my target, together, has run out of memory.

> Valgrind has told you that you are reading more data into a buffer than you
> have allocated space for - it is almost certainly correct. Trust it and
> investigate the problem it is telling you about.

Hmmm... that may be true in the original target (though, at the risk
of stepping on
toes, I am still not convinced that this isn't an artifact of a
failure of large memory
allocation when valgrind is present).  But in the simple test case I
don't do anything
with the allocated memory but free it.  And yet I get some of the same
complaints
from valgrind.

> Ignore the segfault for now - that is just a consequence of you overrunning
> the buffer. It not happening outside of valgrind is of no consequence, it
> just means you are getting lucky. Under valgrind the memory layout will be
> different and you are happening to overwrite something that then causes the
> crash.

Yep.  I do understand all that.  I've been a working programmer for 35 years.
 About a third of that experience was writing in assembly language, and in fact
implementing malloc-type allocators in several architectures.  And I'm
not saying
that to puff up my chest or anything, just to give you an idea what my level of
knowledge is.

Because of the behavior of valgrind on the test case, it seems the best avenue
to pursue is why the test case causes valgrind to report that a block hasn't
been freed, when clearly it *has* been freed.  In the test case the only
possibilities for where the error is are (a) runtime library, (b) compiler, or
(c) valgrind.

> And yes, as John has told you, you can ignore those warnings about large
> address ranges.

Hmmm... but looking at the failure of my test program, the large address
warnings look like the only useful clues.

Thanks,
Bob H