valgrind-users

[Valgrind-users] Errors with gcc -static

[Zachary T. <div...@gm...>]

When I build any program (including an empty main function) using gcc
-static I get enormous amounts of errors in valgrind.  After some
investigation and asking around I think that at least a large portion
of them are incorrect.  In particular, I get many errors about
uninitialized memory and conditional jumps, but when I use
--track-origins=yes I find that the memory it's claiming is
uninitialized comes from sbrk().  As far as I can tell (please correct
me if I'm wrong) this function is guaranteed to return 0-filled
memory, so while it's true that the program is not explicitly
initializing, it should be initialized anyway.

Assuming this assessment is accurate, is it best to deal with this by
adding appropriate suppressions to the default suppressions file, or
by modifying valgrind to be smarter and know that sbrk memory is
always 0-filled?

There are a number of other types of errors that seem to be generated
quite commonly when using -static, but so far I have not been able to
determine if those are harmless.

Re: [Valgrind-users] Errors with gcc -static

[Tom H. <to...@co...>]

On 16/07/09 17:16, Zachary Turner wrote:

> When I build any program (including an empty main function) using gcc
> -static I get enormous amounts of errors in valgrind.  After some
> investigation and asking around I think that at least a large portion
> of them are incorrect.  In particular, I get many errors about
> uninitialized memory and conditional jumps, but when I use
> --track-origins=yes I find that the memory it's claiming is
> uninitialized comes from sbrk().  As far as I can tell (please correct
> me if I'm wrong) this function is guaranteed to return 0-filled
> memory, so while it's true that the program is not explicitly
> initializing, it should be initialized anyway.

When you statically link a program you prevent valgrind from being able 
to intercept the malloc and free functions so you will find you get lots 
of warnings as a result.

Tom

-- 
Tom Hughes (to...@co...)
http://www.compton.nu/

Re: [Valgrind-users] Errors with gcc -static

[Tim P. <ec...@ec...>]

On Thu, 2009-07-16 at 11:16 -0500, Zachary Turner wrote:

> Assuming this assessment is accurate, is it best to deal with this by
> adding appropriate suppressions to the default suppressions file, or
> by modifying valgrind to be smarter and know that sbrk memory is
> always 0-filled?

Its best to test without statically linking, then release after linking
as you need.

Doing the reverse hides bugs you really _want_ to know about. Valgrind
is clever, but not psychic. Test with dynamic linkage, if that looks
good, you have already ruled out the noise that production optimizations
and static libs produce.

Then there's the corner case, when you are provided only with static
libs to link with. In that case, do what you can. However, that remains
a corner case :) 

Cheers,
--Tim

Re: [Valgrind-users] Errors with gcc -static

[Tom H. <to...@co...>]

On 16/07/09 19:23, Tim Post wrote:

> Then there's the corner case, when you are provided only with static
> libs to link with. In that case, do what you can. However, that remains
> a corner case :)

There is no problem linking with static libraries in general, so long as 
you don't make the program completely static, and so long as the C 
library is dynamic so we can pick up malloc/free etc.

Tom

-- 
Tom Hughes (to...@co...)
http://www.compton.nu/

Re: [Valgrind-users] Errors with gcc -static

[Nicholas N. <n.n...@gm...>]

On Fri, Jul 17, 2009 at 2:16 AM, Zachary Turner<div...@gm...> wrote:
> --track-origins=yes I find that the memory it's claiming is
> uninitialized comes from sbrk().  As far as I can tell (please correct
> me if I'm wrong) this function is guaranteed to return 0-filled
> memory

I'm not at all certain this is the case, which is why Memcheck marks
it as undefined.  Evidence to the contrary would be welcome!

Nick

Re: [Valgrind-users] Errors with gcc -static

[Zachary T. <div...@gm...>]

On Thu, Jul 16, 2009 at 7:47 PM, Nicholas
Nethercote<n.n...@gm...> wrote:
> On Fri, Jul 17, 2009 at 2:16 AM, Zachary Turner<div...@gm...> wrote:
>> --track-origins=yes I find that the memory it's claiming is
>> uninitialized comes from sbrk().  As far as I can tell (please correct
>> me if I'm wrong) this function is guaranteed to return 0-filled
>> memory
>
> I'm not at all certain this is the case, which is why Memcheck marks
> it as undefined.  Evidence to the contrary would be welcome!
>
> Nick
>

I'm not certain either :)  When I google around it seems about 50/50
with some saying it does and some saying it doesn't.  But it might be
platform dependent.  The glibc source code's implementation of malloc
uses sbrk and has a comment that says "since sbrk() returns 0-filled
memory, we don't need to initialize the entire buffer." or something
to that effect.

Re: [Valgrind-users] Errors with gcc -static

[John R. <jr...@bi...>]

Nicholas Nethercote wrote:
> On Fri, Jul 17, 2009 at 2:16 AM, Zachary Turner<div...@gm...> wrote:
>> --track-origins=yes I find that the memory it's claiming is
>> uninitialized comes from sbrk().  As far as I can tell (please correct
>> me if I'm wrong) this function is guaranteed to return 0-filled
>> memory

> I'm not at all certain this is the case, which is why Memcheck marks
> it as undefined.  Evidence to the contrary would be welcome!

0)  sbrk() can *decrease* process address space.  No zero fill is done
    for a decrease, not even the fragment on the high end of the last page
    that is beyond the new highest address.  For maximum safety and
    portability, then the bytes in the last page that reside above [the new]
    sbrk(0) should be considered to be uninitialized, but in practice
    it is exceedingly likely that they will retain their previous contents.

1)  If an increase is large enough to require new whole pages,
    then those new whole pages (like all new pages) are zero-filled
    by the operating system.  So if sbrk(0) already is page aligned,
    then sbrk(PAGE_SIZE) *does* zero-fill the new memory.

2)  Any increase that lies within an existing allocated page
    is not changed.  So if (x= sbrk(0)) is not page aligned, then
    sbrk(PAGE_SIZE) yields ((PAGE_SIZE -1) & -x) bytes which
    keep their existing contents, and an additional PAGE_SIZE
    bytes which are zeroed.  ((PAGE_SIZE -1) & x) of them are
    "covered" by the sbrk(), and the rest of them come along for
    the ride because the operating system deals only in whole pages.
    Again, for maximum safety and portability, then anything that
    lives above [the new] sbrk(0) should be considered uninitialized,
    but in practice will retain previous contents [zero in this case.]

Also remember that sbrk(...) can fail  :-)

-- 

Re: [Valgrind-users] Errors with gcc -static

[Nicholas N. <n.n...@gm...>]

On Sat, Jul 18, 2009 at 12:55 AM, John Reiser<jr...@bi...> wrote:
> Nicholas Nethercote wrote:
>
> 0)  sbrk() can *decrease* process address space.  No zero fill is done
>    for a decrease, not even the fragment on the high end of the last page
>    that is beyond the new highest address.  For maximum safety and
>    portability, then the bytes in the last page that reside above [the new]
>    sbrk(0) should be considered to be uninitialized, but in practice
>    it is exceedingly likely that they will retain their previous contents.
>
> 1)  If an increase is large enough to require new whole pages,
>    then those new whole pages (like all new pages) are zero-filled
>    by the operating system.  So if sbrk(0) already is page aligned,
>    then sbrk(PAGE_SIZE) *does* zero-fill the new memory.
>
> 2)  Any increase that lies within an existing allocated page
>    is not changed.  So if (x= sbrk(0)) is not page aligned, then
>    sbrk(PAGE_SIZE) yields ((PAGE_SIZE -1) & -x) bytes which
>    keep their existing contents, and an additional PAGE_SIZE
>    bytes which are zeroed.  ((PAGE_SIZE -1) & x) of them are
>    "covered" by the sbrk(), and the rest of them come along for
>    the ride because the operating system deals only in whole pages.
>    Again, for maximum safety and portability, then anything that
>    lives above [the new] sbrk(0) should be considered uninitialized,
>    but in practice will retain previous contents [zero in this case.]
>
> Also remember that sbrk(...) can fail  :-)

Thanks, John.  Is this info specific to Linux/glibc?  AFAICT sbrk/brk
isn't even POSIX, which makes it even harder to determine what it
should do.

Nick

Re: [Valgrind-users] Errors with gcc -static

[John R. <jr...@bi...>]

> Is this info specific to Linux/glibc?  AFAICT sbrk/brk
> isn't even POSIX, which makes it even harder to determine what it
> should do.

sbrk and brk are not POSIX.  However, any system that is a derivative
of *nix has sbrk and brk because there are too many softwares (such
as the Bourne shell) which rely on the traditional memory map
(.text, .data+.bss, stack) and the existence of sbrk/brk.
A key property of sbrk/brk is that new whole pages that
are supplied by the operating system *do* get initialized to zero.
Memcheck must support and honor this property.

-- 
John Reiser, jr...@Bi...

Re: [Valgrind-users] Errors with gcc -static

[Nicholas N. <n.n...@gm...>]

On Mon, Jul 20, 2009 at 2:13 PM, John Reiser<jr...@bi...> wrote:
>> Is this info specific to Linux/glibc?  AFAICT sbrk/brk
>> isn't even POSIX, which makes it even harder to determine what it
>> should do.
>
> sbrk and brk are not POSIX.  However, any system that is a derivative
> of *nix has sbrk and brk because there are too many softwares (such
> as the Bourne shell) which rely on the traditional memory map
> (.text, .data+.bss, stack) and the existence of sbrk/brk.

AFAICT on Mac OS X the syscall brk() does not exist.  The library
function sbrk() does, and it's implemented using the mach trap
vm_allocate() which does initialise memory.  (I'm not making any kind
of argument here, I just thought you'd be interested to hear this.)

Nick