valgrind-users

[Valgrind-users] Mismatched free/delete with GCC 4.8.3 std::string

[David C. <dc...@gm...>]

Hi,

I've recently switched over to Valgrind 3.10.1 and I'm now see vast numbers
of 'mismatched free/delete' type messages all coming from std::string
shipped with GCC 4.8.3.

I really don't believe what Valgrind is saying but I'd like to be certain.
Firstly, has anyone else seen this behaviour recently and secondly what
criteria does Valgrind use to determine if a mismatch occurred?  Is it
possible to print out more information from Valgrind about its decision
process?

Many thanks,
David.

Re: [Valgrind-users] Mismatched free/delete with GCC 4.8.3 std::string

[Brian B. <bri...@gm...>]

Are you certain all of your libraries have been recompiled against the
newer libstdc++?

On Wed, Jun 10, 2015 at 11:09 AM, David Carter <dc...@gm...> wrote:
> Hi,
>
> I've recently switched over to Valgrind 3.10.1 and I'm now see vast numbers
> of 'mismatched free/delete' type messages all coming from std::string
> shipped with GCC 4.8.3.
>
> I really don't believe what Valgrind is saying but I'd like to be certain.
> Firstly, has anyone else seen this behaviour recently and secondly what
> criteria does Valgrind use to determine if a mismatch occurred?  Is it
> possible to print out more information from Valgrind about its decision
> process?
>
> Many thanks,
> David.
>
> ------------------------------------------------------------------------------
>
> _______________________________________________
> Valgrind-users mailing list
> Val...@li...
> https://lists.sourceforge.net/lists/listinfo/valgrind-users
>

Re: [Valgrind-users] Mismatched free/delete with GCC 4.8.3 std::string

[David C. <dc...@gm...>]

Fairly certain, yes.

On Wednesday, June 10, 2015, Brian Budge <bri...@gm...> wrote:

> Are you certain all of your libraries have been recompiled against the
> newer libstdc++?
>
> On Wed, Jun 10, 2015 at 11:09 AM, David Carter <dc...@gm...
> <javascript:;>> wrote:
> > Hi,
> >
> > I've recently switched over to Valgrind 3.10.1 and I'm now see vast
> numbers
> > of 'mismatched free/delete' type messages all coming from std::string
> > shipped with GCC 4.8.3.
> >
> > I really don't believe what Valgrind is saying but I'd like to be
> certain.
> > Firstly, has anyone else seen this behaviour recently and secondly what
> > criteria does Valgrind use to determine if a mismatch occurred?  Is it
> > possible to print out more information from Valgrind about its decision
> > process?
> >
> > Many thanks,
> > David.
> >
> >
> ------------------------------------------------------------------------------
> >
> > _______________________________________________
> > Valgrind-users mailing list
> > Val...@li... <javascript:;>
> > https://lists.sourceforge.net/lists/listinfo/valgrind-users
> >
>

Re: [Valgrind-users] Mismatched free/delete with GCC 4.8.3 std::string

[Philippe W. <phi...@sk...>]

On Wed, 2015-06-10 at 19:09 +0100, David Carter wrote:
> Hi,
> 
> 
> I've recently switched over to Valgrind 3.10.1 and I'm now see vast
> numbers of 'mismatched free/delete' type messages all coming from
> std::string shipped with GCC 4.8.3.
> 
> 
> I really don't believe what Valgrind is saying but I'd like to be
> certain.  Firstly, has anyone else seen this behaviour recently and
> secondly what criteria does Valgrind use to determine if a mismatch
> occurred?  Is it possible to print out more information from Valgrind
> about its decision process?
Valgrind memcheck understands 4 types of alloc functions:
malloc/new/new[]/custom
which must be freed by the corresponding free/delete/delete[]/custom
'freeing' function.

Normally, if such a mismatch error is detected, you get the allocation
stack trace and the stack trace at which the non matching freeing is
done. That should normally give enough information.

If not, you can always try to debug your program while running it under
valgrind, using gdb+vgdb.
But it is not clear to me what is missing when you have both
stacktraces.

Philippe
 

Re: [Valgrind-users] Mismatched free/delete with GCC 4.8.3 std::string

[<pa...@fr...>]

----- Original Message -----
> 
> Hi,
> 
> I've recently switched over to Valgrind 3.10.1 and I'm now see vast
> numbers of 'mismatched free/delete' type messages all coming from
> std::string shipped with GCC 4.8.3.
>
> I really don't believe what Valgrind is saying but I'd like to be
> certain. Firstly, has anyone else seen this behaviour recently and
> secondly what criteria does Valgrind use to determine if a mismatch
> occurred? Is it possible to print out more information from Valgrind
> about its decision process?


Hi David

Could you post an extract of the results?

Just to eliminate a few things. The string instance isn't itself newed?

I.e, you aren't doing something like

std::string* ps = new std::string;
...

free(ps);

I guess you would have seen this before.

You aren't overloading any operator new or delete or using placement new? You're using the default allocator std::allocator<char>?

Regards
Paul

Re: [Valgrind-users] Mismatched free/delete with GCC 4.8.3 std::string

[Julian S. <js...@ac...>]

> I've recently switched over to Valgrind 3.10.1 and I'm now see vast numbers
> of 'mismatched free/delete' type messages all coming from std::string
> shipped with GCC 4.8.3.

I've seen this a lot in the past year when working with Firefox.  I believe
that it is due to what you could call "differential inlining".  I'm not 100%
sure of the details, but the general problem is like this:

Memcheck intercepts malloc, free, new and delete.  It expects memory
allocated by malloc to be freed by free and memory allocated by new
to be freed by delete (and the same for new[] and delete[]).

Imagine now that some C++ header file contains something like this

   operator new ( size_t n ) { return malloc(n); }
   operator delete ( void* p ) { free(p); }

If g++ decides to inline new but not delete, or the other way round, then
the code still works (of course) but from Memcheck's point of view there is
a problem.  That's because it can't intercept the inlined function -- there
is no single piece of code to intercept.  So what it ends up seeing is,
for example, memory allocated by new (because that isn't inlined) but freed
by free (because delete got inlined).  So it complains -- incorrectly.

I couldn't figure out any sane way to work around this, so I added a new
flag, --show-mismatched-frees=no|yes [default=yes], to the trunk.  This
disables allocator-mismatch checking and gets rid of the noise, but it of
course also gets rid of the ability to detect genuine mismatch errors.

One other option is to play around with optimisation settings for your
app, to see if it changes g++'s inline/no-inline decisions.  For example,
at -O0 it may be that neither function is inlined, and at -O2 both are,
and it's only at -O1 (a.k.a -O) that you get into this differentially
inlined situation.  I've found -Og, which is available in 4.8 and later,
to be not-bad, but YMMV.  Plus, playing with optimisation options is
of limited use since normally you can only recompile your app, not the
vast stack of libraries on which most apps now depend.

J

Re: [Valgrind-users] Mismatched free/delete with GCC 4.8.3 std::string

[Dallman, J. <joh...@si...>]

Julian Seward wrote:

> One other option is to play around with optimisation settings for your app, to see if it changes g++'s inline/no-inline decisions.

g++'s -fno-inline option may help, although it has the same limitation of not applying to libraries.

--
John Dallman
-----------------
Siemens Industry Software Limited is a limited company registered in England and Wales.
Registered number: 3476850.
Registered office: Faraday House, Sir William Siemens Square, Frimley, Surrey, GU16 8QD.

Re: [Valgrind-users] Mismatched free/delete with GCC 4.8.3 std::string

[Philippe W. <phi...@sk...>]

On Thu, 2015-06-11 at 12:14 +0200, Julian Seward wrote:
> > I've recently switched over to Valgrind 3.10.1 and I'm now see vast numbers
> > of 'mismatched free/delete' type messages all coming from std::string
> > shipped with GCC 4.8.3.
Do you see these 'mismatches free/delete' only with Valgrind 3.10.1 ?
I.e. has it appeared with Valgrind 3.10.1 or has it appeared
with a switch to gcc 4.8.3 ?
> 
> I've seen this a lot in the past year when working with Firefox.  I believe
> that it is due to what you could call "differential inlining".  I'm not 100%
> sure of the details, but the general problem is like this:
> 
> Memcheck intercepts malloc, free, new and delete.  It expects memory
> allocated by malloc to be freed by free and memory allocated by new
> to be freed by delete (and the same for new[] and delete[]).
> 
> Imagine now that some C++ header file contains something like this
> 
>    operator new ( size_t n ) { return malloc(n); }
>    operator delete ( void* p ) { free(p); }
> 
> If g++ decides to inline new but not delete, or the other way round, then
> the code still works (of course) but from Memcheck's point of view there is
> a problem.  That's because it can't intercept the inlined function -- there
> is no single piece of code to intercept.  So what it ends up seeing is,
> for example, memory allocated by new (because that isn't inlined) but freed
> by free (because delete got inlined).  So it complains -- incorrectly.
If the problem is due to "differential inlining", then that should
be visible in the stacktraces either of the "new" or of the "delete" :
unless you specify --read-inline-info=no, valgrind stacktraces will
show both the "new/malloc" or "delete/free" in the stacktraces,
but with an identical program counter.

> 
> I couldn't figure out any sane way to work around this, so I added a new
> flag,


>  --show-mismatched-frees=no|yes [default=yes], to the trunk.  This
> disables allocator-mismatch checking and gets rid of the noise, but it of
> course also gets rid of the ability to detect genuine mismatch errors.
What might be done is (assuming that the inline info shows properly the
call to new/delete), is to have an additional check before reporting
a 'mismatched' error: if the stacktrace contains an inlined call to
the expected "freeing function", then do not report the error.
Eg, we might have a third value for --show-mismatched-frees=checkinline
which would activate this verification.
That should remove the false positive errors, and allow to
keep the true positive. The price to pay will be a translation
of program counters to function name and string comparison for
all cases of such "differential inlining" (and that might have to
be done both for the alloc and the free stacktrace).

Philippe

Re: [Valgrind-users] Mismatched free/delete with GCC 4.8.3 std::string

[David C. <dc...@gm...>]

For the alloc'd line Valgrind points at new_allocator.h line 110, the
de-alloc line is pointed to new_allocator.h line 104.  The first looks like
this in GCC sources:

return static_cast<_Tp*>(::operator new(__n * sizeof(_Tp)));

the second looks like this:

deallocate(pointer __p, size_type)
{ ::operator delete(__p); }

I don't see a mismatch there.

The particular use-case I'm looking at is in string_buf, but there are
literally thousands of others.

Regards,
David.

On Thu, Jun 11, 2015 at 6:58 PM, Philippe Waroquiers <
phi...@sk...> wrote:

> On Thu, 2015-06-11 at 12:14 +0200, Julian Seward wrote:
> > > I've recently switched over to Valgrind 3.10.1 and I'm now see vast
> numbers
> > > of 'mismatched free/delete' type messages all coming from std::string
> > > shipped with GCC 4.8.3.
> Do you see these 'mismatches free/delete' only with Valgrind 3.10.1 ?
> I.e. has it appeared with Valgrind 3.10.1 or has it appeared
> with a switch to gcc 4.8.3 ?
> >
> > I've seen this a lot in the past year when working with Firefox.  I
> believe
> > that it is due to what you could call "differential inlining".  I'm not
> 100%
> > sure of the details, but the general problem is like this:
> >
> > Memcheck intercepts malloc, free, new and delete.  It expects memory
> > allocated by malloc to be freed by free and memory allocated by new
> > to be freed by delete (and the same for new[] and delete[]).
> >
> > Imagine now that some C++ header file contains something like this
> >
> >    operator new ( size_t n ) { return malloc(n); }
> >    operator delete ( void* p ) { free(p); }
> >
> > If g++ decides to inline new but not delete, or the other way round, then
> > the code still works (of course) but from Memcheck's point of view there
> is
> > a problem.  That's because it can't intercept the inlined function --
> there
> > is no single piece of code to intercept.  So what it ends up seeing is,
> > for example, memory allocated by new (because that isn't inlined) but
> freed
> > by free (because delete got inlined).  So it complains -- incorrectly.
> If the problem is due to "differential inlining", then that should
> be visible in the stacktraces either of the "new" or of the "delete" :
> unless you specify --read-inline-info=no, valgrind stacktraces will
> show both the "new/malloc" or "delete/free" in the stacktraces,
> but with an identical program counter.
>
> >
> > I couldn't figure out any sane way to work around this, so I added a new
> > flag,
>
>
> >  --show-mismatched-frees=no|yes [default=yes], to the trunk.  This
> > disables allocator-mismatch checking and gets rid of the noise, but it of
> > course also gets rid of the ability to detect genuine mismatch errors.
> What might be done is (assuming that the inline info shows properly the
> call to new/delete), is to have an additional check before reporting
> a 'mismatched' error: if the stacktrace contains an inlined call to
> the expected "freeing function", then do not report the error.
> Eg, we might have a third value for --show-mismatched-frees=checkinline
> which would activate this verification.
> That should remove the false positive errors, and allow to
> keep the true positive. The price to pay will be a translation
> of program counters to function name and string comparison for
> all cases of such "differential inlining" (and that might have to
> be done both for the alloc and the free stacktrace).
>
> Philippe
>
>
>

Re: [Valgrind-users] Mismatched free/delete with GCC 4.8.3 std::string

[Philippe W. <phi...@sk...>]

Thanks for the below info.
Can you post the full stacktraces of a mismatch error
(including program counters etc) ?

Also, can you confirm if the mismatch errors
appeared with a switch to valgrind 3.10.1 ?
Or with a switch to gcc 4.8.3 ?

Thanks
Philippe


On Thu, 2015-06-11 at 23:29 +0100, David Carter wrote:
> For the alloc'd line Valgrind points at new_allocator.h line 110, the
> de-alloc line is pointed to new_allocator.h line 104.  The first looks
> like this in GCC sources:
> 
> 
> return static_cast<_Tp*>(::operator new(__n * sizeof(_Tp)));
> 
> 
> the second looks like this:
> 
> 
> deallocate(pointer __p, size_type)
> { ::operator delete(__p); }
> 
> 
> I don't see a mismatch there.
> 
> 
> The particular use-case I'm looking at is in string_buf, but there are
> literally thousands of others.
> 
> 
> Regards,
> David.
> 
> On Thu, Jun 11, 2015 at 6:58 PM, Philippe Waroquiers
> <phi...@sk...> wrote:
>         On Thu, 2015-06-11 at 12:14 +0200, Julian Seward wrote:
>         > > I've recently switched over to Valgrind 3.10.1 and I'm now
>         see vast numbers
>         > > of 'mismatched free/delete' type messages all coming from
>         std::string
>         > > shipped with GCC 4.8.3.
>         Do you see these 'mismatches free/delete' only with Valgrind
>         3.10.1 ?
>         I.e. has it appeared with Valgrind 3.10.1 or has it appeared
>         with a switch to gcc 4.8.3 ?
>         >
>         > I've seen this a lot in the past year when working with
>         Firefox.  I believe
>         > that it is due to what you could call "differential
>         inlining".  I'm not 100%
>         > sure of the details, but the general problem is like this:
>         >
>         > Memcheck intercepts malloc, free, new and delete.  It
>         expects memory
>         > allocated by malloc to be freed by free and memory allocated
>         by new
>         > to be freed by delete (and the same for new[] and delete[]).
>         >
>         > Imagine now that some C++ header file contains something
>         like this
>         >
>         >    operator new ( size_t n ) { return malloc(n); }
>         >    operator delete ( void* p ) { free(p); }
>         >
>         > If g++ decides to inline new but not delete, or the other
>         way round, then
>         > the code still works (of course) but from Memcheck's point
>         of view there is
>         > a problem.  That's because it can't intercept the inlined
>         function -- there
>         > is no single piece of code to intercept.  So what it ends up
>         seeing is,
>         > for example, memory allocated by new (because that isn't
>         inlined) but freed
>         > by free (because delete got inlined).  So it complains --
>         incorrectly.
>         If the problem is due to "differential inlining", then that
>         should
>         be visible in the stacktraces either of the "new" or of the
>         "delete" :
>         unless you specify --read-inline-info=no, valgrind stacktraces
>         will
>         show both the "new/malloc" or "delete/free" in the
>         stacktraces,
>         but with an identical program counter.
>         
>         >
>         > I couldn't figure out any sane way to work around this, so I
>         added a new
>         > flag,
>         
>         
>         >  --show-mismatched-frees=no|yes [default=yes], to the trunk.
>         This
>         > disables allocator-mismatch checking and gets rid of the
>         noise, but it of
>         > course also gets rid of the ability to detect genuine
>         mismatch errors.
>         What might be done is (assuming that the inline info shows
>         properly the
>         call to new/delete), is to have an additional check before
>         reporting
>         a 'mismatched' error: if the stacktrace contains an inlined
>         call to
>         the expected "freeing function", then do not report the error.
>         Eg, we might have a third value for
>         --show-mismatched-frees=checkinline
>         which would activate this verification.
>         That should remove the false positive errors, and allow to
>         keep the true positive. The price to pay will be a translation
>         of program counters to function name and string comparison for
>         all cases of such "differential inlining" (and that might have
>         to
>         be done both for the alloc and the free stacktrace).
>         
>         Philippe
>         
>         
> 
>