Re: [Valgrind-users] novice drd user. Errors in std streams

SourceForge Headquarters 1320 Columbia Street Suite 310 San Diego, CA 92101 +1 (858) 422-6466

The race on _M_mutate is (most likely) the one discussed at
http://permalink.gmane.org/gmane.comp.debugging.valgrind/10043
 <http://permalink.gmane.org/gmane.comp.debugging.valgrind/10043>--kcc

On Tue, Apr 6, 2010 at 4:14 PM, Jorge Moraleda <jor...@gm...>wrote:

> On Tue, Apr 6, 2010 at 3:30 AM, Bart Van Assche <bva...@ac...>
> wrote:
> > On Tue, Apr 6, 2010 at 12:25 PM, Konstantin Serebryany
> > <kon...@gm...> wrote:
> >>
> >>
> >> On Tue, Apr 6, 2010 at 2:17 PM, Bart Van Assche <bva...@ac...>
> wrote:
> >>>
> >>> On Tue, Apr 6, 2010 at 7:53 AM, Jorge Moraleda <
> jor...@gm...>
> >>> wrote:
> >>> >
> >>> > Dear all,
> >>> >
> >>> > When I compile the following program with:
> >>> > $ g++ -g -pthread threads.cpp
> >>> >
> >>> > // begin program /////////////////////////////////////////////
> >>> > // file: threads.cpp
> >>> > #include <pthread.h>
> >>> > #include <iostream>
> >>> > #include <sstream>
> >>> >
> >>> > void *threadEntry(void *threadid)
> >>> > {
> >>> >   long tid;
> >>> >   tid = (long)threadid;
> >>> >         for (int i = 0; i<5; i++) {
> >>> >                   std::stringstream myStream;
> >>> >                         myStream << "this thread is " << tid;
> >>> >                         std::string myString(myStream.str());
> >>> >                   pthread_yield();
> >>> >         }
> >>> >         pthread_exit(NULL);
> >>> > }
> >>> >
> >>> > int main (int argc, char *argv[])
> >>> > {
> >>> >   pthread_t threads[2];
> >>> >   int rc;
> >>> >   long t;
> >>> >   for(t=0; t<2; t++) {
> >>> >      rc = pthread_create(&threads[t], NULL, threadEntry, (void *)t);
> >>> >   }
> >>> >   pthread_exit(NULL);
> >>> > }
> >>> > // end program /////////////////////////////////////////////
> >>> >
> >>> > and run drd on it with:
> >>> > $ valgrind --tool=drd ./a.out
> >>> >
> >>> > I get the following output:
> >>> >
> >>> > ==16240== drd, a thread error detector
> >>> > ==16240== Copyright (C) 2006-2009, and GNU GPL'd, by Bart Van Assche.
> >>> > ==16240== Using Valgrind-3.5.0-Debian and LibVEX; rerun with -h for
> >>> > copyright info
> >>> > ==16240== Command: ./a.out
> >>> > ==16240==
> >>> > ==16240== Thread 3:
> >>> > ==16240== Conflicting load by thread 3 at 0x05132e98 size 1
> >>> > ==16240==    at 0x4ED44C8: std::ostream&
> >>> > std::ostream::_M_insert<long>(long) (in /usr/lib/libstdc++.so.6.0.13)
> >>> > ==16240==    by 0x400C48: threadEntry(void*) (threads.cpp:12)
> >>> > ==16240==    by 0x4C32870: vgDrd_thread_wrapper
> >>> > (drd_pthread_intercepts.c:272)
> >>> > ==16240==    by 0x55E8A03: start_thread (pthread_create.c:300)
> >>> > ==16240==    by 0x58DD80C: clone (clone.S:112)
> >>> > ==16240== Allocation context: BSS section of
> >>> > /usr/lib/libstdc++.so.6.0.13
> >>> > ==16240== Other segment start (thread 2)
> >>> > ==16240==    at 0x4C29F2F: pthread_mutex_unlock
> >>> > (drd_pthread_intercepts.c:633)
> >>> > ==16240==    by 0x4EA387E: std::locale::locale() (in
> >>> > /usr/lib/libstdc++.so.6.0.13)
> >>> > ==16240==    by 0x4E9FE5F: std::ios_base::_M_init() (in
> >>> > /usr/lib/libstdc++.so.6.0.13)
> >>> > ==16240==    by 0x4EB4768: std::basic_ios<char,
> std::char_traits<char>
> >>> > >::init(std::basic_streambuf<char, std::char_traits<char> >*) (in
> >>> > /usr/lib/libstdc++.so.6.0.13)
> >>> > ==16240==    by 0x4ED7DEA: std::basic_stringstream<char,
> >>> > std::char_traits<char>, std::allocator<char>
> >>> > >::basic_stringstream(std::_Ios_Openmode) (in
> >>> > /usr/lib/libstdc++.so.6.0.13)
> >>> > ==16240==    by 0x400C21: threadEntry(void*) (threads.cpp:11)
> >>> > ==16240==    by 0x4C32870: vgDrd_thread_wrapper
> >>> > (drd_pthread_intercepts.c:272)
> >>> > ==16240==    by 0x55E8A03: start_thread (pthread_create.c:300)
> >>> > ==16240==    by 0x58DD80C: clone (clone.S:112)
> >>> > ==16240== Other segment end (thread 2)
> >>> > ==16240==    at 0x58ACEB7: sched_yield (in /lib/libc-2.10.1.so)
> >>> > ==16240==    by 0x400C63: threadEntry(void*) (threads.cpp:14)
> >>> > ==16240==    by 0x4C32870: vgDrd_thread_wrapper
> >>> > (drd_pthread_intercepts.c:272)
> >>> > ==16240==    by 0x55E8A03: start_thread (pthread_create.c:300)
> >>> > ==16240==    by 0x58DD80C: clone (clone.S:112)
> >>> > ==16240==
> >>> > ==16240== Conflicting load by thread 3 at 0x05132eb9 size 1
> >>> > ==16240==    at 0x4ED44D3: std::ostream&
> >>> > std::ostream::_M_insert<long>(long) (in /usr/lib/libstdc++.so.6.0.13)
> >>> > ==16240==    by 0x400C48: threadEntry(void*) (threads.cpp:12)
> >>> > ==16240==    by 0x4C32870: vgDrd_thread_wrapper
> >>> > (drd_pthread_intercepts.c:272)
> >>> > ==16240==    by 0x55E8A03: start_thread (pthread_create.c:300)
> >>> > ==16240==    by 0x58DD80C: clone (clone.S:112)
> >>> > ==16240== Allocation context: BSS section of
> >>> > /usr/lib/libstdc++.so.6.0.13
> >>> > ==16240== Other segment start (thread 2)
> >>> > ==16240==    at 0x4C29F2F: pthread_mutex_unlock
> >>> > (drd_pthread_intercepts.c:633)
> >>> > ==16240==    by 0x4EA387E: std::locale::locale() (in
> >>> > /usr/lib/libstdc++.so.6.0.13)
> >>> > ==16240==    by 0x4E9FE5F: std::ios_base::_M_init() (in
> >>> > /usr/lib/libstdc++.so.6.0.13)
> >>> > ==16240==    by 0x4EB4768: std::basic_ios<char,
> std::char_traits<char>
> >>> > >::init(std::basic_streambuf<char, std::char_traits<char> >*) (in
> >>> > /usr/lib/libstdc++.so.6.0.13)
> >>> > ==16240==    by 0x4ED7DEA: std::basic_stringstream<char,
> >>> > std::char_traits<char>, std::allocator<char>
> >>> > >::basic_stringstream(std::_Ios_Openmode) (in
> >>> > /usr/lib/libstdc++.so.6.0.13)
> >>> > ==16240==    by 0x400C21: threadEntry(void*) (threads.cpp:11)
> >>> > ==16240==    by 0x4C32870: vgDrd_thread_wrapper
> >>> > (drd_pthread_intercepts.c:272)
> >>> > ==16240==    by 0x55E8A03: start_thread (pthread_create.c:300)
> >>> > ==16240==    by 0x58DD80C: clone (clone.S:112)
> >>> > ==16240== Other segment end (thread 2)
> >>> > ==16240==    at 0x58ACEB7: sched_yield (in /lib/libc-2.10.1.so)
> >>> > ==16240==    by 0x400C63: threadEntry(void*) (threads.cpp:14)
> >>> > ==16240==    by 0x4C32870: vgDrd_thread_wrapper
> >>> > (drd_pthread_intercepts.c:272)
> >>> > ==16240==    by 0x55E8A03: start_thread (pthread_create.c:300)
> >>> > ==16240==    by 0x58DD80C: clone (clone.S:112)
> >>> > ==16240==
> >>> > ==16240==
> >>> > ==16240== For counts of detected and suppressed errors, rerun with:
> -v
> >>> > ==16240== ERROR SUMMARY: 6 errors from 2 contexts (suppressed: 208
> from
> >>> > 140)
> >>> >
> >>> > My question is: Are the errors reported real multi threading errors?
> >>> > If not, how can I tell drd to suppress them? (In my real world
> program,
> >>> > I
> >>> > have thousands of errors in hundreds of contexts, and I am trying to
> >>> > find the real ones.)
> >>> > If yes, can you help me understand what is wrong? Both threads only
> >>> > use local variables. I have been reading about the global object
> >>> > "locale" used by streams for localization, but I have read that it is
> >>> > protected by a global mutex.
> >>>
> >>> Hello Jorge,
> >>>
> >>> Unfortunately not all libraries have been designed with data-race
> >>> detection tools in mind. Several libraries contain code that triggers
> >>> benign data races. Examples are the I/O code in libstdc++ and in libc.
> >>>
> >>> You can either create a suppression pattern to suppress the above
> >>> races, or even simpler, add the following code in main() before thread
> >>> creation starts:
> >>>
> >>>  std::ostringstream dummy;
> >>>  dummy << 0;
> >>>
> >>> The above code will make sure that locale initialization, which is
> >>> triggered by sending an number to an I/O stream, will happen before
> >>> any threads are created and hence no races will be reported anymore on
> >>> locale initialization.
> >>
> >> This will not hide the race on _ZNSs4_Rep20_S_empty_rep_storageE.
> >> And suppressing errors in string guts may hide real races.
> >
> > The above indeed won't hide races on
> > _ZNSs4_Rep20_S_empty_rep_storageE. But I've never claimed that the
> > above code would hide these races. And Jorge didn't report any such
> > races in his original e-mail. So your reply seems off-topic to me.
> >
> > Bart.
> >
>
> Dear Bart and Konstantin,
>
> Thank you very much for your comments. The trick of pre-initializing
> the locale object solved a few of my errors, but I still have tons
> related to strings and streams. This is another minimal test case:
>
> #include <pthread.h>
> #include <string>
> #include <sstream>
>
> void *threadEntry(void *threadid)
> {
>   long tid;
>   tid = (long)threadid;
>          std::string myString;
>          for (int i = 0; i<5; i++) {
>                  myString.clear();
>                  pthread_yield();
>         }
>         pthread_exit(NULL);
> }
>
> int main (int argc, char *argv[])
> {
>   pthread_t threads[2];
>   int rc;
>   long t;
>
>          std::ostringstream dummy;
>         dummy << 0;
>          for(t=0; t<2; t++) {
>                 rc = pthread_create(&threads[t], NULL, threadEntry, (void
> *)t);
>         }
>         pthread_exit(NULL);
> }
>
> ==19065== drd, a thread error detector
> ==19065== Copyright (C) 2006-2009, and GNU GPL'd, by Bart Van Assche.
> ==19065== Using Valgrind-3.5.0-Debian and LibVEX; rerun with -h for
> copyright info
> ==19065== Command: ./a.out
> ==19065==
> ==19065== Thread 3:
> ==19065== Conflicting load by thread 3 at 0x05134160 size 8
> ==19065==    at 0x4EDF1D7: std::string::clear() (in
> /usr/lib/libstdc++.so.6.0.13)
> ==19065==    by 0x400A5B: threadEntry(void*) (threads.cpp:12)
> ==19065==    by 0x4C32870: vgDrd_thread_wrapper
> (drd_pthread_intercepts.c:272)
> ==19065==    by 0x55E8A03: start_thread (pthread_create.c:300)
> ==19065==    by 0x58DD80C: clone (clone.S:112)
> ==19065== Allocation context: BSS section of /usr/lib/libstdc++.so.6.0.13
> ==19065== Other segment start (thread 2)
> ==19065==    at 0x58DD7D1: clone (clone.S:84)
> ==19065==    by 0x55E893F: ??? (allocatestack.c:743)
> ==19065==    by 0x676D90F: ???
> ==19065== Other segment end (thread 2)
> ==19065==    at 0x58ACEB7: sched_yield (in /lib/libc-2.10.1.so)
> ==19065==    by 0x400A60: threadEntry(void*) (threads.cpp:13)
> ==19065==    by 0x4C32870: vgDrd_thread_wrapper
> (drd_pthread_intercepts.c:272)
> ==19065==    by 0x55E8A03: start_thread (pthread_create.c:300)
> ==19065==    by 0x58DD80C: clone (clone.S:112)
> ==19065==
> ==19065== Conflicting load by thread 3 at 0x05134160 size 8
> ==19065==    at 0x4EDE907: std::string::_M_mutate(unsigned long,
> unsigned long, unsigned long) (in /usr/lib/libstdc++.so.6.0.13)
> ==19065==    by 0x400A5B: threadEntry(void*) (threads.cpp:12)
> ==19065==    by 0x4C32870: vgDrd_thread_wrapper
> (drd_pthread_intercepts.c:272)
> ==19065==    by 0x55E8A03: start_thread (pthread_create.c:300)
> ==19065==    by 0x58DD80C: clone (clone.S:112)
> ==19065== Allocation context: BSS section of /usr/lib/libstdc++.so.6.0.13
> ==19065== Other segment start (thread 2)
> ==19065==    at 0x58DD7D1: clone (clone.S:84)
> ==19065==    by 0x55E893F: ??? (allocatestack.c:743)
> ==19065==    by 0x676D90F: ???
> ==19065== Other segment end (thread 2)
> ==19065==    at 0x58ACEB7: sched_yield (in /lib/libc-2.10.1.so)
> ==19065==    by 0x400A60: threadEntry(void*) (threads.cpp:13)
> ==19065==    by 0x4C32870: vgDrd_thread_wrapper
> (drd_pthread_intercepts.c:272)
> ==19065==    by 0x55E8A03: start_thread (pthread_create.c:300)
> ==19065==    by 0x58DD80C: clone (clone.S:112)
> ==19065==
> ==19065== Conflicting load by thread 3 at 0x05134170 size 4
> ==19065==    at 0x4EDE9B0: std::string::_M_mutate(unsigned long,
> unsigned long, unsigned long) (in /usr/lib/libstdc++.so.6.0.13)
> ==19065==    by 0x400A5B: threadEntry(void*) (threads.cpp:12)
> ==19065==    by 0x4C32870: vgDrd_thread_wrapper
> (drd_pthread_intercepts.c:272)
> ==19065==    by 0x55E8A03: start_thread (pthread_create.c:300)
> ==19065==    by 0x58DD80C: clone (clone.S:112)
> ==19065== Allocation context: BSS section of /usr/lib/libstdc++.so.6.0.13
> ==19065== Other segment start (thread 2)
> ==19065==    at 0x58DD7D1: clone (clone.S:84)
> ==19065==    by 0x55E893F: ??? (allocatestack.c:743)
> ==19065==    by 0x676D90F: ???
> ==19065== Other segment end (thread 2)
> ==19065==    at 0x58ACEB7: sched_yield (in /lib/libc-2.10.1.so)
> ==19065==    by 0x400A60: threadEntry(void*) (threads.cpp:13)
> ==19065==    by 0x4C32870: vgDrd_thread_wrapper
> (drd_pthread_intercepts.c:272)
> ==19065==    by 0x55E8A03: start_thread (pthread_create.c:300)
> ==19065==    by 0x58DD80C: clone (clone.S:112)
> ==19065==
> ==19065== Conflicting store by thread 3 at 0x05134170 size 4
> ==19065==    at 0x4EDE97D: std::string::_M_mutate(unsigned long,
> unsigned long, unsigned long) (in /usr/lib/libstdc++.so.6.0.13)
> ==19065==    by 0x400A5B: threadEntry(void*) (threads.cpp:12)
> ==19065==    by 0x4C32870: vgDrd_thread_wrapper
> (drd_pthread_intercepts.c:272)
> ==19065==    by 0x55E8A03: start_thread (pthread_create.c:300)
> ==19065==    by 0x58DD80C: clone (clone.S:112)
> ==19065== Allocation context: BSS section of /usr/lib/libstdc++.so.6.0.13
> ==19065== Other segment start (thread 2)
> ==19065==    at 0x58DD7D1: clone (clone.S:84)
> ==19065==    by 0x55E893F: ??? (allocatestack.c:743)
> ==19065==    by 0x676D90F: ???
> ==19065== Other segment end (thread 2)
> ==19065==    at 0x58ACEB7: sched_yield (in /lib/libc-2.10.1.so)
> ==19065==    by 0x400A60: threadEntry(void*) (threads.cpp:13)
> ==19065==    by 0x4C32870: vgDrd_thread_wrapper
> (drd_pthread_intercepts.c:272)
> ==19065==    by 0x55E8A03: start_thread (pthread_create.c:300)
> ==19065==    by 0x58DD80C: clone (clone.S:112)
> ==19065==
> ==19065== Conflicting store by thread 3 at 0x05134160 size 8
> ==19065==    at 0x4EDE984: std::string::_M_mutate(unsigned long,
> unsigned long, unsigned long) (in /usr/lib/libstdc++.so.6.0.13)
> ==19065==    by 0x400A5B: threadEntry(void*) (threads.cpp:12)
> ==19065==    by 0x4C32870: vgDrd_thread_wrapper
> (drd_pthread_intercepts.c:272)
> ==19065==    by 0x55E8A03: start_thread (pthread_create.c:300)
> ==19065==    by 0x58DD80C: clone (clone.S:112)
> ==19065== Allocation context: BSS section of /usr/lib/libstdc++.so.6.0.13
> ==19065== Other segment start (thread 2)
> ==19065==    at 0x58DD7D1: clone (clone.S:84)
> ==19065==    by 0x55E893F: ??? (allocatestack.c:743)
> ==19065==    by 0x676D90F: ???
> ==19065== Other segment end (thread 2)
> ==19065==    at 0x58ACEB7: sched_yield (in /lib/libc-2.10.1.so)
> ==19065==    by 0x400A60: threadEntry(void*) (threads.cpp:13)
> ==19065==    by 0x4C32870: vgDrd_thread_wrapper
> (drd_pthread_intercepts.c:272)
> ==19065==    by 0x55E8A03: start_thread (pthread_create.c:300)
> ==19065==    by 0x58DD80C: clone (clone.S:112)
> ==19065==
> ==19065== Conflicting store by thread 3 at 0x05134178 size 1
> ==19065==    at 0x4EDE987: std::string::_M_mutate(unsigned long,
> unsigned long, unsigned long) (in /usr/lib/libstdc++.so.6.0.13)
> ==19065==    by 0x400A5B: threadEntry(void*) (threads.cpp:12)
> ==19065==    by 0x4C32870: vgDrd_thread_wrapper
> (drd_pthread_intercepts.c:272)
> ==19065==    by 0x55E8A03: start_thread (pthread_create.c:300)
> ==19065==    by 0x58DD80C: clone (clone.S:112)
> ==19065== Allocation context: BSS section of /usr/lib/libstdc++.so.6.0.13
> ==19065== Other segment start (thread 2)
> ==19065==    at 0x58DD7D1: clone (clone.S:84)
> ==19065==    by 0x55E893F: ??? (allocatestack.c:743)
> ==19065==    by 0x676D90F: ???
> ==19065== Other segment end (thread 2)
> ==19065==    at 0x58ACEB7: sched_yield (in /lib/libc-2.10.1.so)
> ==19065==    by 0x400A60: threadEntry(void*) (threads.cpp:13)
> ==19065==    by 0x4C32870: vgDrd_thread_wrapper
> (drd_pthread_intercepts.c:272)
> ==19065==    by 0x55E8A03: start_thread (pthread_create.c:300)
> ==19065==    by 0x58DD80C: clone (clone.S:112)
> ==19065==
> ==19065==
> ==19065== For counts of detected and suppressed errors, rerun with: -v
> ==19065== ERROR SUMMARY: 54 errors from 6 contexts (suppressed: 205 from
> 142)
>
> Are all the race conditions in  I/O code in libstdc++ bening?
>
> If so, are there more things that I should initialize in my app before
> I start creating threads and/or should I be looking into creating
> suppression patterns for drd? Has anyone created them before? Why are
> they not enabled by default in drd?
>
> If not... What should I be reading,doing? Thank you in advance.
>
> Jorge
>