Re: [Valgrind-developers] mtV : improve Valgrind to run multiple threads in parallel

[Julian S. <js...@ac...>]

On Sunday, June 17, 2012, Philippe Waroquiers wrote:
> (will reply in 2 other mails for the xor and for the memcheck aspects).
> 
> On Sun, 2012-06-17 at 13:08 +0200, Julian Seward wrote:
> > >      Maybe we need the inner to inform the outer of the relationship
> > >      between JIT code and the guest code, so as to let the outer
> > >      apply suppression rules on the JITted guest.
> > 
> > Yes, so we have patch for that.  Maybe we should clean it up and
> > apply it.
> 
> Yes, having the outer be able to understand the JITted code
> should allow to write suppression files for an outer tool detecting
> (false or true) errors in the guest process run by the inner
> (for an outer helgrind, suppress races.
>  for an outer memcheck, suppress undefined jumps etc).

Are you OK to action this, or is it something I should chase?  


> > if we naively annotate clone then we will have 2 mutually redundant h-b
> > edges. Maybe it doesn't matter.
> > 
> > There is a flag "--sim-hints=enable-outer" that we could maybe use for
> > this, to select at run-time the required behaviour.
> 
> Effectively, if two h-b redundant is problematic, we could activate this
> only with enable-outer hint (or maybe helgrind could recognise that the
> clone is due to a pthread call being done using e.g. internal client
> request in the pthread_create intercept ?).

I suppose having two h-b edges is not a a problem.  What makes me nervous
about Helgrind are the correctness aspects, though .. would prefer to only
have on h-b edge (from pthread_create or clone, but not both) so if there
is some other kind of implementation error in hg, it will not be hidden
by the redundant edge.  (Call me paranoid.  I don't care :)


> > This could be a good thing -- probably some version of it is necessary.
> > However, imagine a test that tests some low frequency path, and no
> > other test checks the same path.  Then the big-so-on-Helgrind approach
> > will not detect races in the path (IIUC).  Another option might be to
> > make each test individually multithreaded (by running N versions of
> > main() in parallel).  This would give us a much better chance to these
> > low-freq paths.
> 
> Yes, having (many more) specific multi-threaded tests will be needed.
> I hope that a ".so loader/test driver" will be able to automatically
> parallelise many of the current regression tests
> (i.e. the main becomes an entry point, the test driver loads this .so
> (and/or some others) and then starts 1 or more threads executing
> these .so in parallel (same .so can be executed more than once).

Would be good if you can make it work.  Sounds like a lot of difficult
Makefile.am hackery to me.



> > > Quickly looked at it. On the thread side, the main reasoning
> > > is that the threads are not sharing a lot of code, and so
> > > a shared cache is not such a good idea. To me, it looks
> > > somewhat not intuitive that there is very little sharing between
> > > threads.
> > 
> > Yes, I agree.  They are going to share libc stuff, + supporting library
> > code (X, Qt, etc).
> 
> For sure, that is the case for any kind of "general" application using
> threads.
> For applications which are using threads to parallelise algorithms,
> each thread will run exactly the same bunch of code.
> I suspect this case is becoming more and more frequent.

I completely agree.

J