Re: [Valgrind-users] trouble using Valgrind with MPI?

[Julian S. <js...@ac...>]

On Wednesday 28 April 2010, Joshua R. Tepper wrote:
> Hi all,
> I am trying to use Valgrind to debug an MPI application, but things don't
> seem to work.  I understand that Valgrind explicitly implements wrapper
> functions for MPI calls and uses the PMPI interface.  To the best of my
> understanding, when called in the context of MPI, valgrind should somehow
> check MPI calls,
yes
> avoid giving "garbage" output from the underlying mpi 
> libraries,
no (but see below)
> and suppress the printing of a separate output for each node.  
no (how did you infer that?)

Re the garbage, debugging MPI apps is problematic because the NIC
I/O and control buffers are mapped directly into memory, and memcheck
doesn't have a way to detect state changes in them.

One option is to use the --ignore-ranges= flag, if you can figure out
what the relevant NIC addresses are.  Another is to tell your MPI stack
to not map cards into memory, but just to use TCP/IP via normal
syscalls to communicate.  That is (or at least, used to be) possible
with OpenMPI with the mpirun args --mca btl tcp,self, for example.

If you are using OpenMPI you might want to ask the OpenMPI devs
for advice.  They are pretty Memcheck-aware, afaik.

J