Re: [Moose-g3-devel] parallel MOOSE

[Greg H. <gh...@ps...>]

Upinder S. Bhalla writes:
 > Hi, Greg,
 >    A more practical question. Is there a significant penalty in issuing
 > MPI_Send as opposed to MPI_Isend? As I see it MPI_Send just needs to
 > hand off the data to the sending process, but I may be misunderstanding
 > how the system works.
 > 
 > -- Upi

Upi,
No, I don't think there is much (if any) penalty with MPI_Send.  In
fact, I usually suggest to people that they use the conceptually
simpler MPI_Send, rather than the more esoteric flavors of Send that
MPI provides.  Many of the strange varieties of Send (e.g. ready
sends) are only useful in very specialized circumstances, and even
there usually don't offer enough of a performance boost with current
MPI implementations to justify the additional programming and
debugging complexity that they create in the program.  Also, some may
have perfectly legal, but unexpected, consequences.  For instance, if
you use MPI_Isend, many implementations will just leave the sent data
lying around in the sending process until there is an attempt to check
whether the operation has completed with MPI_Test or MPI_Wait.  So if
you're debugging and step through the MPI_Isend, you won't see the
message appear on the receiving node.  A common misconception is that
MPI has another process off on the side that is doing the
communication operations that have been queued up.  On most systems
that is not the case, and the only opportunity that MPI has to actually
send and receive data is within MPI_* calls that your own code makes.
Thus, if you do something that MPI is allowed to defer until a later
time, and then go off and do a lot of computation, the operation may
never actually happen.
--Greg