The I/O is only parallelized for writing of XDA/XDR files at present.  It is blocked, and it is performed processor-wise.  So you can write a serialized mesh file from a parallel, distributed mesh.  For all other mesh types the mesh is serialized to all processors, and processor 0 performs the write.

The intent is to extend the parallel I/O functionality to any/all formats which support it.  Thus, parallel ExodusII I/O should be pretty high on our list.

Can you generate a 3D exodusII mesh, preferably hybrid element, in serial and distributed format?  Preferably, 1, 2, 4, 8 processor distributions.

-Ben





-----Original Message-----
From: Derek Gaston [mailto:friedmud@gmail.com]
Sent: Mon 6/23/2008 5:15 PM
To: libmesh-devel@lists.sourceforge.net; Kirk, Benjamin (JSC-EG)
Subject: Using Parallel Mesh

Guys,

What do I need to do to actually use ParallelMesh with parallel I/O?
So far I've done --enable-parmesh and used delete_nonlocal_elements
(after doing a build_cube)... then I've computed a solution and
written a single exodus file.  In writing that Exodus file is it
serializing the mesh first?  Or is it doing that block chunked I/O
that Ben put into the code?  Somehow it's getting all of the elements
into that Exodus file....

Here's what I'm thinking I'd like to do: Let's say I have a coarse
mesh in exodus format (all in one file) to start with.  I want to read
this in (preferably in parallel, even if that means I need to do an
offline filetype conversion first) then uniformly refine it to
saturate all available memory... compute a solution... then write it
out (hopefully a parallel write... but doesn't have to be).

Is that doable now?

Also, how much work do you think it would be to be able to read (and
write) decomposed Exodus files?  The writing part sounds especially
easy (I believe that decomposed Exodus files are just regular exodus
files).

The idea is that I'm going to be running a simulation that uses
approximately 40 million elements on about 1,000 procs.  What I do
doesn't have to be pretty... it just needs to work.  If the mesh I run
on is just a cube... so be it... if it looks like a cylinder that
would be a huge bonus.  I am open to all ideas.

Thanks!
Derek