[Libmesh-devel] Parallelizing the Mesh

[John P. <pet...@cf...>]

Roy Stogner writes:
 > 
 > Just to throw out some ideas:
 > 
 > After EquationSystems::init() is called, all of the Systems' DofMap
 > objects have had their chance to walk all over the Mesh, and then in
 > most cases every processor should stick to its local and ghost
 > elements until it's time to do adaptive refinement - am I right about
 > that?  If so, then codes which aren't using AMR/C (and aren't using
 > MeshFunction, etc) could call Mesh::parallelize() at this time, and
 > all "parallelize()" would have to do would be to delete a lot of Elem
 > and Node objects.
 > 
 > Even for codes which are using adaptive refinement, would it be a good
 > stopgap solution to temporarily serialize the Mesh during
 > EquationSystems::reinit() calls?  The sequence would go something
 > like:
 > 
 > 1. Delete System matrices (which are invalid anyway since we're
 > changing the mesh), making room for a serial Mesh
 > 2. Serialize the Mesh (which might not be much harder than the code in
 > Mesh::read; there's just lots of tricks like remembering to copy
 > old_dof_objects)
 > 3. Repartition the mesh, call System::project_vector, etc.
 > 4. Parallelize the Mesh
 > 
 > Of course, this won't work as is (the MeshRefinement flagging schemes
 > assume a serial Mesh, for instance), but it might be a good place to
 > start without creating a huge new ParallelMesh class or breaking
 > existing code.
 > 
 > 
 > If anyone has a better name suggestion for Mesh::parallelize()
 > (perhaps Mesh::raze()?) or any thoughts to add, let me know.


I think this definitely takes us in the right direction, though I don't
doubt there will be several gotchas even with this first step.  We should
also think about meshes which can't fit on a single processor...

Such a mesh would be ungainly (how would we store it, multiple
data files?)  but you can certainly build_square() something which
is too large for a single CPU, and it would be cool if that eventually
worked in parallel w/o ever having to store the whole thing on 1 CPU.

-John