From: Roy S. <roy...@ic...> - 2007-07-13 13:48:30
|
On Thu, 12 Jul 2007, Benjamin Kirk wrote: > First off, let me say I am really, really glad to see this discssion. ;-) Yeah, it's been running around my head a couple times, but it occured to me that even a disjointed post to libmesh-devel would probably be more productive than an internal monologue. >>> After EquationSystems::init() is called, all of the Systems' DofMap >>> objects have had their chance to walk all over the Mesh, and then in >>> most cases every processor should stick to its local and ghost >>> elements until it's time to do adaptive refinement - am I right about >>> that? If so, then codes which aren't using AMR/C (and aren't using >>> MeshFunction, etc) could call Mesh::parallelize() at this time, and >>> all "parallelize()" would have to do would be to delete a lot of Elem >>> and Node objects. > > The only immediate gotcha I see is the Mesh::write() methods. In GMV, for > example, it is assumed that processor 0 holds the entire mesh and thus can > write the nodal positions and element connectivity every time GMVIO::write() > is called. This is a gotcha in the sense that my existing application code will break, I'll grant you that. ;-) But there is a short term workaround (codes which don't do any refinement don't need to write out the mesh more than once, which they can do before parallelize()) for this problem. And basically the goal of starting with just Mesh::parallelize() would be to root out such problems with testing rather than leaving us to speculate about them theoretically. > Note that this could be handled in kinda the reverse process of broadcasting > the mesh. A gather operation could be put into MeshCommunication. This > could be a minimal operation that gathers the local elements and nodes for > each processor one at a time. That way processor 0 can still write the > entire buffer, but it can do it in subdomain size chunks and avoid having to > store the entire list of nodes or connectivity. Sounds reasonable. > In the spirit of object-oriented-ness, though, I don't see this as a strong > argument against a ParallelMesh class. Rather, we probably should move in > this direction. It is just at the moment there will be little difference > between Mesh and ParallelMesh. > > I would think we should create ParallelMesh which is derived from Mesh. > They might as well be identical at the moment, thus making ParallelMesh > easy: > > class ParallelMesh : public Mesh > {}; Alright, I'm convinced; we can start from there. I'll add a "parallel_mesh.h" and "parallel_mesh.C" file to CVS soon, even if it may be a month or two before I can work much on this in earnest. > We can then implement the other changes you suggest very easily with > operator overloading and/or (boo, hiss) templates. For example, > EquationSystems could (should?) be templated on mesh type, and we might wind > up with some code like > > if (EquationSystems::MeshType == ParallelMesh) > ... > > Or else a partial specialization for > EquationSystems<ParallelMesh>::reinit(); > > Thoughts? You and your templates. I swear the masses of switch statements in FE<> are at least as inefficient as virtual function calls, and they're definitely twice as ugly. ;-) Let's leave Mesh and ParallelMesh as simple derived classes for now? Even if they're separate classes most of the code will be shared, and I suspect there won't be any virtual functions called frequently enough that we'd want to try to optimize them away with templates. --- Roy |