From: Roy S. <roy...@ic...> - 2009-07-22 15:33:06
|
On Wed, 22 Jul 2009, Tim Kroeger wrote: > All clear. The crash was due to an incomplete rebuild. Sorry for that. The > results are now as follows: > > PetVec ghost #cores #nodes hh:mm:ss > ---------------------------------------- > new ghost 18 3 07:02:54 > new ghost 9 3 08:19:55 > new no 9 3 08:35:28 > old no 9 3 08:21:52 > old ghost 9 3 09:36:12 > old ghost 18 3 07:30:14 > > Conclusion: New PetscVector implementation speeds up ghosted vectors > considerably, while leaving non-ghosted vectors approximately unchanged. If > the new PetscVector implementation is enabled, the ghosted vectors on equal > number of cores are approximately equally fast as the non-ghosted vectors. > If memory is the limitation, ghosted vectors will allow more cores per > processor and therewith speed up the computation considerably. > > Are there any more tests you would like me to perform? Presuming you verified correctness too on those runs, there's nothing else I can think of. Unless another developer objects, I'd say we should switch to your new implementation and turn on ghosted vectors by default. I'll post an announcement to libmesh-users when we do, in case anyone else wants to try verifying or performance benchmarking the changes. If someone finds a case where the new implementation clearly hurts us then we can revert it. Would you send me a new patch against the SVN head? I'd hate to dig through my email for the last patch and then accidentally get the wrong version. The next thing we need to do in this direction is get a ghosted vector implementation from EpetraVector; if we had that we could get rid of the solution/current_local_solution dichotomy and make the whole library simpler/smaller/faster. I won't have the time to do that myself any time soon, though. I'll Cc: Derek, as the active developer most interested in Trilinos support... but IIRC INL's libMesh stuff is running on around a thousand dedicated CPUs with lots of RAM, and getting a slight performance boost may not be his highest priority right now. --- Roy |