Very interesting, we are all about making sensible default choices for our users.  We might make MOOSE default to sfc_hilbert for meshes built with the internal generator.  At least until we figure out how to make Metis/Parmetis work better.  I can't even come close to getting this problem to run on this many processors when using Metis right now.  I run out of memory at about 1/8th this number of cores...  More investigation will be necessary.

Cody

On Wed, Oct 30, 2013 at 8:56 AM, Kirk, Benjamin (JSC-EG311) <benjamin.kirk@nasa.gov> wrote:
> Thanks Ben!  I wasn't even aware of that Partitioner.  I just tried it on my very large 3D cube domain simulation and it's giving me a 5% boost in performance over linear with no other changes.  I'm running on 120 processors across 60 nodes + threading (using tons of memory).  I guess the communication pattern really makes that much difference.  Also, that's a low estimate, I have an expensive postprocessor that runs at the end of the timestep that's being added into the timestep timer so the actual solve performance boost might be closer to 10%!


Excellent - that's an old space filling curve partitioner from a Carter Edwards class project.  It has Hilbert and Morton ordering, but I believe Hilbert is the default.  For general meshes I'd expect a graph partitioner to be a better default, but for cubes and sensical numbers of processors the Hilbert space filling curve could be faster.

-Ben