John Peterson wrote:
> On Sun, May 31, 2009 at 12:52 PM, Jed Brown <jed@...> wrote:
>> I'm using libmesh to provide a reference implementation of Q_2 elements
>> for Poisson, and Q_2-Q_1 for Stokes. If possible, I would also like to
>> compare with third-order elements, but HIERARCHIC appears to produce
>> insanely ill-conditioned matrices. For example,
>> ./ex4-opt -d 3 -n 4 -f HIERARCHIC -o THIRD -ksp_monitor -ksp_converged_reason -pc_type lu -pc_factor_mat_solver_package superlu
>> does not converge. All of petsc, umfpack, mumps, and spooles are
>> unsuccessful as well. Apparently there is no third-order Lagrange
>> implementation so perhaps I'm out of luck here.
>> For the Stokes solver, I modified ex11 for 3D, but no Schur complement
>> preconditioners are available (I could plug it into the new
>> factorization version of PCFieldSplit, but don't have time at the
>> moment). Since BoomerAMG  and ML fail when naively given indefinite
>> matrices, the libmesh Stokes implementation isn't really getting a fair
>> shake (I probably just won't include it). I know some people on this
>> list solve (Navier-)Stokes, so I'm curious if anyone has an
>> algorithmically scalable implementation they'd like to compare.
> Did you modify the boundary conditions from ex11 to account for a
> non-interpolary basis? Looks like we aren't doing the full
> L2-projection in ex11... the BC's in ex4 are about what you'd want.
The issue with HIERARCHIC elements occurs for ex4, I never bothered
trying for Stokes. Skipping the L^2 projection just means solving a
slightly different problem (and it won't impact existance of a solution
for the lid-driven cavity). In other words, I didn't change this, but I
would be surprised if it made a difference.
> Also, did you pin a value of the pressure (see e.g. ex13)? This
> should not be strictly necessary, but sometimes it helps in practice
> to do it.
It shouldn't make any difference as long as you remove the null space
and you're not trying to use a direct solver. The issue with applying
normal preconditioners to indefinite problems is pretty well-known.
Interestingly, libmesh's ordering is remarkably good for ILU. For
./ex11-opt -ksp_monitor_true_residual -ksp_rtol 1e-12 -pc_type ilu -pc_factor_mat_ordering_type rcm
This performs ILU(0) with an ordering computed by reverse Cuthill-McKee.
It takes care of the penalty and then stagnates, but with the natural
ordering it converges completely. If you play with various orderings,
you'll find that it's really easy to make ILU fail.
General support for "block factorization" preconditioners is coming soon
in PCFieldSplit. For incompressible flow, you would provide an IS for
all the velocity degrees of freedom and we'll be able to offer a full
suite of preconditioners based on block factorization with various
choices for the Schur complement (e.g. variants of BFBt).
>> Heads up on a change in petsc-dev: The arguments to MatGetSubMatrix used
>> to reflect implementation details for MPIAIJ/MPIBAIJ and was
>> fundamentally unscalable (though fine for the sizes we've seen so far).
>> We decided to drop the 'csize' argument and take a parallel index set
>> for 'iscol' (instead of the serial gathered one used through 3.0.0).
>> The current implementations still do the gather so there is still a
>> scalability issue when taking submatrices with number of columns similar
>> to single-node memory, but at least the interface is now scalable (I
>> take submatrices of MatShell so this is good). Speak up if you run into
>> this issue, it can be made scalable but it's not a high priority at the
> Thanks for the info! If something is in petsc-dev, does that mean it
> will go into a patch-level change or will this take effect only when
> 3.0.1 is released?
Just 3.0.1 (I don't know when that will be). Since I build against
petsc-dev, I made this one-line change locally.