These large nearly-diagonal blocks pessimal for BAIJ. If you want to split into a few blocks that are each nearly-dense then a combination of BAIJ and Nest should be good.
Oops accidentally hit send! Look below for the real email :-)
Sent from my iPad
On Jul 27, 2013, at 5:45 AM, "Kirk, Benjamin (JSC-EG311)"
> So the petsc matrices assume a constant blocking factor for the entire matrix, which I think is the bug fundamental driver here. If you solved decoupled flow/temperature in 1 matrix with equal order basis functions you'd have 4 fully coupled variables in 3D, with 1 tagalog. I don't think petsc will optimize for that case, and in fact you *might* be better memory wise splitting into two matrices where the flow vars could be coupled.
I gotcha. There might still be a way to do this in one matrix with
MatNest. Maybe you would nest two block matrices inside the matrix?
> Is there another use case with non-full dof coupling I could be overlooking? I'd be happy to try to address it...
Yes. The normal one for us is block-diagonal (just for
preconditioning). We might have 2000 coupled variables (literally!)
that are all linear lagrange.... But we use a block-diagonal
preconditioning matrix. Is there something we can do about that case?
Regardless of the actual coupling present in our problem we build our
matrices with arbitrary amounts of off-diagonal blocks... it all
depends on how important that block may or may not be for