Roy Stogner wrote:
> Yes. All my problems are solving for physics parameters in coupled
> models, where p has global influence and has a direct effect on every
> processor's residual.
Right, the problems I'm thinking of are distributed parameter inversion
and boundary control.
> An interface that supports parallel parameter vectors which are
> discretized onto FEM spaces would be nice, but it's not something I
> need or have any way to test at first. It's something that could be
> added later without changing the method I'm proposing for global
Yes, but it would be nice for the high-level interface to not
distinguish, especially because it would get messy when you have a few
global parameters and also distributed parameters.
> An interface that supports both globally and locally supported
> parameters transparently would be ideal, but I don't see a good way
> to do that. I'm certainly open to suggestions.
My preference is to encode this in the scatter. Each parameter appears
exactly once in the global vector. In your case, all parameters might
be owned by rank 0, with none of the others owning any dofs. When doing
operations locally, you first ensure that your local vector is updated
via a GlobalToLocal scatter, and then collect modifications afterwards.
This way it is not visible to user code that the global parameters are
all owned by one process, but it looks like a proper vector to the
solver. In your case, the local vector holds all the parameters, hence
the scatter is really a Bcast. Hessians are usually dense so you
wouldn't form them anyway, thus the unbalanced ownership doesn't hurt.
DMComposite is a lightweight object that manages systems where you have
an arbitrary collection of arrays of parameters and Vecs representing
spatially distributed data.