Re: [Libmesh-users] FEMSystem performance

[Roy S. <roy...@ic...>]

On Thu, 24 Jun 2010, David Fuentes wrote:

> My initial work with libMesh was based on the ex13 approach to solving
> my nonlinear system of equations. I really like the FEMSystem design and
> am thinking of switching to the FEMSystem approach seen in ex18 with
> petsc's nonlinear solvers. Have you seen any significant performance
> differences between matrix vector assembly in the ex13 and ex18 type
> approaches ? Most computationally intensive portions seen in profiles ?
> Seems that there may be a few more functions calls in FEMSystem approach
> but may be irrelevant ?

It was completely irrelevant for the problems where I cared enough
about performance to repeatedly oprofile everything... once you're up
to a dozen or two DoFs per cell, calculating the element jacobian
dwarfs a few extra virtual function calls along the way.  I'm told by
another ICES person who tried some optimization work that the
situation for ex18 is similar.

I suspect that if you're doing a few low-approximation-order variables
per System (whether that's because your non-smooth solutions don't
have many variables or because you're doing things decoupled) then
FEMSystem might be noticeably slower.

The FEMSystem numeric jacobians are definitely slower than they need
to be.  That was partly intentional design decision (I only use them
for prototyping and verifying analytic jacobians, and in both cases
central differencing is worth the extra evaluations) and partly
laziness (I'm not going to code faster alternatives that I wouldn't
use myself).  You mostly run stuff real-time, though, right?  In that
case you wouldn't even be using efficient FDM jacobians either.


I wish there was a way to make these decisions irrelevant.  I'd like
to be able to tell code that a function is virtual, have it usually
behave that way for flexibility, but then compile it with some
slightly-changed header that specifies a single concrete subclass.
E.g. for most of our users for most of the time it would be reasonable to
tell the code "every NumericVector is a PetscVector".  For some of our
users some of the time it would be worth going to the effort of
building a whole library as "every FEMSystem is a MySpecificSystem".
I don't know how to get C++ to do that, though... and it probably
wouldn't be worth it just to get rid of virtual function lookups;
you'd need to compile with inter-compilation-unit inlining as well.
---
Roy