Re: [Libmesh-devel] Change Dof Ordering

[Derek G. <fri...@gm...>]

This is an email from 3 years ago... no one responded :-)

This is coming up again because we're looking at "array variables" again...
and this would be a large optimization.

Any comments?

Derek


On Wed, Aug 3, 2016 at 9:38 AM Derek Gaston <fri...@gm...> wrote:

> I'm working on some low-level optimization stuff... and one of the things
> I want to do is more vectorization when computing the value of variables
> and when computing residuals, etc.  I'm using the variable groups stuff to
> be able to do large vector operations.
>
> To that end... I think that the current choice for dof-ordering within
> variable groups could be changed to be more amenable to vectorization.
>
> Currently DofObject uses dof numbering based on this ordering for variable
> groups:
>
> id = base + var_in_vg*ncomp + comp
>
> The problem with this is that I would like to do a vector operation that
> is like this:
>
> phi_i * all_dofs_in_var_group_corresponding_to_i
>
> With any FE types that have more than one component the above ordering
> means that the dofs corresponding to that shape function are spread out in
> memory (i.e. they're NOT contiguous) and that would preclude vectorization
> of the above operation.
>
> Instead, if we use a dof ordering like this:
>
> id = base + comp*n_var_in_vg + var_in_vg
>
> All of the dofs that need to multiply the same shape function would be
> contiguous and easily vectorized.
>
> I don't think this change would effect anyone.  We've never guaranteed
> this ordering (and it's fairly new anyway)... I think everyone is probably
> using the API instead of thinking of raw memory access like this (And I
> know I probably should be too... but I've been doing it that way for over
> 10 years and I have a few applications that have hundreds to
> tens-of-thousands of variables now that could really use this optimization).
>
> What say you?
>
> Derek
>
>