I see your nutty and raise you batshit crazy:
PETSc allows you to provide your own buffer for storing the local elements when you create a vector... Does Trilinos have something similar? If so, I propose we take the implementation in DistributedVector, push it down into NumericVector, and beef it up.
If both PETSc and Trilinos allow the user to manage the local element storage, that could certainly be exploited for performance. operator(i) would be inline with some error checking and then return _val[iifirst] or something like that... no embedded function call. We would still let PETSc do summation to remote values and that sort of thing.
Can you see if trilinos would allow something like this? If so, I'd like to do it. I've resisted implementing a sparse parallel matrix, not so much because it is hard but more because that seems like a slippery slope to writing Krylov solvers and, ultimately parallel preconditioners... All of which are wheels we need not reinvent. But this? Why not...
Ben
Original Message
From: Derek Gaston [mailto:friedmud@...]
Sent: Wed 11/5/2008 6:27 PM
To: Roy Stogner
Cc: Kirk, Benjamin (JSCEG); libmeshusers@...
Subject: Re: [Libmeshusers] Norm of a NumericVector for specific variable...
So... after hashing this out with some people around here... I'm now
thinking that the vector multiply and call linear algebra package norm
is a bad idea. Firstly, we have some systems where we're solving up
to 9 variables! That means we'd have to do all of that stuff every
time.... not too mention wasting memory with all of the zeroes hanging
around in that vector....
I'm thinking now of a NumericVector::subset_norm() that takes a
std::set of the local dof indices to perform the norm on and a norm
type (or we could make separate norm functions... either way).
The std::set could easily be created in System::calculate_norm() for
each variable that you want the discrete norm for and then call
subset_norm() the appropriate number of times.
And here's where it gets nutty: we just do our own damn norm
calculation. It's not like it's rocket science. We do it right in
NumericVector using the interface functions (like operator() ) so that
every NumericVector gets this capability.
What do you think?
Derek
On Nov 5, 2008, at 5:03 PM, Roy Stogner wrote:
>
>
> On Wed, 5 Nov 2008, Kirk, Benjamin (JSCEG) wrote:
>
>> Could the api allow you to specify the indices from userspace? The
>> reason I ask is because if all your variables are the same type it
>> is pretty easy to figure out the range from the first_local_index,
>> last_local_index, n_vars, and var_num.
>>
>> That's a little hackish to do inside the library...
>
> A *little* hackish? No, requiring the command line option
> "node_major_dofs" to use that coefficient numbering, that was a
> little hackish. *Breaking* that option (as well as TaylorHood
> elements, etc.) would be a *lot* hackish. ;)
>
> But as an optimization which could fall back on my more general code,
> it's a very good idea. Use NumericVector::subrange_norm() only if
> we're using varmajor numbering, and have the DofMap cache the
> first_local_index_var[n] for each variable so we get the range limits
> right even if there's differing FE types in use.
> 
> Roy
