On Wed, 14 Jan 2009, Roy Stogner wrote:
> On Wed, 14 Jan 2009, Tim Kroeger wrote:
>> Now, the first problem I encounter -- but actually it's not really a
>> problem -- is that PETSc supplies the possibility to automatically
>> communicate any changes of vector components to the other processors that
>> may have those indices as ghost indices. While, in a long term view, using
>> that functionality would help to get rid of the duplicated solution
>> representation, I think that I should for now *not* use that functionality
>> since the current construction of current_local_solution does not expect
> Interesting possibility, but even in the long run I don't think we'd
> want to use it unless it's quite efficient. I can't see how they'd
> accomplish that without requiring every set() to perform tests, every
> ghost dof set() to do non-blocking message sends, and every get() to
> test for a message receipt. Synchronizing data only in large batches
> when we know it's necessary isn't quite as intuitive but is probably
I formulated my statement misleading. The communication is not done
automatically after each set() operation, it is (as yours) done in
blocks on request. There are two advantages I see over the current
implementation: First, the necessity to have two vectors would be
removed. Second, communication is reduced by only transfering those
values that have actually changed (It seems to me that PETSc manages a
list of them).
But still, I think we should not use that, at least at the moment.
But there is another problem (things turn out to be more difficult
than I thought): In the ghost cell case, PETSc does not provide the
appropriate global-to-local mapping that would be required for e.g.
NumericVector::operator()(unsigned int). I asked this in the
petsc-users list. Jed Brown commented on this by arguing the natural
thing would be that libMesh's DofMap should work on local dof numbers.
(A local-to-global mapping seems to be provided by PETSc.)
My idea would be that PetscVector creates the global-to-local mapping
(for the ghost cells only) itself (and stores it, e.g. as a
std::map<unsigned int, unsigned int>). This should still save a lot
of memory compared with the serial vector version.
Do you agree?
Dr. Tim Kroeger
tim.kroeger@... Phone +49-421-218-7710
tim.kroeger@... Fax +49-421-218-4236
Fraunhofer MEVIS, Institute for Medical Image Computing
Universitaetsallee 29, 28359 Bremen, Germany