We have a need for adding more "ghosted entries" to existing ghosted vectors.  For a while now I've been handling this with this pattern:

1.  eq_sys.reinit()
2.  Find new ghosted entries for use in DofMap::augment_send_list()
3.  eq_sys.reinit() // This will call augment_send_list to get the new entries.

This is clunky, slow and redundant... and it's now also failing for me in one instance (having to do with mesh adaptivity).  What I would really like to do is something like this:

1.  eq_sys.reinit()
2.  Find new ghosted entries for use in DofMap::augment_send_list()
3.  eq_sys.reinit_ghosted_vectors() // This will call augment_send_list to get the new entries.

The problem comes in how to implement #3 (well, if this is a good idea at all).  I can see two options:

Option #1:

void System::reinit_ghosted_vectors()
{
  getSendList()
  foreach ghosted_vector:
    temp_vector.copy(ghosted_vector)
    ghosted_vector->init(new_send_list)
    ghosted_vector.copy(temp_vector)
}

Option #2:

void System::reinit_ghosted_vectors()
{
  getSendList()
  foreach ghosted_vector:
    ghosted_vector->reinit(new_send_list)
}

In Option #2 the "reinit()" function would do whatever was necessary to use the new ghosts.  In the case of PETSc it would probably create a new Vec with the new ghosting and copy the old one into it.

Both options are good in their own way.  Option #1 should work for all types of vectors without having to go code something up for each one.  Option #2 might be more optimal depending on the implementation... but means that someone needs to go implement those functions.

What do you guys think?  Is there a better option all the way around?

Derek