Re: [Paramesh-users] Customized facevar prolongation (was Re: remarks...)

[tingxing dong <tingxingdong@gm...>]

Dear Klaus:

I am sorry I did not notice your email on 2009/5/27 until now. Oh,new
version of gmail is rather complicated.
As you mentioned

> This appears to be a very reasonable approach for initializing face var
> values on block boundaries.  But I think in order to do this correctly in
> amr_1blk_fc_prol_user, the IF you mention above would have to be quite
> complex, somethink like:
>   IF( [We are being call from amr_prolong not amr_guardcell]
>     .AND. [Neighbor has nodetype 1]
>     .AND. [Neighbor is NOT a newchild]
>     ...) THEN
>      [Use neighbor's face var values at the surface instead of
>       values from interpolation]



> (Btw, you may not want the second condition.)
> Implementation of the third condition  may not be easily possible, since
> the newchild flag for a neighbor may not be available locally if the
> neighbor is remote.
>

  According to my current understandings about paramesh's
communication mechanism, tree info is also stored in send buffer
and  communicated to  the local. But I am not sure if  it is right  to
use newchild(remote_blk) directly,  since I  did use the third condition
[Neighbor is NOT a newchild] in my codes. If  newchild flag is not available
locally, I'm afraid  my codes was wrong.

  However, anyway,so far, it has not  reported  any error or
warnings.Besides, the result obtained seems to be acceptable ,although it is
far from being  perfect.

So, could you and Kevin or any other developer please describe what happens
 in mpi_amr_comm_setup.F90?

Any contribution would be highly appreciated.  If anywhere in my letter is
wrong,please correct it.

Best wishes.

Re: [Paramesh-users] [Paramesh-developers] Contact PARAMESH developers for help

[Klaus Weide <klaus@fl...>]

On Wed, 13 May 2009, Tingxing Dong wrote [to paramesh-developers]:
============ begin quote ============
     I am a postgraduate student from China and now studying  on paramesh.
 Because of guidances from PARAMESH developers, here I share my problems
with all members of the mailing list.
After gmake successfully, in running  an error  jumps as following:

    Running on       4  processors
    Paramesh error : pe  2  needed full blk      19951      2
     but could not find it or only  found part of it in the message buffer.
      Contact PARAMESH developers for help.
     p2_18462:  p4_error: : 0
(the maxblocks I set in amr_runtime_parameter is 5000)

  this warning occurs in mpi_amr_get_remote_block.F90 which I explicitly
called in my own constructed subroutine of amr_1blk_fc_prol_user to ensure
that face values of  coarse blocks copy from neighboring old refined blocks
 rather than interpolating.

Any help would be highly appreciated.
Best wishes!
                                        sincerely

                                   tingxing dong
============ end quote ============

Later, Tingxing Dong wrote [to me]:
============ begin quote ============
Dear Klaus:

Unfortunately, the Paramesh-Developers mailing list has been idle away for a
long time. Even somebody posted advertisements on the list.

Therefore, I have to fix it myself. Luckily after a few weeks of
examination, I made it. And my solution is to  comment out those lines which
prompted errors in  mpi_amr_get_remote_block.F90.

The reason is that in my codes, coarse meshes  only copy certain range of
Index from their adjacent fine meshes, not the full block request.Thus, the
limitation of vtype=14 or 14+27 is invalid for cases of requesting certain
parts of remote blocks .

 And I think that still leaving  those prompting lines in paramesh is not
sensible, so here I make a bold suggestion that  you  deleted them or
replaced with new comments which cover exceptions to avoid incurring other
users' bewilderment.
 Best wishes.
============ end quote ============

My remarks on this:

1) I am not sure why you need to call mpi_amr_get_remote_block in
order to "ensure that face values of coarse blocks copy from
neighboring old refined blocks rather than interpolating."  I am not
not even sure why you need your own amr_1blk_fc_prol_user.  In
general, the guard cells of a coarse (leaf) block, in a direction
where the block's neighbor is more refined, are not filled by
interpolation at all, but by direct copying from the neighbor at the
same level (i.e., a parent block) in that direction.  To ensure that
there is valid data in the (interior) cells of parent block, the guard
cell fill operation should be preceded by a call to amr_restrict (if
you are not using advance_all_levels). 

If you are considering prolongation that may happened in the course of
guard cell filling:
If you use amr_guardcell (implying that you don't have
no_permanent_guardcells set), the call to amr_restrict is already
included in mpi_amr_guardcell.F90.

If you are considering prolongation via amr_prolong to fill newly
created children after amr_refine_derefine:
A newly created child N can only have a more refined neighbor if the
more refined leaf blocks in that direction are also newly created AND
their parent, the same-level neighbor of new child N, existed as a
leaf block before the amr_refine_derefine and therefore presumably has
valid data in its interior cells.

The above should be true for face variables as well as cell-centered.
(The only special consideration regarding face variables that I am
aware of is related to the force_consistency flag: If you have this
turned on, then face variables in some cells may change slightly. I
don't know if this is relevant here at all.)

2) In normal usage (as when called by paramesh itself),
mpi_amr_get_remote_block is only invoked where all interior cells are
needed.  The '(dtype.ne.14.And.dtype.ne.14+27)' test is therefore
appropriate in normal usage.

3) The way to access cells of (possibly) remote blocks in situations
where only a subset of the block's cells is needed is by calling
amr_1blk_fc_cp_remote (and/or amr_1blk_cc_cp_remote, etc., depending
on the type of data).  The region of cells needed (and specified as
actual arguments) should match what has actually been but in the
temprecv_buf by the previous communications step (invocation of
mpi_amr_comm_setup).

Normally this should already have been called, via
amr_1blk_guardcell_srl, just before an amr_1blk_fc_prol_<something>
routine is invoked.

4) Your fix, removing the 'vtype=14 or 14+27' check, may happen to
work, as long as the cell regions of the remote block that you need
happen to be included in those actually received in the communication
step. But you now have no check to detect if there is a mismatch
between what was sent and what was expected (unless you add
additional tests).


Hoping this is useful,

Klaus

Re: [Paramesh-users] [Paramesh-developers] Contact PARAMESH developers for help

[Jean-Noel Pederzani <jp4wy@vi...>]

Dear Klaus,

what version of paramesh are you using? I had a similar issue using 4.0 but
the problem was solved by switching to 4.1

Jean-Noel Pederzani

On Fri, May 22, 2009 at 12:40 PM, Klaus Weide <klaus@...>wrote:

>
>
> On Wed, 13 May 2009, Tingxing Dong wrote [to paramesh-developers]:
> ============ begin quote ============
>     I am a postgraduate student from China and now studying  on paramesh.
>  Because of guidances from PARAMESH developers, here I share my problems
> with all members of the mailing list.
> After gmake successfully, in running  an error  jumps as following:
>
>    Running on       4  processors
>    Paramesh error : pe  2  needed full blk      19951      2
>     but could not find it or only  found part of it in the message buffer.
>      Contact PARAMESH developers for help.
>     p2_18462:  p4_error: : 0
> (the maxblocks I set in amr_runtime_parameter is 5000)
>
>  this warning occurs in mpi_amr_get_remote_block.F90 which I explicitly
> called in my own constructed subroutine of amr_1blk_fc_prol_user to ensure
> that face values of  coarse blocks copy from neighboring old refined blocks
>  rather than interpolating.
>
> Any help would be highly appreciated.
> Best wishes!
>                                        sincerely
>
>                                   tingxing dong
> ============ end quote ============
>
> Later, Tingxing Dong wrote [to me]:
> ============ begin quote ============
> Dear Klaus:
>
> Unfortunately, the Paramesh-Developers mailing list has been idle away for
> a
> long time. Even somebody posted advertisements on the list.
>
> Therefore, I have to fix it myself. Luckily after a few weeks of
> examination, I made it. And my solution is to  comment out those lines
> which
> prompted errors in  mpi_amr_get_remote_block.F90.
>
> The reason is that in my codes, coarse meshes  only copy certain range of
> Index from their adjacent fine meshes, not the full block request.Thus, the
> limitation of vtype=14 or 14+27 is invalid for cases of requesting certain
> parts of remote blocks .
>
>  And I think that still leaving  those prompting lines in paramesh is not
> sensible, so here I make a bold suggestion that  you  deleted them or
> replaced with new comments which cover exceptions to avoid incurring other
> users' bewilderment.
>  Best wishes.
> ============ end quote ============
>
> My remarks on this:
>
> 1) I am not sure why you need to call mpi_amr_get_remote_block in
> order to "ensure that face values of coarse blocks copy from
> neighboring old refined blocks rather than interpolating."  I am not
> not even sure why you need your own amr_1blk_fc_prol_user.  In
> general, the guard cells of a coarse (leaf) block, in a direction
> where the block's neighbor is more refined, are not filled by
> interpolation at all, but by direct copying from the neighbor at the
> same level (i.e., a parent block) in that direction.  To ensure that
> there is valid data in the (interior) cells of parent block, the guard
> cell fill operation should be preceded by a call to amr_restrict (if
> you are not using advance_all_levels).
>
> If you are considering prolongation that may happened in the course of
> guard cell filling:
> If you use amr_guardcell (implying that you don't have
> no_permanent_guardcells set), the call to amr_restrict is already
> included in mpi_amr_guardcell.F90.
>
> If you are considering prolongation via amr_prolong to fill newly
> created children after amr_refine_derefine:
> A newly created child N can only have a more refined neighbor if the
> more refined leaf blocks in that direction are also newly created AND
> their parent, the same-level neighbor of new child N, existed as a
> leaf block before the amr_refine_derefine and therefore presumably has
> valid data in its interior cells.
>
> The above should be true for face variables as well as cell-centered.
> (The only special consideration regarding face variables that I am
> aware of is related to the force_consistency flag: If you have this
> turned on, then face variables in some cells may change slightly. I
> don't know if this is relevant here at all.)
>
> 2) In normal usage (as when called by paramesh itself),
> mpi_amr_get_remote_block is only invoked where all interior cells are
> needed.  The '(dtype.ne.14.And.dtype.ne.14+27)' test is therefore
> appropriate in normal usage.
>
> 3) The way to access cells of (possibly) remote blocks in situations
> where only a subset of the block's cells is needed is by calling
> amr_1blk_fc_cp_remote (and/or amr_1blk_cc_cp_remote, etc., depending
> on the type of data).  The region of cells needed (and specified as
> actual arguments) should match what has actually been but in the
> temprecv_buf by the previous communications step (invocation of
> mpi_amr_comm_setup).
>
> Normally this should already have been called, via
> amr_1blk_guardcell_srl, just before an amr_1blk_fc_prol_<something>
> routine is invoked.
>
> 4) Your fix, removing the 'vtype=14 or 14+27' check, may happen to
> work, as long as the cell regions of the remote block that you need
> happen to be included in those actually received in the communication
> step. But you now have no check to detect if there is a mismatch
> between what was sent and what was expected (unless you add
> additional tests).
>
>
> Hoping this is useful,
>
> Klaus
>
>
> ------------------------------------------------------------------------------
> Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT
> is a gathering of tech-side developers & brand creativity professionals.
> Meet
> the minds behind Google Creative Lab, Visual Complexity, Processing, &
> iPhoneDevCamp asthey present alongside digital heavyweights like Barbarian
> Group, R/GA, & Big Spaceship. http://www.creativitycat.com
> _______________________________________________
> Paramesh-users mailing list
> Paramesh-users@...
> https://lists.sourceforge.net/lists/listinfo/paramesh-users
>

Re: [Paramesh-users] [Paramesh-developers] Contact PARAMESH developers for help

[Klaus Weide <klaus@fl...>]

On Fri, 22 May 2009, Jean-Noel Pederzani wrote:

> Dear Klaus,
> 
> what version of paramesh are you using? I had a similar issue using 4.0 but
> the problem was solved by switching to 4.1

My remarks were based on Paramesh SourceForge CVS code from some months 
ago, so basically, 4.1.  However, I think that all I wrote should also 
apply to 4.0.

Klaus

[Paramesh-users] Customized facevar prolongation (was Re: remarks...)

[Klaus Weide <klaus@fl...>]

On Sun, 24 May 2009, tingxing dong wrote:

> But my purpose of calling get_remote_block.F90 (in fc_prol_user.F90) is not
> to fill guardcell.Instead,I use it for newly created children after
> amr_refine_derefine.(Considering fc_prol_user.F90 is used both in
> amr_guardcell and amr_prolong,a divergence is needed.but this's not
> difficult. Adding a IF would be OK)

Dear Tingxing Dong,

Thank you for clarifying the context.

> My presumption is that for a new created child N, if its parent has a more
> refined neighbor(which is already exist before amr_refine_define but now is
> the same refined level with the child )in one direction, then the child's
> face values in their shared faces should copy from its neighbor instead of
> using interpolated value from its parents. Since old values usually are more
> precise,while interpolation would introduce errors more or less.

This appears to be a very reasonable approach for initializing face var
values on block boundaries.  But I think in order to do this correctly in
amr_1blk_fc_prol_user, the IF you mention above would have to be quite
complex, somethink like:
   IF( [We are being call from amr_prolong not amr_guardcell]
     .AND. [Neighbor has nodetype 1]
     .AND. [Neighbor is NOT a newchild]
     ...) THEN
      [Use neighbor's face var values at the surface instead of
       values from interpolation] 

(Btw, you may not want the second condition.)
Implementation of the third condition  may not be easily possible, since 
the newchild flag for a neighbor may not be available locally if the 
neighbor is remote.

> Actually, until now I have not find a mechanism in Paramesh to ensure
> that,so I call get_remote_block this API fuction myself to acchive it.

Perhaps paramesh should provide a wys for doing this in a simpler way.

> 2) Therefore in my codes, mpi_amr_get_remote_block is used to gain remote
> blocks' face values in the shared faces but not full request of interior
> cells. However, Paramesh does use the subrountine for full request in
> mpi_1blk_guardcell.F90, in which case,the '(dtype.ne.14.And.dtype.ne.14+
>  27)' check sentence is necessary.
> 
> 3) amr_1blk_fc_cp_remote does access subset of cells but it is devised for
> guardcell filling between same refine level blocks. Besides my objective is
> not  for guard cells
> 
> 4) 'vtype=14 or 14+27' check is necessary in most cases,since it is only
> used in amr_1blk_guardcell.F90 in Paramesh. But it does not apply to special
> cases,for instance,like mine.Thus I mean although right now this check is
> probably ok, but a substitute including all cases would make this check more
> perfect.

You can of course have a variant of mpi_amr_get_remote_block (under a 
different name) that does not have the dtype check. Then call the variant,
rather than the original, from your fc_prolong_user.  But your variant routine
should
(1) not drop the dtype.ne.14... check entirely, but replace it with a less
strict check, which would depend on the face.  Basically, check that the dtype
is such that all the needed face values from the neighbor are present.
(2)make sure that when copying face var values from an incomplete (i.e.,
dtype.ne.14) received block in the buffer, you are copying the values
from the correct locations.

Actually it seems that the existing file mpi_amr_get_remote_block_fvar.F90
implements some of this logic.

Hope this was helpful,

Klaus

Re: [Paramesh-users] [Paramesh-developers] Customized facevar prolongation (was Re: remarks...)

[Kevin Olson <Kevin.M.Olson@dr...>]

All,

Using the 'force_consistency' flag will ensure that face variables are  
IDENTICAL at block boundaries between blocks of the same refinement  
level.  The user need not write any new routines.

Best.
Kevin Olson

On May 26, 2009, at 10:51 PM, Klaus Weide wrote:

> On Sun, 24 May 2009, tingxing dong wrote:
>
>> But my purpose of calling get_remote_block.F90 (in fc_prol_user.F90)  
>> is not
>> to fill guardcell.Instead,I use it for newly created children after
>> amr_refine_derefine.(Considering fc_prol_user.F90 is used both in
>> amr_guardcell and amr_prolong,a divergence is needed.but this's not
>> difficult. Adding a IF would be OK)
>
> Dear Tingxing Dong,
>
> Thank you for clarifying the context.
>
>> My presumption is that for a new created child N, if its parent has a  
>> more
>> refined neighbor(which is already exist before amr_refine_define but  
>> now is
>> the same refined level with the child )in one direction, then the  
>> child's
>> face values in their shared faces should copy from its neighbor  
>> instead of
>> using interpolated value from its parents. Since old values usually  
>> are more
>> precise,while interpolation would introduce errors more or less.
>
> This appears to be a very reasonable approach for initializing face var
> values on block boundaries.  But I think in order to do this correctly  
> in
> amr_1blk_fc_prol_user, the IF you mention above would have to be quite
> complex, somethink like:
>    IF( [We are being call from amr_prolong not amr_guardcell]
>      .AND. [Neighbor has nodetype 1]
>      .AND. [Neighbor is NOT a newchild]
>      ...) THEN
>       [Use neighbor's face var values at the surface instead of
>        values from interpolation]
>
> (Btw, you may not want the second condition.)
> Implementation of the third condition  may not be easily possible,  
> since
> the newchild flag for a neighbor may not be available locally if the
> neighbor is remote.
>
>> Actually, until now I have not find a mechanism in Paramesh to ensure
>> that,so I call get_remote_block this API fuction myself to acchive it.
>
> Perhaps paramesh should provide a wys for doing this in a simpler way.
>
>> 2) Therefore in my codes, mpi_amr_get_remote_block is used to gain  
>> remote
>> blocks' face values in the shared faces but not full request of  
>> interior
>> cells. However, Paramesh does use the subrountine for full request in
>> mpi_1blk_guardcell.F90, in which case,the  
>> '(dtype.ne.14.And.dtype.ne.14+
>>  27)' check sentence is necessary.
>>
>> 3) amr_1blk_fc_cp_remote does access subset of cells but it is  
>> devised for
>> guardcell filling between same refine level blocks. Besides my  
>> objective is
>> not  for guard cells
>>
>> 4) 'vtype=14 or 14+27' check is necessary in most cases,since it is  
>> only
>> used in amr_1blk_guardcell.F90 in Paramesh. But it does not apply to  
>> special
>> cases,for instance,like mine.Thus I mean although right now this  
>> check is
>> probably ok, but a substitute including all cases would make this  
>> check more
>> perfect.
>
> You can of course have a variant of mpi_amr_get_remote_block (under a
> different name) that does not have the dtype check. Then call the  
> variant,
> rather than the original, from your fc_prolong_user.  But your variant  
> routine
> should
> (1) not drop the dtype.ne.14... check entirely, but replace it with a  
> less
> strict check, which would depend on the face.  Basically, check that  
> the dtype
> is such that all the needed face values from the neighbor are present.
> (2)make sure that when copying face var values from an incomplete  
> (i.e.,
> dtype.ne.14) received block in the buffer, you are copying the values
> from the correct locations.
>
> Actually it seems that the existing file  
> mpi_amr_get_remote_block_fvar.F90
> implements some of this logic.
>
> Hope this was helpful,
>
> Klaus
>
> ----------------------------------------------------------------------- 
> -------
> Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT
> is a gathering of tech-side developers & brand creativity  
> professionals. Meet
> the minds behind Google Creative Lab, Visual Complexity, Processing, &
> iPhoneDevCamp as they present alongside digital heavyweights like  
> Barbarian
> Group, R/GA, & Big Spaceship. http://p.sf.net/sfu/creativitycat-com
> _______________________________________________
> Paramesh-developers mailing list
> Paramesh-developers@...
> https://lists.sourceforge.net/lists/listinfo/paramesh-developers
>

Re: [Paramesh-users] [Paramesh-developers] Customized facevar prolongation (was Re: remarks...)

[Klaus Weide <klaus@fl...>]

On Wed, 27 May 2009, Kevin Olson wrote:

> All,
> 
> Using the 'force_consistency' flag will ensure that face variables are
> IDENTICAL at block boundaries between blocks of the same refinement level.
> The user need not write any new routines.

Kevin,

In my understanding of Paramesh, the following is true; please correct if 
wrong.

When 'force_consistency' is set, face variables will have identical values  
at block boundaries between leaf blocks of the same level, but only after
guard cells have been filled at least once.  Values are not equalized in
this way in the permanent facevarx(y,z) arrays for newchild blocks 
immediately after amr_prolong returns.

Also, while 'force_consistency' may ensure that face variables have 
identical values at block boundaries, it does not ensure that that value 
is the 'best'.  To do that, for the definition of 'best' introduced by 
Tingxing Dong, it seems the user does need to write new code.


Klaus

Re: [Paramesh-users] Customized facevar prolongation (was Re: remarks...)

[tingxing dong <tingxingdong@gm...>]

Dear Klaus:

I am sorry I did not notice your email on 2009/5/27 until now. Oh,new
version of gmail is rather complicated.
As you mentioned

> This appears to be a very reasonable approach for initializing face var
> values on block boundaries.  But I think in order to do this correctly in
> amr_1blk_fc_prol_user, the IF you mention above would have to be quite
> complex, somethink like:
>   IF( [We are being call from amr_prolong not amr_guardcell]
>     .AND. [Neighbor has nodetype 1]
>     .AND. [Neighbor is NOT a newchild]
>     ...) THEN
>      [Use neighbor's face var values at the surface instead of
>       values from interpolation]



> (Btw, you may not want the second condition.)
> Implementation of the third condition  may not be easily possible, since
> the newchild flag for a neighbor may not be available locally if the
> neighbor is remote.
>

  According to my current understandings about paramesh's
communication mechanism, tree info is also stored in send buffer
and  communicated to  the local. But I am not sure if  it is right  to
use newchild(remote_blk) directly,  since I  did use the third condition
[Neighbor is NOT a newchild] in my codes. If  newchild flag is not available
locally, I'm afraid  my codes was wrong.

  However, anyway,so far, it has not  reported  any error or
warnings.Besides, the result obtained seems to be acceptable ,although it is
far from being  perfect.

So, could you and Kevin or any other developer please describe what happens
 in mpi_amr_comm_setup.F90?

Any contribution would be highly appreciated.  If anywhere in my letter is
wrong,please correct it.

Best wishes.

Re: [Paramesh-users] Customized facevar prolongation (was Re: remarks...)

[Klaus Weide <klaus@fl...>]

[I had written:]
> >   IF( [We are being call from amr_prolong not amr_guardcell]
> >     .AND. [Neighbor has nodetype 1]
> >     .AND. [Neighbor is NOT a newchild]
> >     ...) THEN
> >      [Use neighbor's face var values at the surface instead of
> >       values from interpolation]
> 
> > Implementation of the third condition  may not be easily possible, since
> > the newchild flag for a neighbor may not be available locally if the
> > neighbor is remote.
> >
> 
On Mon, 1 Jun 2009, tingxing dong wrote:
>   According to my current understandings about paramesh's
> communication mechanism, tree info is also stored in send buffer
> and  communicated to  the local. But I am not sure if  it is right  to
> use newchild(remote_blk) directly,  since I  did use the third condition
> [Neighbor is NOT a newchild] in my codes. If  newchild flag is not available
> locally, I'm afraid  my codes was wrong.

Tingxing Dong,

It appears that the newchild flags of neighboring blocks do get cached locally,
so this should not be a problem after all.  I had not checked for this when
I wrote the above, and made a false assumption that newchild of neighbors was
not always available.

>   However, anyway,so far, it has not  reported  any error or
> warnings.Besides, the result obtained seems to be acceptable ,although it is
> far from being  perfect.

If there was a mistake in the logic, it would perhaps only lead to
occasionally copying of data from a neighbor when they shouldn't have
been copied, or occasionally NOT copying the slightly more accurate
version from a neighbor. But the neighbor's data should be nearly the
same anyway. So such mistakss could very easily be missed.

But anyway, I am not claiming that there is anything wrong in your logic.

[By the way if this exchange continues, I will limit my messages to
Paramesh-Users, there is no need to send to both -Users and -Developers.]

Klaus