#280 Ggi interface with Dynamic mesh crashes when ggi interface and processor boundary are parallel

[Eelco van Vliet]

Dear Foam developpers

I have made a modified version of the pimpleDyMFoam movingCylinders tutorial with the following changes: 1) removed one sphere out of block mesh, changed the motion from a oscillation to a linear translation, 3) changed the solver from pimpleDyMFoam to interDyMFoam to allow to move a cylinder though a free surface.

The case works fine in serial mode. However, the case crashes in parallel mode at the moment that the growing cell layer is crossing the processor boundary.

I have found that in case all the processor boundaries are parallel to the motion of the sphere (so the growing cell layer is never parallel to the processor boundary), the case also runs fine in parallel. So changing the line in the decompePardict from n (2 2 1); to n (4 1 1); will fix the case.

Nevertheless, the question is: is the crash for the (2 2 1) decomposition caused by a bug in the ggi interface ? I have added the case . Running Allrun will show the crash around 1.8 secs.

Regards
Eelco

[Hrvoje Jasak]

Hi Eelco,
This is a known limitation. GGI relies on global face zones to do the interpolation and if one of the processors modifies the face zone, it is no longer in sync with other processors. Therefore, GGI interfaces that are subject to topological change in parallel cannot be split among processors.

I do not have a good way to generalise the bug fix without massive loss of parallel performance (the topo change in GGI would have to br broadcast and a global face zone re-built) and I would prefer to live with the limitation. In fact, I am not certain how I would do this in general anyway and the number of constrained cases is quite small.

This is what you need to do: build a decomposition where each of the GGI is on a single processor and avoid using the global face zone.

I have re-run the case with a different decomposition and it runs fine. I suspect you will have a more complex 3-D case, but the principle is the same; we see the same thing when doing IC engines with topo changes in parallel.

Hrv

[Eelco van Vliet]

Hi Hrvoje,

Thanks for you answer. What I only still don't understand that in case I make a patchContrained decomposition using the following set up

patchConstrainedCoeffs
{
method metis;
numberOfSubdomains 4;
patchConstraints
(
(leftblock_side1 1)
(leftblock_side2 1)
(rightblock_side2 1)
(rightblock_side1 1)
);
}

the case still crashes when the growing cell cross the cell boundary, whereas I have enforced all the ggi interface to be on processor 1. (I checked it, and indeed the left and right ggi are on proc 1 only).

So although the GGI topology does not change on processor one, also the crossing of the layerFaces of the dynamic mesh causes a crash (which are not GGI interface). Does that make sense ? Does that imply that also the faceZones belonging to the layerFaces (defined using setBatchMotion) are required to stay on one processor as well? In that case the decomposition with the simple setup yields the only valid processor decomposition.

Well, at least good to know it is a know issue and not because of my setup

Regards
Eelco

[Antonio Martín Alcántara]

Hi, I'm curious to know if you finally solve this issue. I have the same problem...

Regards,
Antonio