Menu
▾
▴
#123 Bug in groovyBC in parallel computation
acknowledged
normal
major
always
none
other
2014-04-13
2012-02-11
No
I found a bug which seems to be connected with groovyBC. If in groovyBC expressions the "max" and "min" statements are used, the parallel run of simpleFoam and pisoFoam fails.
This bug is specific for OF-1.6-ext. I tried the same case in OF-2.1.0 (with some changes specific for 2.1.0 version) and anything worked OK.
The swak4foam was downloaded by:
svn checkout https://openfoam-extend.svn.sourceforge.net/svnroot/openfoam-extend/trunk/Breeder_1.7/libraries/swak4Foam/
The OF-1.6-ext was downloaded by:
git clone git://openfoam-extend.git.sourceforge.net/gitroot/openfoam-extend/OpenFOAM-1.6-ext
There were neither swak4foam nor groovyBC in downloaded OF-1.6-ext
I attach an example case. As a prototype for this case the "pitzDaily" tutorial for simpleFoam was used. I changed the inlet BC from fiexdValue to Poiselle's profile (implemented via groovyBC). This case runs simpleFoam OK on a single core but fails in parallel simpleFoam run.
Bug was also discussed in this thread http://www.cfd-online.com/Forums/openfoam-bugs/97200-bug-groovybc-parallel-computation.html (just so that I don't have to search it)
I think the problem is that this fix http://www.cfd-online.com/Forums/openfoam-bugs/66024-inconsistent-behaviour-gmax-gmin-parallel-runs.html never made it to 1.6-ext (but I'll have to check this)
But I'll have a look at the test-case and hopefully I can then give a better opinion on the problem
Looking at the output posted on the message-board I think it might be something else.
This will take a little while as I currently have no Debug-version of 1.6-ext
(this is more a note to myself than a progress report)
It seems to be a different Problem from the one I mentioned above: the maximum/minimum-calculation works for the evaluations before the equations are solved (see the output below where I instrumented the code with Pout/Info). After the equation-solution the reduce-operation for the minimum fails.
Getting the suspicion that this is not so much a problem with swak but with something that 1.6-erxt does on the MPI-level (during equation solving)
The relevant output:
Time = 1
[1] max -1e+40
[0] max 0.0254
max 0.0254
[0] min 0
[1] min 1e+40
min 0
[0] max 0.0254
[1] max -1e+40
max 0.0254
[0] min 0
[1] min 1e+40
min 0
DILUPBiCG: Solving for Ux, Initial residual = 1, Final residual = 0.0505907, No Iterations 1
DILUPBiCG: Solving for Uy, Initial residual = 0, Final residual = 0, No Iterations 0
DICPCG: Solving for p, Initial residual = 1, Final residual = 0.00991172, No Iterations 173
time step continuity errors : sum local = 0.231488, global = -0.00142618, cumulative = -0.00142618
[0] max 0.0254
[1] max -1e+40
[bgs-Greybook.local:77069] *** An error occurred in MPI_Recv
[bgs-Greybook.local:77069] *** on communicator MPI_COMM_WORLD
[bgs-Greybook.local:77069] *** MPI_ERR_TRUNCATE: message truncated
[bgs-Greybook.local:77069] *** MPI_ERRORS_ARE_FATAL (goodbye)
Pleas have a look.
http://www.cfd-online.com/Forums/openfoam-bugs/97200-bug-groovybc-parallel-computation.html#post381526
I am facing the above said error, but I am not using groovyBC.
OK. Wrote my comment to the thread before I saw your addition to the ticket. Anyway can you confirm whether the pressure solver is an AMG-type or a CG-type and whether that makes a difference?
I can produce the same error without any (intentional) calls to gMin or gMax.
This expression fails:
valueExpression "vector(Ux,0,0)";
variables "Ux=1";
This expression works:
valueExpression "vector(1,0,0)";
variables "";
Thanks in advance for having a look. Error still occurs when I use preservePatches to keep the patch with groovyBC on one processor. Please tell me if I can provide additional info.
That is a strange one.
What does it say exactly? Which OF-version? Which swak-version?
Am a bit surprised that the variable assignment works by itself. In the good old days one had to write
variables "Ux=1;";
(semicolon inside the string) but maybe I relaxed that requirement some time ago
A similar problem was reported at http://www.cfd-online.com/Forums/openfoam-solving/132416-swak4foam-0-3-0-make-parallel-run-crash-foam-extend-3-0-a-post485428.html (works on OF2.3 but not on FE-3). Can't be the reduct-operation as that example doesn't use min, max or similar.
Nevertheless: my suspicion is that FE-3/OF-1.6-ext "optimizes" away certain calls to the BC if the processor doesn't have faces on that BC thus making an MPI-operation fail
OK. I was informed that groovyBC does things in evaluate that it's not supposed to do.
Anyway: the problem is *not* specific to 1.6-ext/foam-extend-3.0, it is specific to the commsType which is the default for those versions: have a look at etc/controlDict in your Distro. In the section OptimizationSwitches you'll find an entry
commsType blocking;
Change that to
commsType nonBlocking;
and groovyBC should work (don't know about the performance implications of this change). The only differernce to OF-2.3 and similar is that these are delivered with nonBlocking as the default (if you change to blocking it will fail too).
I'll have look on how to make it work for blocking