On Tue, 9 Sep 2008, Benjamin Kirk wrote:
> WAAAY back on the thread I saw that his mesh size is ~99,000 elements with
> ~23,000 nodes at the end of the simulation.
How do you know? Anyway, you are right, it's actually 94160 elements
and 19754 nodes.
> So at 40 processors that is
> only 575 degrees of freedom per processor, assuming linear Lagrange. That
> is way too small of a problem size.
Well, at least I would expect it to be large enough to scale well, but
maybe I'm wrong there.
Actually, the task I want to do is perform a convergence study. To
demonstrate the bottle neck problem to you, I chose the coarsest
resolution that I want to compute, so I wouldn't have to wait too long
for each test example.
I have attached the output (10 and 20 CPUs with one CPU per node each)
when I run the program on three different grids (that is the one from
last time plus two finer grids). The finest grid has 687920 elements
and 156370 nodes at the end of the simulation. Still, the scalability
The next grid in the series cannot be computed any more since it runs
out of memory on initializing the EquationSystems; it would have
2331864 elements and 529497 nodes.
I should note that my application currently is computing only one time
step where there will be a lot of steps in the future.
Should the solution be that the interconnect of the cluster I'm using
is just to slow? In that case I would have to try to get access to a
different cluster (and get my application running on that). This will
be a lot of work (both administrative and coding), but if I can be
sure that it will help, I think I should do it, rather than waiting
for non-existent bugs to be fixed. On the other hand, if it should
turn out that it won't help, it would be too bad about the work.
Let's try it the other way round: Is there any easy test example that
I could run on "my" cluster that you would expect to scale well with
the number of CPUs I'm typically using? (It should be a test example
that uses a sufficiently high number of dofs, solves some linear
systems, and refines adaptively so that it requires to call
EquationSystems::reinit() several times.)
Dr. Tim Kroeger Phone +49-421-218-7710
tim.kroeger@..., tim.kroeger@... Fax +49-421-218-4236
MeVis Research GmbH, Universitaetsallee 29, 28359 Bremen, Germany
Amtsgericht Bremen HRB 16222
Geschaeftsfuehrer: Prof. Dr. H.-O. Peitgen