Menu
▾
▴
#92 Vers 0.5.6: SteadyRunner vs PlotRunner -- CPU usage
acknowledged
None
normal
minor
always
none
A Utility
general
2011-09-21
2011-08-26
No
Currently I've linked PyFoam to an optimisation algorithm and I like
seeing the residuals being actively plotted using PlotRunner,
but I've experienced the following problem with using PlotRunner:
It requires a lot of CPU, which slows down the OpenFoam solution time
drastically when compared to running just SteadyRunner
And It seems to slow down further with each iteration of the optimisation
algorithm (not confirmed)
I'm not sure what the workings are behind PlotRunner --> GnuPlotRunner, or
if it is even possible to improve the CPU usage of this PyFoam application??
I have seen such a behaviour (overproportional usage of CPU by Python) with small cases with a large number of timesteps (where a lot of time is used to resample the data for the plot. Without doing that Gnuplot would need more CPU-time). But that could be optimized. Could you verify that if you start PyFoamPlotWatcher on the Logfile of one of those runs it uses a similar amount of CPU-time? And if yes upload the Logfile and the customRegexp-File (if you have one)
About getting slower during successive runs: that is a strange on. Am I right in assuming that the optimization loop is a Python-Program and you don't call the utilities but the application-classes (in other word: it is always the same python-interpreter). In that case it looks like a memory leak and I'll definitely will have to look at that)
Yup all code is in python and I call the application-classes.
It doesnt seem to consistently slow down... what i mean by this is
that the 1st iteration or 2 are rather quick... then it slows down and more or less stays at that slow speed.
From watching my resources now while SteadyRunner is running with pyFoamPlotWatcher monitoring the simulation i noticed that the CPU usage (of pyFoamPlotWatcher) goes up as the timestep increase (which is to be expected).
The CPU usage (towards the end of the simulation) is about half that of PlotRunner.
When SteadyRunner finishes the simulation the CPU usage drops to 0...
I've uploaded the logfile in any case (not using a customRegexp-File)
I THINK I have the reason for this: as more data is gathered the pickling-file becomes larger and larger until the time it takes to write it is longer than the intervall in which the file gets automatically written. I have developed a fix, but could you first verify, that using the option
--no-pickled-file
with PlotRunner makes python use significantly less CPU?
I'll give this a try,
Is the "--no-pickled-file" option added to the latest release of PyFoam ??
Don't know if this info helps, but i noticed, when running SteadyRunner
and monitoring with PlotWatcher, that PlotWatcher lags behind SteadyRunner in
terms of the time step when the time step gets large
I.e. SteadyRunner time step will be far ahead of what PlotRunner is showing
Yeah. Sorry. My bad. That I did add during the investigation on this bug. As a workaround try setting the --frequency-option to a high value (60s or so)
By how much does the Watcher lag? And is the lag constant? (It might be possible that the Watcher doesn't get the output immediatly as the OS might not immediately write it to disk (only when enough data is present). This would depend on the OS, the Filesystem etc