#155 pyFoamPlotWatcher.py logfile.gz --> crashes due to gzip._read_eof()

[Logan Page]

Version 0.5.5

1) [during an active simulation] using "pyFoamPlotWatcher.py" with a zipped logfile created using "pyFoamSteadyRunner.py --compress" crashes due to the gzip._read_eof() function.

2) [after a simulation has finishe] using "pyFoamPlotWatcher.py" with a zipped logfile created using "pyFoamSteadyRunner.py --compress" skips plotting a lot of data towards the end of the logfile.

I'm not sure if it possible to create a workaround for this as I know too little atm about this. I will play around with it a bit more and see what I can come up with.

[Bernhard Gschaider]

Any reason why you use this quite old version of PyFoam?

Nevertheless I'm afraid that the new version won't fix the problem but that it is a problem with the way the gzip-files are written (certain blocks of data have to be present to be decompressable)

But that would only explain the first problem. The second problem is strange. Could you try the following:
- decompress the logfile and see if the Watcher now plots the whole data
- use gzip to compress the file and see if the watcher works correctly with this "handzipped" file

[Logan Page]

[Logan Page]

[Logan Page]

[Logan Page]

I've just been too lazy to update to the newer version :P

Watcher gives the following results:
- plot_01.png with the decompressed logfile - works properly
- plot_01.png with the manual / hand zipped logfile - works properly
- plot_03.png with the origionally zipped logfile (from steadyRunner) - doesn't work properly

I've also attached the origionally zipped logfile

[Logan Page]

[Logan Page]

The PyFoamGzipFile.py file I attached seems to solve the 1st problem.
However some testing is needed.

I also noticed that "PyFoamPlotWatcher.py --solver-not-running-anymore" terminates early, so I wasn't able to test if the code I wrote terminates correctly if EOF is reached and PlotWatcher stops waiting for further output.

[Logan Page]

[Logan Page]

I've attached the code I modified in order to fix all the above mentioned problems.

Each line I added or modified in the code is commented with "#XXX - comment"

Problem 1 below is for cases when the simulation is complete.
Problem 3 below is for active simulations.
Problem 2 below is for either active or completed simulations.

Problem 1: PlotWatcher terminates early with .gz logfile:
This was due to the "currSize > self.reader.bytesRead(fh)" check in the "BasicWatcher.py" file. bytesRead was calculated as "bytes += len(line read)" which, if based on the decompressed information, means that bytesRead quicky becomes larger than currSize (total size of the logfile) and so the BasicWatcher terminates early.

Problem 2: PlotWatcher with --solver-not-running-anymore terminates early with .gz logfile:
See Problem 1 above

Problem 3: PlotWatcher with .gz file crashes at eof
This crashes due to a CRC check on a half read member during an active simulation. Fixed by adding a time.sleep(wat_time) to check if the logfile was modified before moving on to _read_eof() function. Unforetunetly the wat_time needs to be quite high, cause if there is a slight delay or pause with the solver or write data then the same problem occurs. I upped the wait_time to 10s, thus the plots lag for 10s, then catches up, then lags for 10s, etc.

I hope this code will be useful :)

[Bernhard Gschaider]

Will have a look at it (this gzip stuff is not very thoroughly tested)