gnuplot-py-users

[Gnuplot-py-users] Re: gnuplot-py problems

[Michael H. <hag...@jp...>]

Dear Jonathan,

Jonathan Ricketson writes:
> Hey Michael, the interface that you have written for gnuplot is
> really good.  But I am having a problem with using it from a web
> browser.
> 
> The funny thing is that it works fine from the command line but when
> I try to run the script from a browser it has this problem:
> 
> gnuplot> set title "Some Title"
> gnuplot> set term png
> gnuplot> set output "/var/www/sample.png"
> gnuplot> set data style linespoints
> gnuplot> plot '/tmp/@18326.0' notitle
> 
> gnuplot> plot '/tmp/@18326.0' notitle
>               ^
>          can't read data file "/tmp/@18326.0"
>          line 0: (No such file or directory)
> 
> the problem seems to be that the tmp file is deleted too quickly, or
> doesn't exist yet... Because from the command line I can replicate
> the problem by the following code:
> 
>   g.plot(data)
>   raw_input('Please press return to continue...\n')
> 
> If I keep the raw input line, then it works, otherwise if I take it
> out, then it has the same error as above.
> 
> If you have any idea what the problem might be I would really
> appreciate your help.
> 
> Jonathan

Your diagnosis is exactly correct.  Communication with gnuplot is via
temporary files by default, and the temporary files are deleted as
soon as the "Gnuplot.Data" object is garbage collected, which in your
example probably occurs when the script ends or when you plot
something else.  (Until you plot something else, the Gnuplot object
keeps a reference to all of the old plot items to prevent they're
being garbage collected.)  There is no way for Gnuplot.py to know when
gnuplot is done reading the temporary file, so there is no general way
for it to know when it is actually safe to delete the file.

Possible solutions are:

1. Use "inline data" instead of temporary files to communicate with
   gnuplot.  This is already available in gnuplot for many plotting
   modes if you specify the "inline=1" option to the constructor of
   the Data object, or if you set GnuplotOpts.prefer_inline_data=1.

2. Introduce a delay between the time you plot and the time you allow
   the Data object to be deleted.  You could just use time.sleep(), or
   in your example you might watch for the creation of the
   /var/www/sample.png file and at that point assume that gnuplot is
   done with the temporary file.

3. Improve Gnuplot.py for all of us by adding an option to use named
   pipes instead of temporary files.  I think it should be possible
   for Gnuplot.py to know when gnuplot has finished reading the data
   from the named pipe so that there is no more ambiguity.  Note that
   it will take some effort to make such a solution portable across
   operating systems, which is the main reason that I haven't
   undertaken it already.

I have directed this reply to the gnuplot-py-users mailing list.
Please join this list if you are interested in Gnuplot.py issues.  You
can do so at the following web site:

    http://lists.sourceforge.net/lists/listinfo/gnuplot-py-users

Yours,
Michael

--
Michael Haggerty
hag...@jp...

Re: [Gnuplot-py-users] Re: gnuplot-py problems

[Leonardo M. <lm...@ud...>]

Hi Michael, Jonathan:

> > gnuplot> plot '/tmp/@18326.0' notitle
> >               ^
> >          can't read data file "/tmp/@18326.0"
> >          line 0: (No such file or directory)

Yes, that was my first and really only problem with Gnuplot. 
It is well documented in the source/docs, but who reads the 
manual before using software ;-)

Anyway, what I did as a dirty horrible quick workaround
is to inherit the Gnuplot class in my own Gnuplot class:

##### Code begins

class myGnuplot(Gnuplot.Gnuplot):
    """
    My addons to Gnuplot

    It will hopefully become a high level simple library some day ...
    """

    def __del__(self):
        """
        Destructor 
        """
        time.sleep(1)

    # (... some other methods are defined below in the class ...)

####

I always call instances of myGnuplot so that I make sure there is
a 1 second delay before deleting the instance, and thus the temporary
files. This works for me for regular 2D plots (y = y (x)).
For surface plots you need more than 1 second though,
but there is no way to estimate in general :-(

As Michael suggested, we need a robust solution
(and I think we really need it, that's one of the things
I had in mind when I suggested opening the list BTW)

Michael suggests:

> There is no way for Gnuplot.py to know when
> gnuplot is done reading the temporary file, so there is no general way
> for it to know when it is actually safe to delete the file.

It seems so, I tried many dirty tricks to find out when gnuplot is 
done with the tmp files with no luck :-(

> Possible solutions are:
> 
> 1. Use "inline data" instead of temporary files to communicate with
>    gnuplot.  This is already available in gnuplot for many plotting
>    modes if you specify the "inline=1" option to the constructor of
>    the Data object, or if you set GnuplotOpts.prefer_inline_data=1.

I am pretty new to gnuplot, so allow me a stupid question: why isn't 
inline data available for all plotting modes ?. Is it a limitation
of gnuplot or a limitation (i.e. non-implemented feature) of 
Gnuplot.py ? . If the latter is the case, would it make sense
to implement in-line data for all modes and use it as a default ?

> 3. Improve Gnuplot.py for all of us by adding an option to use named
>    pipes instead of temporary files.  I think it should be possible
>    for Gnuplot.py to know when gnuplot has finished reading the data
>    from the named pipe so that there is no more ambiguity.  Note that
>    it will take some effort to make such a solution portable across
>    operating systems, which is the main reason that I haven't
>    undertaken it already.

Could you please elaborate a bit more on named pies Michael ?. I am 
very curious. It does seem to be a non-trivial job as you
said. Python2 has at least 4 different "popen()'s ", and one 
of them may be the one we need. I tried playing  a bit some
time ago but I didn't find the answer and had no time to
keep trying :-(

Cheers
-- Leo

Re: [Gnuplot-py-users] Re: gnuplot-py problems

[Michael H. <hag...@jp...>]

Leonardo Milano writes:
> > > gnuplot> plot '/tmp/@18326.0' notitle
> > >               ^
> > >          can't read data file "/tmp/@18326.0"
> > >          line 0: (No such file or directory)
> 
> Anyway, what I did as a dirty horrible quick workaround
> is to inherit the Gnuplot class in my own Gnuplot class:
> 
> ##### Code begins
> 
> class myGnuplot(Gnuplot.Gnuplot):
>     """
>     My addons to Gnuplot
> 
>     It will hopefully become a high level simple library some day ...
>     """
> 
>     def __del__(self):
>         """
>         Destructor 
>         """
>         time.sleep(1)
> 
>     # (... some other methods are defined below in the class ...)
> 
> ####
> 
> I always call instances of myGnuplot so that I make sure there is
> a 1 second delay before deleting the instance, and thus the temporary
> files. This works for me for regular 2D plots (y = y (x)).
> For surface plots you need more than 1 second though,
> but there is no way to estimate in general :-(

This technique will also fail if you don't create a new Gnuplot
instance for every new plot; e.g.,

    >>> g = myGnuplot()
    >>> g.plot(dataset1)
    >>> g.plot(dataset2)

In this example the temporary file holding dataset1 will be deleted as
soon as the second plot statement is executed.  In principle one
should add the delay to the TempFile PlotItem.

One kludgey way we could handle this would be to rewrite the __del__
method of TempFile in such a way that it starts up a new thread which
first sleeps a while before deleting the file.  But again one wouldn't
know how long to sleep, and also the delays would keep the script
alive longer than necessary.

> > Possible solutions are:
> > 
> > 1. Use "inline data" instead of temporary files to communicate with
> >    gnuplot.
> 
> I am pretty new to gnuplot, so allow me a stupid question: why isn't 
> inline data available for all plotting modes?

It's been a while since I looked at this; I think the inline mode is
available for all types of plots but only for ascii data--that is, it
is incompatible with the "binary=1" option (which BTW is the default
on some platforms for GridData).

> > 3. Improve Gnuplot.py for all of us by adding an option to use named
> >    pipes instead of temporary files.
> 
> Could you please elaborate a bit more on named pies Michael?  [...]
> Python2 has at least 4 different "popen()'s ", and one of them may
> be the one we need.

Named pipes are like files in that they have filenames and exist
somewhere on the filesystem, but they are like pipes in that they read
data from another process.  You don't use popen(), you use os.mkfifo()
to create a named pipe.  I see in the Python documentation that
mkfifo() is only available under Unix, so it would at best be a
solution for that platform.

I have no experience with named pipes so I don't know whether they
would be practicable.

I should have mentioned a fourth possibility:

4. Change Gnuplot.py to implement two-way communication between
   gnuplot and Gnuplot.py.  Then, for example, Gnuplot.py could delete
   temporary files when the next gnuplot prompt appears.  This would
   be the most work of all but it would allow other new features such
   as detecting gnuplot errors, reading gnuplot fit command output
   back to python, etc.

Michael

--
Michael Haggerty
hag...@jp...