gnuplot development

emman@... wrote:

>... +----------+----------+----------+----------+----------+----------+ ...
>   2800        2400       2000       1800       1600       1400       1200
>
>
>
>   set irscale
>
>Do you think this might be worth adding?  If so, can you think of a better
>way to implement it, from the user's point of view?  I'd like to have some
>expert opinions before I start messing around too much.  Just now someone tells
>me there's a simple way to do this already ;)
>

This can be done by passing the x data through a nonlinear function, and 
then manually altering the x-tics so they reflect the numbers accurate, 
for example as you've given above.  (The supplying of the tics is the 
tedious part... and you *must* always make sure that the tics break 
right at a location where the nonlinear scaling changes... otherwise 
there is no meaningful way to interpret the data)  As an example, try 
the following:

plot 'battery.dat' using 1:2

vs.

plot 'battery.dat' using ((($1)<=20)*($1) + (($1)>20)*(($1-20)*0.5 
)):2+20)):2

Of course, this is very tedious.  But those who are accomplished with 
the awk scripts in gnuplot might be able to suggest if this can be made 
easy.

I think "irscale" is too specific to be of use in gnuplot.  However, a 
more generalized description or name of what you are suggesting might be 
"piece-wise-linear" scales.

Dan

In looking at this scale thing I mistyped and forgot some parentheses in 
the using string.  The result was something I did not expect.  For example,

plot 'battery.dat' using 1:2

and then

plot 'battery.dat' using 1:log(2)
plot 'battery.dat' using 1:sin(2)

The second plot, is that or is that not making sense?

Dan

On Friday 29 October 2004 09:45 am, Daniel J Sebald wrote:
> plot 'battery.dat' using 1:log(2)
> plot 'battery.dat' using 1:sin(2)
> 
> The second plot, is that or is that not making sense?

Neither command makes any sense to me.
I thought that a "using" specifier had to be either a bare number
or an expression in parentheses.

But whatever it's doing, it was already doing it in version 3.7


-- 
Ethan A Merritt       merritt@...
Biomolecular Structure Center
Mailstop 357742
University of Washington, Seattle, WA 98195

Daniel J Sebald wrote:

> plot 'battery.dat' using 1:log(2)
> plot 'battery.dat' using 1:sin(2)
> 
> The second plot, is that or is that not making sense?

They're both making a perverse kind of sense.  'Using' specifiers 
classify as extended or not depending on the presence of 'extra' 
enclosing parentheses, and those alone.  In this sense using 1:sin(2) is 
just an overly complicated way of writing 'using 1:0'

This can be useful in the case where you want to loop over column 
numbers, or have any other reason to calculate column numbers on the 
spot rather than hard coding them on the spot.  E.g. consider this script:

	p 'file.dat' u 1:n
	n = n + 1
	if (n<20) reread

Pawel,

Could you please give us some idea of the quantity of data and its 
nature?  For example, 2D plot 500 lines?  3D plot 500 surfaces?  I 
assume you say that X11 is slow because that is the only terminal you 
are using.  I say that because there are several parts of gnuplot, 
bringing data into gnuplot, generating the plot internally, sending the 
plot across a pipe to the X11 driver, interpretting the plot commands in 
the driver.

Technically HBB is correct that gnuplot really isn't for real-time 
applications.  (Such applications--ones where the frame refresh must 
occur at required moments in time--need more low-level approaches where 
only necessary portions of the visual are updated.  For example, if you 
are trying to generate some kind of oscilloscope, gnuplot won't work.) 
 But perhaps what you have in mind isn't necessarily "real-time" but 
just something adequately fast.  (E.g., very low frame rate with 
allowable occassional timing blip due to OS behavior.)

Also, I imagine running through and emulator will slow things down.

Dan


pawel@... wrote:

>Hi,
>I would appreciate answer to following problem related to fast updated
>real time data.
>
>I plot an ASCII file using a "plot" command. Upon arrival of new data
>(with a frequency close to 1 second) I issue a "replot" command.
>
>In effect I see that gnuplot_x11 takes al large amount of CPU resurces 
>(no matter if the gnuplot is iconified or not).
>
>Worst, when X11 is displayed on a PC through an emulator Exceed, Exceed
>runs
>out of memory (or so it says) no matter how it is configured.
>
>Questions are
>  1) What is the best way to use gnuplot for real-time visualization
>  (are there any buffering options reducing resouces needed for redraw)
>
>  2) How to iconify gnuplot windows (other through -noraise option)
>   so when window is inconified gnuplot does not eat X11 resources.
>
>
>Thanks
>
> Pawel
>
>------------------------------------------------------------------------
>
>---------------------------------------------------------
>
>This e-mail contains information some or all of which may be confidential, proprietary and/or legally privileged.  If an addressing or transmission error has misdirected this e-mail, please notify the sender by replying to this e-mail.  If you are not the intended recipient you must not use, disclose, distribute, copy, print or rely on this e-mail.
>
>---------------------------------------------------------
>  
>

-- 

Dan Sebald
email: daniel DOT sebald AT ieee DOT org
URL: http://acer-access DOT com/~dsebald AT acer-access DOT com/

On Thursday 28 October 2004 08:46 am, pawel@... wrote:
> 
> In effect I see that gnuplot_x11 takes al large amount of CPU resurces 
> (no matter if the gnuplot is iconified or not).

Please explain your configuration in more detail.
Is gnuplot_x11 running on the same PC?  Or is it running on another
machine and displaying on a separate X-server (the PC + Exceed)?

Also, please tell us exactly what version of gnuplot you are running.
If you built from the CVS source, please look in the ChangeLog to
see what was the most recent date of bug-fixes.

> Worst, when X11 is displayed on a PC through an emulator Exceed, Exceed
> runs out of memory (or so it says) no matter how it is configured.

There have been at least 2 memory leaks fixed in gnuplot_x11
since the release of version 4.0  (one that also affected 4.0, one
that did not). Neither was terribly serious for a typical X11 configuration
on a single unix/linux machine, but I could imagine that the effect on a
more complicated setup would be larger.  So it is at least possible
that switching to very recent CVS versions of gnuplot_x11 would fix
one of your problems.

> Questions are
>   1) What is the best way to use gnuplot for real-time visualization
>   (are there any buffering options reducing resouces needed for redraw)

If "real-time" means event-driven - there is no good way that I know of.
If "real-time" means repeated refresh - it's pretty easy:

	load 'setup.gnu'	    # prepare the plot description
	load 'loop.gnu'      # loop forever

where loop.gnu contains
	replot
	pause 1		# wait 1 second before looping
	reread

On my machine this 1 second loop and repeat consumes less than
0.5% of the CPU even for a reasonably complicated 3D surface.

>   2) How to iconify gnuplot windows (other through -noraise option)
>    so when window is inconified gnuplot does not eat X11 resources.

That is an interesting question. You will definitely need the -noraise
option (or "set term x11 noraise"). But also the gnuplot_x11 program
itself should check to see if it is currently iconfied, and skip at least 
some of its processing if so. I will have a look at what is possible.
I'm not sure there is much to be gained, however.  If things are working
properly it shouldn't be consuming substantial resources in any case.


-- 
Ethan A Merritt       merritt@...
Biomolecular Structure Center
Mailstop 357742
University of Washington, Seattle, WA 98195

pawel@... wrote:

>Daniel,
>Thanks for your reply.
>

You're welcome.

>What I am trying to do is to replot simple time series file every second
>(or every update whichever is less frequent). The file would look like 
>12:44:51 GOOG.O +193.00 
>12:44:52 GOOG.O +193.058 
>12:44:53 GOOG.O +193.95 
>.....
>and although file can be large (several thousand lines) the performance
>issue is not
>in just gnuplot itself (which takes about 10% of CPU), bbut gnuplot_x11
>that takes upto 20% and crashes my Exceed emulator of X. So I would
>think that objective is to reduce
>amount of data  sent to:
>

Well, several thousand lines is starting to get up there.  It would 
probably work on the setup Ethan described.  However, when you think of 
a video screen, it's what?  Maybe 800x1200?  Several thousand lines 
would mean each line averaging the space of a pixel.  In other words 
both gnuplot and gnuplot_x11 are doing a lot of number crunching to 
compute at a resolution that ends up being discarded.  (Unless you are 
going to do zooming!!  But in real time that's a big if.)  So reducing 
that data up front might help.  You can't just discard data points for 
fear of aliasing, perhaps something like averaging of the samples, i.e., 
reduce 2000 points to 200 points by averaging groups of ten.  (I'm 
talking uniform sampling now.)  Just a thought.

Dan

On Thursday 28 October 2004 08:46 am, pawel@... wrote:
> 
>   2) How to iconify gnuplot windows (other through -noraise option)
>    so when window is inconified gnuplot does not eat X11 resources.

I have placed a patchset on SourceForge that causes gnuplot_x11
to skip most of the actual X11 commands if the plot window is currently
iconified.  It still reads and buffers new commands coming from gnuplot,
however, so the CPU does not go to zero.

I don't really think this will accomplish much in practice.
The only way I was able to test it was to put gnuplot in an infinite
loop of continual replotting.  But this is not a reasonable thing to do.

Anyhow, you are welcome to try it out.  If it really does solve a 
problem, please let us know.

-- 
Ethan A Merritt       merritt@...
Biomolecular Structure Center
Mailstop 357742
University of Washington, Seattle, WA 98195

You could try adding this line somewhere near the top of gplt_x11.c

#define NOEXPORT

This will disable at least some of the selection code.
I have not looked in detail to see if there are remaining bits and
pieces that will not be disabled by this flag.



On Friday 29 October 2004 07:39 am, pawel@... wrote:
> 
> Dear Gnuplot Developers,
> 
> I found out that the gnuplot has a feature (bug?) that on a normal
> operation under plot or replot <SOMETIMES> grabs the X11 selection
> buffer.
> 
> That leads to various problems  like:
>   a) lost selection from other windows.
>   b) complete disfunctionality under Exceed X11 emulator which keeps
> trying to copy
>     X selection and runs out of memory.
>   c) sucks up CPU (it seem)
> 
> Why and when gnuplot grabs the selection and why, and is there a way to
> avoid.
> Thank you very much for your help. This would icidentally resolve all my
> other
> real-time plot problems.
> 
> 
> Thanks
>      Pawel

-- 
Ethan A Merritt       merritt@...
Biomolecular Structure Center
Mailstop 357742
University of Washington, Seattle, WA 98195

> I would appreciate if you could add a link on the gnuplot pages to
> http://ageco.tamu.edu/faculty/mccarl/gnuplot/
>
> GNUPLTXY is an interface from GAMS (Generalized Algebraic Modelling System)
> to GNUPLOT.

Ok, it's there (at gnuplot.sf.net -- to be mirrored soon).

BTW, you could replace your 3.6 manual to that of 4.0 (or link to gnuplot's
online version).

---
PM

On Thu, 28 Oct 2004, Daniel J Sebald wrote:

[...]

> So reducing that data up front might help.  You can't just discard data
> points for fear of aliasing, perhaps something like averaging of the
> samples, i.e., reduce 2000 points to 200 points by averaging groups of
> ten.  (I'm talking uniform sampling now.)  Just a thought.

In fact, something like the 'smooth uniq' filter may already help with 
that, if applied correctly.  Unfortunately though, the modifications
of the data needed to make 'smooth uniq' do something useful in a case 
like this are not possible for time data.  Pawel would have to

	plot 'data' using (int($1/5)*5):2

but time data don't allow extended using specs.

Eventually, though, the critical clue is that you need a circular
buffering strategy, and that has to be done outside gnuplot.  I.e. instead
of always plotting the *entire* file, a true real-time plot should have a
sliding window that only plots the last N seconds' (or whatever the x axis
unit is) worth of data, or the data since the full minute at least 10
minutes ago, or something like that.  But this has to be done outside
gnuplot: there's no 'every ::-500' or whatever to signify "select only the
last 500 datapoints".

-- 
Hans-Bernhard Broeker (broeker@...)
Even if all the snow were burnt, ashes would remain.

> > If this is so, then maybe you could explain in a little more detail how
> > you would like to accomplish this. How does the communication between
> > the widget class and gnuplot work (maybe pipes)? How do you display
> > gnuplot's output in the calling application? I vaguely remember a
> > discussion on the gnuplot-beta list about the feasibility of a Qt
> > terminal type. Do you want to use gnuplot in it's interactive mode
> > (zooming, data point labeling, etc.)? I'm sure the people on the
> > gnuplot-beta list would be interested to know, and can probably give a
> > lot of helpful tips.
> Currently the X11 child window is quite impossible to work with, so I am
> just dumping a png to a temp file and then drawing it on the widget.  I
> will probably have to implement the zooming, data point labeling and such
> manually.

Would not it be easier to implement a new Qt/GTK/FLTK/wx -terminal, that
would allow to be encapsulated in a particular widget? What would help many
other projects as well.

That could actually be made more easier than the current X11; and connected
to gnuplot either with a binary bidirectional piping (like OS/2) or compiled
together with gnuplot inside one executable without piping (like MSW). Then
it will be faster for plotting than X11.

---
PM

> Or maybe I'll code up a user preference setting for line spacing.
> That is, how much vertical space is implied by an embedded "\n"
> 'set line_spacing <foo>'
> 	term.c (write_multiline):
> 		y += user_prefs.line_spacing * t->v_char
> where right now line_spacing is always 1.0

So we will need:
global:
	set style linespacing 1.2
or
	set linespacing 1.2

and the same for a given text
	set label ... linespacing 1.2
	set key title ... linespacing 1.2

---
PM

Bernhard Seiwald wrote:

> loding the two attached scripts (t1.gpl plots cos(x) on x11 and to 
> eps-file; t2.gpl plot sin(x) onx11 and to eps-file):
> 
> load "t1.gpl"    # produces test1.eps  -> ok
> load "t2.gpl"    # produces test2.eps  -> ok   AND  destroyes test1.eps!!!

At least part of the responsibility for this problem lies in your 
scripts.  They fail to close the output files.  I.e. before you

	set term x11

you should

	set output

to close the PostScript output file.  The fact that the sequence
you used didn't work is a bug in gnuplot, but your scripts are at
least somewhat incorrect, too.

On Thursday 28 October 2004 03:41 am, Hans-Bernhard Broeker wrote:
> At least part of the responsibility for this problem lies in your
> scripts.  They fail to close the output files.  I.e. before you
>
> 	set term x11
>
> you should
>
> 	set output

That's not what the documentation says.
It says:
	When both `set terminal` and `set output` are used together
	it is safest to give `set terminal` first

It sounds like the bug being reported may be the same one that
is corrupting my PostScript output when I try to use push/pop 
term.  Do you know exactly what bit of the code is doing this?
I have been looking, but failing to find it.

Ethan Merritt wrote:

> That's not what the documentation says. It says: When both `set 
> terminal` and `set output` are used together it is safest to give 
> `set terminal` first

That's meant to be talking about when you open a terminal.  My comment
was about how you close it. Bernhard's scripts, executed in the given
order, do this:

	set term post
	set output 'file1'
	# plot something ...
	set term x11
	set term post
	set output 'file2'
	# ...
	reset
	set term x11

What I was suggesting was that there should usually be a 'set output'
to close the file, *before* the 'set term x11'.

> It sounds like the bug being reported may be the same one that is 
> corrupting my PostScript output when I try to use push/pop term.

It may very well be.  "set term pop" and "set term x11" are essentially
the same thing, and they both lack the closing 'set output'.

In a nutshell, 'set term' to a different driver, while output is
directed to a file, has to close that file (i.e. simulate 'set output')
at some point before the new driver writes anything to output file.  The
mechanisms to actually do that are quite intricate (check variable
term_initialised, and watch for calls to term_set_output()).