gnuplot-beta — Developers and beta testers of new gnuplot code

How to install demo files?

[Mojca M. <moj...@gm...>]

Hello,

Somebody on macports mailing list requested installation of demo
files. My question: is there any configure-time option that would
install the demos during "make install"? And even if not: what is the
usual unix location where these files should go?

Thank you,
    Mojca

Axes and autoscaling on new gnuplot

[Tait <gnu...@t4...>]

I'm sorry this is so vague, but has anybody noticed any unusual
behavior around autoscaling and "plot 'file' ... axes ..." in the
new gnuplot builds (on Windows, if it matters)?

I will try to investigate and create something reproducible, but I was
in a hurry at the time. It seemed like either autoscale wasn't working
at all, or that writeback was somehow enabled, even though gnuplot
reported nowriteback.

Re: relative performance of different interactive terminals

[<pl...@pi...>]

On 04/13/13 02:18, Ethan Merritt wrote:
> On my computer, x11 updates this plot between 26 and 28 times in 10 seconds

Hey, you can count quicker that I can ;)

This is good idea, is there a way to create a command line version of 
this or similar test that can be actually timed with 'time' command?

I use a fast 32 bit CPU. I use almost exclusively wxt and confirm that 
it's rather sluggish with data files of that size. But since I mostly do 
straight , with lines, plots I just wait a second or so. Useful to know 
x11 is much faster.

I have only done 3D plots on relatively small data files. It was a bit 
slow but usable. Next time I'll use x11.

It would be interesting to see where the bottlenecks are but I think 
there is alway a significant hit when passing through an extra layer of 
software.

The first graphics plotting software I produced in Turbo Pascal wrote 
straight to video memory and was plenty fast enough on an original XT PC 
with 480kB of RAM.

The more layers of abstraction you add the slower it gets.

Peter.

Re: relative performance of different interactive terminals

[Dima K. <gn...@di...>]

Jonathan Thornburg <jt...@as...> writes:

> Dima Kogan <gn...@di...> replied
>> My only reason for sticking with x11 is performance. Last year I tried
>> wxt and qt, and they were both noticeably more sluggish. I just tried it
>> again, and while qt was very slow, wxt seemed about as fast as x11. So
>> maybe I'll jump ship too in a bit.
>
> On my computer wxt is subjectively *much* more sluggish than x11.
> I just tried a quantitative performance test (details below), and wxt
> was about a factor of 3 slower than x11 when interactively rotating
> the 'splot' of a medium-sized data file.
>
> It's interesting that the relative performance of wxt vs x11 differs
> so much from one system to another.  I don't know whether this is the
> underlying graphics hardware/acceleration, some feature(s) of the
> different X servers used by wxt vs x11, or what.
>
> It would be interesting to run such performance tests on a range of
> different systems and see how the relative performance of different
> terminals varies from one {gnuplot version, compiler, computer system}
> to another.


I just tried out your test, and I do see the wxt being slow. I retract
my statement that it seems quick-enough now. :) I see similar
differences in performance on two thinkpad laptops (integrated Intel
graphics and nvidia nvs140). Previously I used a thinkpad with a radeon
x1400 (such as what you have), and it was similarly slow.

It looks line an important factor to the slowness is the 'lines' plots.
If you change your test case to plot with points instead of lines, the
x11 becomes a LOT slower, but the wxt only somewhat so, and the two
become comparable.

dima

Re: relative performance of different interactive terminals

[Ethan M. <merritt@u.washington.edu>]

On Friday, April 12, 2013 03:39:21 pm Jonathan Thornburg wrote:
> Ethan Merritt <merritt@u.washington.edu> writes:
> > Or maybe hardly anyone is using x11 anymore.  I know I strongly prefer
> > either wxt or qt to the x11 terminal.  I'm not saying we shouldn't fix
> > this if possible, but the reality is that I view the x11 terminal as having 
> > been superseded by newer, more capable terminal options.  There
> > are many things that it just doesn't handle very well (font scaling,
> > non-ascii encodings, transparency, anti-aliasing, ...).
> 
> Dima Kogan <gn...@di...> replied
> > My only reason for sticking with x11 is performance. Last year I tried
> > wxt and qt, and they were both noticeably more sluggish. I just tried it
> > again, and while qt was very slow, wxt seemed about as fast as x11. So
> > maybe I'll jump ship too in a bit.
> 
> On my computer wxt is subjectively *much* more sluggish than x11.
> I just tried a quantitative performance test (details below), and wxt
> was about a factor of 3 slower than x11 when interactively rotating
> the 'splot' of a medium-sized data file.

I don't think anyone is disputing that x11 is faster than wxt or qt 
for 3D rotation.  Qt in particular is slow in 3D, glacially slow for
image plots.  If that is your typical usage then I can certainly understand
a preference for the faster terminal[s].
 
> It would be interesting to run such performance tests on a range of
> different systems and see how the relative performance of different
> terminals varies from one {gnuplot version, compiler, computer system}
> to another.

32-bit linux   Intel(R) Core(TM)2 Duo CPU     E6750  @ 2.66GHz
gcc 4.4.3

qt  size 1200,1000  glacial
wxt size 1200,1000	approx 2.5 frames per second
x11 size 1200,1000  no visible lag in response to mouse movement

So yeah, the perceived speed runs from "glacial" to "infinitely fast".
On the other hand, wxt is quite fast enough to use for composing
a figure even though it doesn't spin smoothly.  If I understand your
report correctly then my setup refreshes wxt at about the same rate
as yours refreshes x11.

The x11 terminal throughput was improved by 10%-20% for large plots in
version 4.7.  That was accomplished by reducing the amount of data sent
through the pipe from gnuplot to gnuplot_x11, which doesn't necessarily
translate into an equivalent increase in the perceived interactive
responsiveness.

I have not looked into where the speed bottlenecks are in the
wxt terminal.  Maybe it could be sped up, maybe not.
Note that the speed I see for your test case does not seem to depend
on the display window size.  I think the time is almost entirely spent
copying data around internally, not by the actual graphics rendering.

I am told that using the OpenGL rendering mode for Qt makes it
significantly faster, but unfortunately that doesn't seem to be supported
by any of my test machines so I can't benchmark it.

	Ethan

 
> 
> Details of my performance test
> ==============================
> 
> This test measures the responsiveness of interactively rotating the
> 'splot' of a medium-sized data file.  The data file can be downloaded at
>    http://www.astro.indiana.edu/~jthorn/spool/gnuplot-speed-test.dat
> The file contains is a bit under 1/2 megabyte in size, and contains
> 16791 lines (16310 non-blank non-comment); each line containing 4 columns
> of data (of which only the first 3 are used in this test).
> 
> The actual performance test is to enter the gnuplot commands
>    set terminal ....
>    unset hidden3d
>    set style data lines
>    splot 'gnuplot-speed-test.dat'
> then enlarge the terminal window to a size of about 1200 x 1000 pixels,
> and then interactively rotate the 3-D plot.  Measure the frequency of
> updates by dragging the mouse (i.e., rotating the plot) continuously for
> 10 seconds and counting the number of times the plot updates.  If you
> have a rapidly-updating CPU-usage meter, you can watch it to see what
> fraction of the (a) CPU gnuplot and its outboard terminal driver are
> using.
> 
> 
> Test results
> ============
> I'm using
> >         G N U P L O T
> >         Version 4.6 patchlevel 0    last modified 2012-03-04 
> >         Build System: OpenBSD amd64
> 
> > Compile options:
> > -READLINE  +LIBREADLINE  -HISTORY  
> > -BACKWARDS_COMPATIBILITY  +BINARY_DATA  
> > +GD_PNG  +GD_JPEG  +GD_TTF  +GD_GIF  +ANIMATION  
> > -USE_CWDRC  +X11  +X11_POLYGON  +MULTIBYTE  +X11_EXTERNAL +USE_MOUSE  +HIDDEN3D_QUADTREE  
> > +DATASTRINGS  +HISTOGRAMS  +OBJECTS  +STRINGVARS  +MACROS  +IMAGE  +USER_LINETYPES +STATS 
> 
> compiled with gcc 4.2.1 -O2.  I'm run the tests on a Lenovo Thinkpad
> T60 laptop, Intel Core 2 T7200 dual-core cpu locked at its minimum clock
> rate of 1.0GHz (this makes timing easier), ATI Radeon Mobility X1400
> graphics, running OpenBSD 5.1-stable.
> 
> On my computer, x11 updates this plot between 26 and 28 times in 10 seconds
> (using about 70-80% CPU on one core), while wxt updates around 8 or 9
> times in 10 seconds (using about 100% CPU on one core).
> 
> ciao,
> 
> 

-- 
Ethan A Merritt
Biomolecular Structure Center,  K-428 Health Sciences Bldg
University of Washington, Seattle 98195-7742

relative performance of different interactive terminals

[Jonathan T. <jt...@as...>]

Ethan Merritt <merritt@u.washington.edu> writes:
> Or maybe hardly anyone is using x11 anymore.  I know I strongly prefer
> either wxt or qt to the x11 terminal.  I'm not saying we shouldn't fix
> this if possible, but the reality is that I view the x11 terminal as having 
> been superseded by newer, more capable terminal options.  There
> are many things that it just doesn't handle very well (font scaling,
> non-ascii encodings, transparency, anti-aliasing, ...).

Dima Kogan <gn...@di...> replied
> My only reason for sticking with x11 is performance. Last year I tried
> wxt and qt, and they were both noticeably more sluggish. I just tried it
> again, and while qt was very slow, wxt seemed about as fast as x11. So
> maybe I'll jump ship too in a bit.

On my computer wxt is subjectively *much* more sluggish than x11.
I just tried a quantitative performance test (details below), and wxt
was about a factor of 3 slower than x11 when interactively rotating
the 'splot' of a medium-sized data file.

It's interesting that the relative performance of wxt vs x11 differs
so much from one system to another.  I don't know whether this is the
underlying graphics hardware/acceleration, some feature(s) of the
different X servers used by wxt vs x11, or what.

It would be interesting to run such performance tests on a range of
different systems and see how the relative performance of different
terminals varies from one {gnuplot version, compiler, computer system}
to another.


Details of my performance test
==============================

This test measures the responsiveness of interactively rotating the
'splot' of a medium-sized data file.  The data file can be downloaded at
   http://www.astro.indiana.edu/~jthorn/spool/gnuplot-speed-test.dat
The file contains is a bit under 1/2 megabyte in size, and contains
16791 lines (16310 non-blank non-comment); each line containing 4 columns
of data (of which only the first 3 are used in this test).

The actual performance test is to enter the gnuplot commands
   set terminal ....
   unset hidden3d
   set style data lines
   splot 'gnuplot-speed-test.dat'
then enlarge the terminal window to a size of about 1200 x 1000 pixels,
and then interactively rotate the 3-D plot.  Measure the frequency of
updates by dragging the mouse (i.e., rotating the plot) continuously for
10 seconds and counting the number of times the plot updates.  If you
have a rapidly-updating CPU-usage meter, you can watch it to see what
fraction of the (a) CPU gnuplot and its outboard terminal driver are
using.


Test results
============
I'm using
>         G N U P L O T
>         Version 4.6 patchlevel 0    last modified 2012-03-04 
>         Build System: OpenBSD amd64

> Compile options:
> -READLINE  +LIBREADLINE  -HISTORY  
> -BACKWARDS_COMPATIBILITY  +BINARY_DATA  
> +GD_PNG  +GD_JPEG  +GD_TTF  +GD_GIF  +ANIMATION  
> -USE_CWDRC  +X11  +X11_POLYGON  +MULTIBYTE  +X11_EXTERNAL +USE_MOUSE  +HIDDEN3D_QUADTREE  
> +DATASTRINGS  +HISTOGRAMS  +OBJECTS  +STRINGVARS  +MACROS  +IMAGE  +USER_LINETYPES +STATS 

compiled with gcc 4.2.1 -O2.  I'm run the tests on a Lenovo Thinkpad
T60 laptop, Intel Core 2 T7200 dual-core cpu locked at its minimum clock
rate of 1.0GHz (this makes timing easier), ATI Radeon Mobility X1400
graphics, running OpenBSD 5.1-stable.

On my computer, x11 updates this plot between 26 and 28 times in 10 seconds
(using about 70-80% CPU on one core), while wxt updates around 8 or 9
times in 10 seconds (using about 100% CPU on one core).

ciao,

-- 
-- "Jonathan Thornburg [remove -animal to reply]" <jt...@as...>
   Dept of Astronomy & IUCSS, Indiana University, Bloomington, Indiana, USA
   on sabbatical in Canada starting August 2012
   "Washing one's hands of the conflict between the powerful and the
    powerless means to side with the powerful, not to be neutral."
                                      -- quote by Freire / poster by Oxfam

Re: 4.6.3 to include windows update [was: Any last-minute fixes or requests for inclusion in 4.6.2?]

[Bastian M. <bma...@we...>]

Am 11.04.2013 22:11, schrieb Ethan Merritt:
> On Friday, April 05, 2013 12:08:04 pm Bastian Märkisch wrote:
>>
>> I just uploaded new binaries which include a few fixes backported from
>> 4.7. The -persist mode (Bugs #1103 and #1145) should now be fixed. Happy
>> testing!
>
> I have just bumped the 4.6 cvs repository to patchlevel 3.
> Yes it's only been a month since the 4.6.2 release, but Bastian has updated
> the Windows support in order to prepare Windows distribution packages and I
> don't like the idea of having the linux and windows package contents be out
> of sync.
>
> Also there were a couple of regressions in 4.6.2 having to do with
> color handling ( monochrome option not always respected,
> fill and text colors out of sync in some of the latex terminals )
> that we might as well take the opportunity to fix.
>
> Bastian:
>
> Can you re-run your packaging scripts with the current (i.e. 4.6.3)
> CVS source?  Then we can put both an updated tarball and the
> windows packages on SourceForge labelled as 4.6.3.
>

You can now find Windows binary packages for 4.6.3 at
http://www.gnuplot.info/development/binaries/

   Bastian

4.6.3 to include windows update [was: Any last-minute fixes or requests for inclusion in 4.6.2?]

[Ethan M. <merritt@u.washington.edu>]

On Friday, April 05, 2013 12:08:04 pm Bastian Märkisch wrote:
> 
> I just uploaded new binaries which include a few fixes backported from 
> 4.7. The -persist mode (Bugs #1103 and #1145) should now be fixed. Happy
> testing!

I have just bumped the 4.6 cvs repository to patchlevel 3.
Yes it's only been a month since the 4.6.2 release, but Bastian has updated
the Windows support in order to prepare Windows distribution packages and I
don't like the idea of having the linux and windows package contents be out
of sync.

Also there were a couple of regressions in 4.6.2 having to do with
color handling ( monochrome option not always respected,
fill and text colors out of sync in some of the latex terminals )
that we might as well take the opportunity to fix.

Bastian:

Can you re-run your packaging scripts with the current (i.e. 4.6.3) 
CVS source?  Then we can put both an updated tarball and the 
windows packages on SourceForge labelled as 4.6.3. 

	Ethan

-- 
Ethan A Merritt
Biomolecular Structure Center,  K-428 Health Sciences Bldg
University of Washington, Seattle 98195-7742

Re: [PATCH] The autoscaling of images during volatile data refresh works again

[Dima K. <gn...@di...>]

Ethan Merritt <merritt@u.washington.edu> writes:

> On Friday, 05 April 2013, Dima Kogan wrote:
>> I'm attaching a patch that fixes another set of issues with refreshing
>> volatile data: autoscaling of images.
>
> I very much dislike that factor of 1e-3.  A better fix would remove it altogether.
> How about the attached patch instead.

Sounds great. I'm not a fan of that either.


>> Re-enabling the first two commented-out lines in the test case reveals
>> yet another set of issues. I'll look at those in a bit.
>
> The remaining problem I see is shown by adding to your test script
> set x2tics
> set y2tics
>
> Since there is no x2 or y2 data the autoscaling on those axes
> never happens, leading to an error equivalent to the one you
> found.  Possible fixes:
>
> If the axis min/max is still set to the initial autoscaling magic
> constants +/-VERYLARGE either
>   1) don't try to draw tick marks or
>   2) clone primary axis limits into secondary axis

Looks like you fixed this in the repo. Thanks!

dima

Re: [PATCH] some sizing fixes in the x11 terminal

[Dima K. <gn...@di...>]

Dima Kogan <gn...@di...> writes:

> I'm attaching a patch to fix some sizing errors that were introduced by
> some of my earlier X11 fixes. These were mostly confusion between height
> and gheight. An observable problem that this patch fixes is plotting
> something, and then doing a replot. With the same data file the replot
> shouldn't have any visible effects, but there was some geometric
> shifting as a result of the errors.

This wasn't applied yet and has received no comment. Is there something
wrong with this fix, or did it simply fall through the cracks?

Thanks
dima

Re: Intermittent error plotting images with the x11 terminal

[Dima K. <gn...@di...>]

Ethan Merritt <merritt@u.washington.edu> writes:
> On Sunday, 07 April 2013, Dima Kogan wrote:
>> Hi all.
>> 
>> I'm seeing an intermittent error that happens when plotting an image
>> with the x11 terminal.

I'm attaching two patches to fix this. I simply send a new command to
indicate the end of an image block, and the plotter now doesn't try to
process any commands in the middle of an image block.


> Strange that no one has reported this problem before now.
> On the other hand, I had a very hard time reproducing it with your
> test case.  I must have tried 50 times before I saw a single failure.
> So maybe it's just very rare.  Or maybe something on your machine
> configuration makes it unusually prone to this error.

I use a tiling window manager (notion), and I suspect this is the
reason. In any case, this was undeniably a bug.


> Or maybe hardly anyone is using x11 anymore.  I know I strongly prefer
> either wxt or qt to the x11 terminal.  I'm not saying we shouldn't fix
> this if possible, but the reality is that I view the x11 terminal as having 
> been superseded by newer, more capable terminal options.  There
> are many things that it just doesn't handle very well (font scaling,
> non-ascii encodings, transparency, anti-aliasing, ...).

My only reason for sticking with x11 is performance. Last year I tried
wxt and qt, and they were both noticeably more sluggish. I just tried it
again, and while qt was very slow, wxt seemed about as fast as x11. So
maybe I'll jump ship too in a bit.

dima

Re: Intermittent error plotting images with the x11 terminal

[Ethan M. <merritt@u.washington.edu>]

On Sunday, 07 April 2013, Jonathan Thornburg wrote:
> On Sun, 7 Apr 2013, Ethan Merritt wrote:
> > Or maybe hardly anyone is using x11 anymore.  I know I strongly prefer
> > either wxt or qt to the x11 terminal.  I'm not saying we shouldn't fix
> > this if possible, but the reality is that I view the x11 terminal as having 
> > been superseded by newer, more capable terminal options.  There
> > are many things that it just doesn't handle very well (font scaling,
> > non-ascii encodings, transparency, anti-aliasing, ...).
> 
> I very much hope we retain full support for the x11 terminal. 

Sure, for a definition of "full support" that includes acknowledging that 
the x11 terminal doesn't provide the entire current set of features.

> x11 is (and remains) my strongly-preferred interactive terminal for two key
> reasons:
> * x11 builds easily & cleanly even if 3rd-party libraries are broken
>   or live in odd places

I can see that being an advantage if you find yourself having to
rebuild from source on lots of different machines.  It doesn't count
for much in cases where you can run pre-built binaries, whether they
were provided by a distro or self-built on a fully-configured machine
for deployment on less completely configured machines like in a
computer lab.

> * x11 respects X resource specifications in ~/.Xdefaults
>   (e.g. I have the line  "gnuplot*reverseVideo: on" there, so x11 comes
>   up with a black background.  wxt ignores that and comes up with a white
>   background which I find ugly.  I don't know what qt does.)

The qt terminal provides a widget for selecting the background color.  
The choice is saved in ~/.qt/qtrc.

Current gnuplot allows you to place linetype preferences in your
~/.gnuplot initialization file, which is a more general solution than
using X resources.

	Ethan

> ciao,
> 
> 

Re: Intermittent error plotting images with the x11 terminal

[Jonathan T. <jt...@as...>]

On Sun, 7 Apr 2013, Ethan Merritt wrote:
> Or maybe hardly anyone is using x11 anymore.  I know I strongly prefer
> either wxt or qt to the x11 terminal.  I'm not saying we shouldn't fix
> this if possible, but the reality is that I view the x11 terminal as having 
> been superseded by newer, more capable terminal options.  There
> are many things that it just doesn't handle very well (font scaling,
> non-ascii encodings, transparency, anti-aliasing, ...).

I very much hope we retain full support for the x11 terminal.  x11 is
(and remains) my strongly-preferred interactive terminal for two key
reasons:
* x11 builds easily & cleanly even if 3rd-party libraries are broken
  or live in odd places
* x11 respects X resource specifications in ~/.Xdefaults
  (e.g. I have the line  "gnuplot*reverseVideo: on" there, so x11 comes
  up with a black background.  wxt ignores that and comes up with a white
  background which I find ugly.  I don't know what qt does.)

ciao,

-- 
-- Jonathan Thornburg <jt...@as...>
   Dept of Astronomy & IUCSS, Indiana University, Bloomington, Indiana, USA
   on sabbatical on Thetis Island, BC, Canada starting August 2012
   "Washing one's hands of the conflict between the powerful and the
    powerless means to side with the powerful, not to be neutral."
                                      -- quote by Freire / poster by Oxfam

Re: Intermittent error plotting images with the x11 terminal

[Ethan M. <merritt@u.washington.edu>]

On Sunday, 07 April 2013, Dima Kogan wrote:
> Hi all.
> 
> I'm seeing an intermittent error that happens when plotting an image
> with the x11 terminal. To reproduce:
> 
> - Fresh checkout of gnuplot
> - $ ./prepare && ./configure && make -j2
> - $ export GNUPLOT_DRIVER_DIR=$PWD/src
> - $ src/gnuplot /tmp/tst.gp
> 
> The data file is attached (it's mostly the same as the one in an
> unrelated email a few days ago). Normally, that command will make a plot
> and exit immediately; the plot window should quickly flicker. There is a
> bug, however, that causes this plot command to fail occasionally. If I
> run that last command repeatedly, I sometimes get
> 
>  GNUPLOT (gplt_x11):  Couldn't read image parameters correctly.
> 
> When this error pops up, the resulting plot is generally corrupt. The
> error happens much more readily if the plot window comes up under the
> mouse cursor. I'm using a more-or-less stock Debian/unstable box, and I
> see this error often, even when I'm using gnuplot to get work done, not
> just trying to coax out the bug.
> 
> I dug into this, and I see what's happening. There's quite a bit of
> state in gplt_x11.c and it doesn't all stay in sync all the time. The
> image is transferred into gplt_x11.c from gnuplot itself using lots of
> 'i' commands. The bug occurs when an X EnterNotify event is received,
> and causes display() to be called in the middle of the image transfer.
> The display() then processes all the 'i' commands that have come through
> so far; not all of them arrived yet, so it doesn't actually draw
> anything. There is state in processing the 'i' commands, however. This
> state is normally reset when the last 'i' command is processed to
> complete the image. Here, we never completed the image, so the state is
> never reset, and gnuplot is confused the next time display() is called
> when all the data is finally available. The static variables in question
> are in exec_cmd() in this block:
> 
>      else if (*buffer == X11_GR_IMAGE) {	/* image */
> 
>          static unsigned char *iptr;
>          static TBOOLEAN transferring = 0;
>          static unsigned short *image;
>          static int M, N;
>          static int pixel_1_1_x, pixel_1_1_y, pixel_M_N_x, pixel_M_N_y;
>          static int visual_1_1_x, visual_1_1_y, visual_M_N_x, visual_M_N_y;
>          static int color_mode;
>          static unsigned int i_remaining;
> 
> 
> There are multiple ways to address this, so I'm not making fixes yet. We
> could disallow calling display() with incomplete data, or we could reset
> the state at the start of the display() call. As a separate comment,
> making this less stateful and simplifying the image transfer would be a
> great thing to do in either case, I think.
> 
> Is any particular approach favored here?

I am not familiar with the code, but from your description I'd say that
the proper approach is to accumulate the image data in a separate
buffer and only link it into the display buffer when the transfer 
has completed.  If the the code already does that and the problem
happens anyhow, then events (or at least event handling) need to
be locked out while the display buffer is being modified.

Strange that no one has reported this problem before now.
On the other hand, I had a very hard time reproducing it with your
test case.  I must have tried 50 times before I saw a single failure.
So maybe it's just very rare.  Or maybe something on your machine
configuration makes it unusually prone to this error.

Or maybe hardly anyone is using x11 anymore.  I know I strongly prefer
either wxt or qt to the x11 terminal.  I'm not saying we shouldn't fix
this if possible, but the reality is that I view the x11 terminal as having 
been superseded by newer, more capable terminal options.  There
are many things that it just doesn't handle very well (font scaling,
non-ascii encodings, transparency, anti-aliasing, ...).

	Ethan



> 
> dima
> 
> 

Intermittent error plotting images with the x11 terminal

[Dima K. <gn...@di...>]

Hi all.

I'm seeing an intermittent error that happens when plotting an image
with the x11 terminal. To reproduce:

- Fresh checkout of gnuplot
- $ ./prepare && ./configure && make -j2
- $ export GNUPLOT_DRIVER_DIR=$PWD/src
- $ src/gnuplot /tmp/tst.gp

The data file is attached (it's mostly the same as the one in an
unrelated email a few days ago). Normally, that command will make a plot
and exit immediately; the plot window should quickly flicker. There is a
bug, however, that causes this plot command to fail occasionally. If I
run that last command repeatedly, I sometimes get

 GNUPLOT (gplt_x11):  Couldn't read image parameters correctly.

When this error pops up, the resulting plot is generally corrupt. The
error happens much more readily if the plot window comes up under the
mouse cursor. I'm using a more-or-less stock Debian/unstable box, and I
see this error often, even when I'm using gnuplot to get work done, not
just trying to coax out the bug.

I dug into this, and I see what's happening. There's quite a bit of
state in gplt_x11.c and it doesn't all stay in sync all the time. The
image is transferred into gplt_x11.c from gnuplot itself using lots of
'i' commands. The bug occurs when an X EnterNotify event is received,
and causes display() to be called in the middle of the image transfer.
The display() then processes all the 'i' commands that have come through
so far; not all of them arrived yet, so it doesn't actually draw
anything. There is state in processing the 'i' commands, however. This
state is normally reset when the last 'i' command is processed to
complete the image. Here, we never completed the image, so the state is
never reset, and gnuplot is confused the next time display() is called
when all the data is finally available. The static variables in question
are in exec_cmd() in this block:

     else if (*buffer == X11_GR_IMAGE) {	/* image */

         static unsigned char *iptr;
         static TBOOLEAN transferring = 0;
         static unsigned short *image;
         static int M, N;
         static int pixel_1_1_x, pixel_1_1_y, pixel_M_N_x, pixel_M_N_y;
         static int visual_1_1_x, visual_1_1_y, visual_M_N_x, visual_M_N_y;
         static int color_mode;
         static unsigned int i_remaining;


There are multiple ways to address this, so I'm not making fixes yet. We
could disallow calling display() with incomplete data, or we could reset
the state at the start of the display() call. As a separate comment,
making this less stateful and simplifying the image transfer would be a
great thing to do in either case, I think.

Is any particular approach favored here?

dima

Re: [PATCH] The autoscaling of images during volatile data refresh works again

[Ethan M. <merritt@u.washington.edu>]

On Friday, 05 April 2013, Dima Kogan wrote:
> I'm attaching a patch that fixes another set of issues with refreshing
> volatile data: autoscaling of images. To observe the issue being fixed,
> start up gnuplot and issue
> 
>  load "volatile_image_refresh.gp"
> 
> where the given file is the attached test case. This is the blutux.rgb
> demo image, but stored inline, rather than as a separate file. The load
> will succeed. If you then refresh the plot with 'e', the axis ranges
> will get confused.
> 
> The attached patch fixes this issue, which is actually a regression from
> an earlier bug fix. The earlier bug fix reduced the range of the data
> that gnuplot can use by 1e6 for the volatile-refresh case. This was
> confusing things, so the attached patch makes this range adjustment for
> /all/ data. If we can figure out exactly what this original range
> reduction was fixing, maybe a better solution can be found. Comment from
> the patch and the code:
> 
>  This was a regression, caused by an unrelated bug fix that modified the meaning
>  of VERYLARGE in some cases. Relevant comment:
> 
>  An earlier bug fix introduced this factor in some narrow circumstances
>  (AXIS_INIT2D_REFRESH macro) to get around some overflow issues. However, this
>  broke some autoscaling functionality (specifically 2d image autoscaling when
>  refreshing volatile data) because the STORE_WITH_LOG_AND_UPDATE_RANGE() macro
>  has == comparisons with VERYLARGE. Here, I apply the factor to VERYLARGE always.
>  The bug fix in question is in the commit titled
> 
>   Date: Sun Mar 23 21:39:26 2008 +0000
>   Fix macros AXIS_INIT2D_REFRESH and AXIS_UPDATE2D_REFRESH so that the refresh
>   command for volatile data works correctly for log axes (SF 1916494).
> 
>  The bug it refers to is http://sourceforge.net/p/gnuplot/bugs/636/
> 
>  The comment in that commit was
> 
>  if an already VERYLARGE x2 and y2 ranges are
>  calculated after zoom-out by mouse, then they would become even larger

I very much dislike that factor of 1e-3.  A better fix would remove it altogether.
How about the attached patch instead.

> Re-enabling the first two commented-out lines in the test case reveals
> yet another set of issues. I'll look at those in a bit.

The remaining problem I see is shown by adding to your test script
set x2tics
set y2tics

Since there is no x2 or y2 data the autoscaling on those axes
never happens, leading to an error equivalent to the one you
found.  Possible fixes:

If the axis min/max is still set to the initial autoscaling magic
constants +/-VERYLARGE either
  1) don't try to draw tick marks or
  2) clone primary axis limits into secondary axis

	Ethan 

Re: Any last-minute fixes or requests for inclusion in 4.6.2?

[Bastian M. <bma...@we...>]

Am 04.04.2013 20:56, schrieb Bastian Märkisch:
> Am 01.04.2013 21:52, schrieb Ethan Merritt:
>> On Friday, March 22, 2013 11:34:33 am Bastian Märkisch wrote:
>>> Please find a release candidate of a Windows installer and a zip package at
>>> http://www.gnuplot.info/development/binaries/
>>>
>>> Please test them thoroughly so that we can make them available to the
>>> public.
>>
>> What's the status on this - should it be moved/copied to the SourceForge
>> download site?
>>
>> 	Ethan
>>
>
> The only feedback so far was from Hans-Bernhard. I am reluctant
> to proceed with the release because the "-persist" mode is broken in
> current CVS and 4.6.2 on Windows in two ways: First of all it is not
> working at all. Apparently a code snippet for Windows got removed by
> accident on Feb 3rd. Second, even if it would work, it would still
> create zombie processes if the wxt terminal is used, see bug #1103.
> This bug has been fixed in CVS and I am working on backporting it
> together with some other fixes to the 4.6 tree.
>
>     Bastian
>

I just uploaded new binaries which include a few fixes backported from 
4.7. The -persist mode (Bugs #1103 and #1145) should now be fixed. Happy
testing!

   Bastian

Re: 30-year time offset reading nonuniform matrix %s input

[Ethan M. <merritt@u.washington.edu>]

On Friday, April 05, 2013 08:46:42 am Dave Denholm wrote:
> Ethan Merritt <merritt@u.washington.edu> writes:
> 
> > I'd still like to know what the origianl rationale was for 
> > choosing 2000 rather than 1970.  I know plenty of problems that
> > this choice caused, but what problems did it solve?
> > Are they still relevant?
> >
> 
> Sorry, I think it was probably me.  I didn't use unix back then,
> so 2000 seemed as arbitrary as 1970.
> 
> Originally it wasn't intended that it be visible to the user anyway,
> but it was discovered later that you could use numbers rather than strings
> in things like setting ranges for time data.
> 
> I think I also had in mind that, because time was stored a real number,
> it wasn't good to carry a large offset around since that limited the
> resolution available for small numbers. But I think that's probably not
> relevant in practise.
>
> Dave D

Thank you for insight into the history.
I'll go ahead and change it in CVS for the development version, 
with comments attached in the relevant places.  
Reversion, if necessary, would be trivial.

We know this fixes several problems, including the one that kicked
off the current thread of discussion.  The only down side I know of
is that the javascript scripts used for mousing svg and HTML5 plots
must now match the version of gnuplot used to generate the plot.
This was probably true already for other reasons, but now a mismatch
would be evident when tracking time coordinates as well.


	Ethan


-- 
Ethan A Merritt
Biomolecular Structure Center,  K-428 Health Sciences Bldg
University of Washington, Seattle 98195-7742

Re: 30-year time offset reading nonuniform matrix %s input

[Dave D. <drd...@gm...>]

Ethan Merritt <merritt@u.washington.edu> writes:

>
> I'd still like to know what the origianl rationale was for 
> choosing 2000 rather than 1970.  I know plenty of problems that
> this choice caused, but what problems did it solve?
> Are they still relevant?
>

Sorry, I think it was probably me.  I didn't use unix back then,
so 2000 seemed as arbitrary as 1970.

Originally it wasn't intended that it be visible to the user anyway,
but it was discovered later that you could use numbers rather than strings
in things like setting ranges for time data.

I think I also had in mind that, because time was stored a real number,
it wasn't good to carry a large offset around since that limited the
resolution available for small numbers. But I think that's probably not
relevant in practise.

Dave D

[PATCH] The autoscaling of images during volatile data refresh works again

[Dima K. <gn...@di...>]

I'm attaching a patch that fixes another set of issues with refreshing
volatile data: autoscaling of images. To observe the issue being fixed,
start up gnuplot and issue

 load "volatile_image_refresh.gp"

where the given file is the attached test case. This is the blutux.rgb
demo image, but stored inline, rather than as a separate file. The load
will succeed. If you then refresh the plot with 'e', the axis ranges
will get confused.

The attached patch fixes this issue, which is actually a regression from
an earlier bug fix. The earlier bug fix reduced the range of the data
that gnuplot can use by 1e6 for the volatile-refresh case. This was
confusing things, so the attached patch makes this range adjustment for
/all/ data. If we can figure out exactly what this original range
reduction was fixing, maybe a better solution can be found. Comment from
the patch and the code:

 This was a regression, caused by an unrelated bug fix that modified the meaning
 of VERYLARGE in some cases. Relevant comment:

 An earlier bug fix introduced this factor in some narrow circumstances
 (AXIS_INIT2D_REFRESH macro) to get around some overflow issues. However, this
 broke some autoscaling functionality (specifically 2d image autoscaling when
 refreshing volatile data) because the STORE_WITH_LOG_AND_UPDATE_RANGE() macro
 has == comparisons with VERYLARGE. Here, I apply the factor to VERYLARGE always.
 The bug fix in question is in the commit titled

  Date: Sun Mar 23 21:39:26 2008 +0000
  Fix macros AXIS_INIT2D_REFRESH and AXIS_UPDATE2D_REFRESH so that the refresh
  command for volatile data works correctly for log axes (SF 1916494).

 The bug it refers to is http://sourceforge.net/p/gnuplot/bugs/636/

 The comment in that commit was

 if an already VERYLARGE x2 and y2 ranges are
 calculated after zoom-out by mouse, then they would become even larger


Re-enabling the first two commented-out lines in the test case reveals
yet another set of issues. I'll look at those in a bit.

dima

Re: [PATCH] volatile-data plots now handle 'reversed' ranges during replots

[Ethan M. <merritt@u.washington.edu>]

On Wednesday, April 03, 2013 12:46:23 pm Dima Kogan wrote:
> Ethan Merritt <merritt@u.washington.edu> writes:
> 
> > On Wednesday, April 03, 2013 02:57:01 am Dima Kogan wrote:
> >> I'm attaching another patch. This one adds support to the "reverse"
> >> ranges setting for replots of volatile data. This is similar to the
> >> previous ranging fixes, where the setting was respected for the original
> >> plot, but ignored for replots.
> >> 
> >> I'm also including the test cases I used to validate this. For all the
> >> test cases, I made sure that both the original plot and a replot looked
> >> good.
> >
> > I'm not seeing any difference between the patched and the unpatched
> > behavior.  Could you give me a specific set of instructions to test,
> > inclding any mouse or replot/refresh commands?
> >
> > Both the patched and unpatched code versions lose track of the "reverse"
> > setting after an unzoom operation using the "u" hotkey.
> 
> That's odd. I just tried again with a clean checkout, and the patch
> seems to work, both through replots ('e' key) and zoom/unzoom (mouse and
> 'u' key). 

I must have gotten confused as to what patches had been applied
to the version I was testing.  It all looks good now that I have 
started from a clean copy.

	Ethan




> The tree I'm looking at has both of the patches I sent
> yesterday, so the last few commits are titled
> 
>  document use of "smooth frequency" to generate and plot a histogram
>  Allow level 1 PostScript interpreters to skip large (>64kB) embedded images
>  x11 terminal: corrected some sizing inconsistencies
>  volatile-data plots now handle 'reversed' ranges during replots
> 
> After the tree is checked out, I do
> 
>  export GNUPLOT_DRIVER_DIR=$PWD/src
>  ./prepare && ./configure && make -j2 && src/gnuplot
> 
> Then inside gnuplot, I load one of the testcases
> 
>  load "/tmp/testcases/reverse_keywordspec.gp"
> 
> Then I press 'e' to replot, do some zooming with the mouse, and press
> 'u' to unzoom. The right answer is a downward sloping line (reversed y
> axis). I see this even through 'e' and zoom/'u'. Before the attached
> patch only the initial plot looked right.
> 
> dima
> 
> ------------------------------------------------------------------------------
> Minimize network downtime and maximize team effectiveness.
> Reduce network management and security costs.Learn how to hire 
> the most talented Cisco Certified professionals. Visit the 
> Employer Resources Portal
> http://www.cisco.com/web/learning/employer_resources/index.html
> _______________________________________________
> gnuplot-beta mailing list
> gnu...@li...
> https://lists.sourceforge.net/lists/listinfo/gnuplot-beta
> 

-- 
Ethan A Merritt
Biomolecular Structure Center,  K-428 Health Sciences Bldg
University of Washington, Seattle 98195-7742

Re: 30-year time offset reading nonuniform matrix %s input

[Juhász P. <pet...@gm...>]

On Thu, 2013-04-04 at 17:08 +0200, Petr Mikulik wrote:
> >> In the first case, the year is correctly set to 2013. In the second,
> >> it is displayed as 2043. This seems to be a mistake in the nonuniform
> >> matrix time input?
> >
> > This probably fixes it, but the possible consequences elsewhere are not
> > certain.  At a minimum it makes all references to the year 2000 in the
> > manual incorrect, since it resets the epoch boundary to 1970.
> 
> What about adding
>  	set datafile epoch 2000
> so that the year can be set by the user?
> 
> ---
> PM

While I can imagine some specific use cases where this would not only
make sense, but would be actually useful (e.g. for an astronomical time
series plot one might want to set the epoch to MJD 0), in general I
think the added complexity and confusion makes such a feature not worth
it.

Peter

Re: Any last-minute fixes or requests for inclusion in 4.6.2?

[Bastian M. <bma...@we...>]

Am 01.04.2013 21:52, schrieb Ethan Merritt:
> On Friday, March 22, 2013 11:34:33 am Bastian Märkisch wrote:
>> Please find a release candidate of a Windows installer and a zip package at
>> http://www.gnuplot.info/development/binaries/
>>
>> Please test them thoroughly so that we can make them available to the
>> public.
>
> What's the status on this - should it be moved/copied to the SourceForge
> download site?
>
> 	Ethan
>

The only feedback so far was from Hans-Bernhard. I am reluctant
to proceed with the release because the "-persist" mode is broken in
current CVS and 4.6.2 on Windows in two ways: First of all it is not
working at all. Apparently a code snippet for Windows got removed by
accident on Feb 3rd. Second, even if it would work, it would still
create zombie processes if the wxt terminal is used, see bug #1103.
This bug has been fixed in CVS and I am working on backporting it
together with some other fixes to the 4.6 tree.

   Bastian

Re: 30-year time offset reading nonuniform matrix %s input

[Daniel J S. <dan...@ie...>]

On 04/04/2013 01:38 PM, Ethan Merritt wrote:

> I'd still like to know what the origianl rationale was for
> choosing 2000 rather than 1970.  I know plenty of problems that
> this choice caused, but what problems did it solve?
> Are they still relevant?

The threat of planetary collapse due to the millennium time change?

Dan

Re: 30-year time offset reading nonuniform matrix %s input

[Ethan M. <merritt@u.washington.edu>]

On Thursday, April 04, 2013 08:08:08 am Petr Mikulik wrote:
> Tait wrote:
> >> In the first case, the year is correctly set to 2013. In the second,
> >> it is displayed as 2043. This seems to be a mistake in the nonuniform
> >> matrix time input?
> 
> Ethan wrote:
> > This probably fixes it, but the possible consequences elsewhere are not
> > certain.  At a minimum it makes all references to the year 2000 in the
> > manual incorrect, since it resets the epoch boundary to 1970.

Petr wrote:
> What about adding
>  	set datafile epoch 2000
> so that the year can be set by the user?

The epoch affects time calculation throughout the program, so tying 
it to "set datafile" would be misleading.  For example
gnuplot> print time(0)
prints number of seconds since the epoch boundary without any 
data file being accessed.

Or did you mean that we should add a per-datafile option that
would temporarily override the program's default epoch only 
for the purpose of reading that one file?  That would be
possible, but I am not sure that it solves any real-world problem.
If the data file contains time represented as a number of raw seconds,
which is I think the only case this would affect, then you can
already read it in as a plain number and convert to time/date using 
whatever epoch adjustment you like.  Right?  Or maybe not? I don't
normally use the time format routines so I am kind of hazy on what
they can or can't do.

I'd still like to know what the origianl rationale was for 
choosing 2000 rather than 1970.  I know plenty of problems that
this choice caused, but what problems did it solve?
Are they still relevant?

	Ethan

-- 
Ethan A Merritt
Biomolecular Structure Center,  K-428 Health Sciences Bldg
University of Washington, Seattle 98195-7742