gnuplot-beta — Developers and beta testers of new gnuplot code

AW: filenames on MS Windows

[Bastian M. <bma...@we...>]

Allin,

Thank you for this nice contribution.  On Windows, we already replace
fopen() (and many other functions), though.  In particular, win_fopen() in
winmain.c already handles encodings, including UTF-8. There are also two
functions AnsiText() and UnicodeText() to convert to/from UTF16 according to
gnuplot's encoding. They use the simple-to-use Windows functions
WideCharToMultiByte() MultiByteToWideChar(), so no need for glib.

That really poses the question if we should change the default internal
encoding from "ANSI" (whatever that may be depends on the Windows locale) to
UTF-8.  I agree with that (and in fact my personal gnuplot.ini includes "set
encoding utf8" since a long time).

But how do we make this backward compatible since that will inevitably break
load commands in old "ANSI" encoded user scripts? (as does your current
patch btw) Possible solutions include a new command line option (like -u /
--utf8)  or  a wgnuplot.ini setting.

  Bastian


> -----Ursprüngliche Nachricht-----
> Von: Allin Cottrell <cot...@wf...>
> Gesendet: Sonntag, 1. November 2020 01:36
> An: gnuplot-beta <gnu...@li...>
> Betreff: Re: filenames on MS Windows
> 
> On Sun, 25 Oct 2020, Allin Cottrell wrote:
> 
> > I tried googling this but didn't find an answer -- sorry if I should
> > have just tried harder! My question is: can gnuplot on Windows handle
> > a unicode filename argument passed in UTF-16? As in
> >
> > path/to/wgnuplot.exe <UTF-16 input filename>
> 
> OK, that question was under-researched, but now I've done my homework.
> Sorry, this is a bit long but I hope I can arouse some interest in the
topic.
> 
> Why bother with UTF-16 filename arguments? Nowadays a fair number of
> Windows users construct paths (directory names or filenames) which are
"out
> of codepage" -- that is, unicode names which cannot be represented in the
> (retro) "system codepage", which is typically just an 8-bit encoding.
Since
> Windows has supported unicode since NT came out, it's a reasonable
> expectation that any filename one can construct on the platform should be
> accessible via any program of interest. But a program that restricts
itself to the
> "ANSI" form of filenames simply cannot access files with out-of-codepage
> paths.
> (Sane modern OSes don't have this problem because they use UTF-8
> throughout.)
> 
> So what about gnuplot? I may be wrong but it seems to me that gnuplot on
> Windows is stuck with "ANSI" filenames at present. Even with UNICODE and
> _UNICODE defined when compiling the program, the command-line arguments
> are retrieved in winmain.c using either _argv or __argv (depending on the
> compiler), and these get the ANSI-form arguments (as opposed to __wargv
> which gets the arguments in UTF-16 form).
> 
> It would be easy to swap out __argv for __wargv but by itself this would
be
> very disruptive. The subsequent code in winmain.c, and then the code in
plot.c
> (gnu_main) to which the args array is passed, all assumes the elements of
argv
> are plain "char *", not "wide char"
> arrays. Handling UTF-16, which is chock-full of NUL bytes, would require
lots of
> messy "ifdefs".
> 
> I have a proposal for fixing this. I realise it may not be acceptable as
it stands
> but maybe someone else might want to take it up. I'm attaching patches for
> src/win/winmain.c and src/misc.c for reference but here I'll try to
explain the
> strategy.
> 
> 1) In winmain.c, grab the command-line arguments as UTF-16 but immediately
> convert them to UTF-8, so they can handled by the regular string.h APIs,
both
> here and in plot.c (gnu_main).
> 
> 2) When we actually go to open a command-line file argument
> (loadpath_fopen, in misc.c, called from gnu_main), we first try opening
the file
> using the filename as-is, but it that fails (and the filename validates as
UTF-8)
> we convert it to UTF-16 and try again.
> 
> Since UTF-8 is a superset of ASCII, ASCII filename arguments should pass
> through transparently. Within-codepage non-ASCII filenames should get
> converted back to UTF-16 and opened OK. And the bonus is that out-of-
> codepage arguments should also be converted and opened OK.
> 
> I've tested this on Windows 10, with the system codepage set to Windows
> 1252 ("Western Europe"), and have successfully opened files with names in
> Russian and Greek. (I think this should also work if the user has the
system
> codepage set to UTF-8 (65001), which is a "beta" option on Windows.)
> 
> My implementation uses GLib APIs (nice and simple) to convert from
> UTF-16 to UTF-8 and back again (if needed). GLib is required anyway if one
is
> building the Cairo-based terminals. I suppose one could use native Windows
> APIs to the same purpose but I suspect it would be a lot more bother.
> 
> In my test setup this whole deal is triggered by the CFLAGS define
> 
> -DWIDE_ARGS
> 
> which is respected only when building for Windows -- and admittedly has
only
> been tested when cross-compiling for Windows from Linux using Mingw-w64.
> In my mingw Makefile, I have:
> 
> WIDE_ARGS = 1
> 
> ...
> 
> ifdef WIDE_ARGS
>    CFLAGS += -DWIDE_ARGS
>    CFLAGS += $(shell pkg-config --cflags glib-2.0) endif
> 
> --
> Allin Cottrell
> Department of Economics
> Wake Forest University

AW: filenames on MS Windows

[Bastian M. <bma...@we...>]

Right now, gnuplot is able to "load" file names with Unicode encoded names, i.e. the sequence
	set encoding utf8
	load 'абвгдежзийклмнопрстуфхцчшщъыьэюяѐёђѓєѕіїјљњћќѝўџ.plt'
will work just fine. Only loading via the command line does not work and that should indeed be improved. (I don't agree to use glib for that purpose - but that is easy to change).

  Bastian

> -----Ursprüngliche Nachricht-----
> Von: Allin Cottrell <cot...@wf...>
> Gesendet: Sonntag, 1. November 2020 21:24
> An: Ethan A Merritt <me...@uw...>
> Cc: gnuplot-beta <gnu...@li...>
> Betreff: Re: filenames on MS Windows
> 
> On Sun, 1 Nov 2020, Ethan A Merritt wrote:
> 
> > On Saturday, 31 October 2020 17:36:03 PST Allin Cottrell wrote:
> >> On Sun, 25 Oct 2020, Allin Cottrell wrote:
> >>
> >>> I tried googling this but didn't find an answer -- sorry if I should
> >>> have just tried harder! My question is: can gnuplot on Windows
> >>> handle a unicode filename argument passed in UTF-16? As in
> >>>
> >>> path/to/wgnuplot.exe <UTF-16 input filename>
> >>
> >> OK, that question was under-researched, but now I've done my
> >> homework. Sorry, this is a bit long but I hope I can arouse some
> >> interest in the topic.
> >
> > I don't have any direct insight into this issue other than to note
> > that the filesytem itself may be an issue.
> 
> In some contexts, no doubt. But if we set aside exotica such as surrogate pairs,
> NTFS filenames are UTF-16 to a very good approximation. As such they are
> easily converted to UTF-8 to permit handling with good old C char * APIs, and
> easily converted back to
> UTF-16 for _wfopen() if required.
> 
> > The following entry from the R developer blog is of interest
> >
> >
> > https://developer.r-project.org/Blog/public/2020/05/02/utf-8-support-o
> > n-windows/
> >
> > I gather from the discussion there that Windows-10 can be made to
> > support UTF-8 as a native encoding, calling it "extended ASCII".
> 
> Interesting, yes, but at this point kinda science fiction. The practical issue at
> present is whether gnuplot wants to support out-of-codepage UTF-16
> filenames on the Windows command line. It's not terribly difficult, as I tried to
> show.
> 
> Sorry if I'm being repetitive, but right now if a create, say, a Russian-language
> filename on Windows and pass it as command-line argument to gnuplot,
> gnuplot will not be able to open the file because its name cannot be
> represented in my "system codepage". A program that reads the command line
> as UTF-16, however, will have no problem opening the file.
> 
> Allin Cottrell

Re: with pointcolor

[Dima K. <gn...@di...>]

Ethan A Merritt <me...@uw...> writes:

> Plot elements are drawn using the current pen, and the keyword
> for specifying the color of the current pen is "linecolor".

Thanks for the clarification. The name of the option is un-ideal, but
I think a bit more documentation would help, specifically "help points"
and "help lines" should at least mention "linecolor". I'll write
something in the near future.

Thanks

Re: filenames on MS Windows

[Allin C. <cot...@wf...>]

On Sun, 1 Nov 2020, Ethan A Merritt wrote:

> On Saturday, 31 October 2020 17:36:03 PST Allin Cottrell wrote:
>> On Sun, 25 Oct 2020, Allin Cottrell wrote:
>>
>>> I tried googling this but didn't find an answer -- sorry if I should have
>>> just tried harder! My question is: can gnuplot on Windows handle a unicode
>>> filename argument passed in UTF-16? As in
>>>
>>> path/to/wgnuplot.exe <UTF-16 input filename>
>>
>> OK, that question was under-researched, but now I've done my
>> homework. Sorry, this is a bit long but I hope I can arouse some
>> interest in the topic.
>
> I don't have any direct insight into this issue other than to note
> that the filesytem itself may be an issue.

In some contexts, no doubt. But if we set aside exotica such as 
surrogate pairs, NTFS filenames are UTF-16 to a very good 
approximation. As such they are easily converted to UTF-8 to permit 
handling with good old C char * APIs, and easily converted back to 
UTF-16 for _wfopen() if required.

> The following entry from the R developer blog is of interest
>
>   https://developer.r-project.org/Blog/public/2020/05/02/utf-8-support-on-windows/
>
> I gather from the discussion there that Windows-10 can be made to
> support UTF-8 as a native encoding, calling it "extended ASCII".

Interesting, yes, but at this point kinda science fiction. The 
practical issue at present is whether gnuplot wants to support 
out-of-codepage UTF-16 filenames on the Windows command line. It's 
not terribly difficult, as I tried to show.

Sorry if I'm being repetitive, but right now if a create, say, a 
Russian-language filename on Windows and pass it as command-line 
argument to gnuplot, gnuplot will not be able to open the file 
because its name cannot be represented in my "system codepage". A 
program that reads the command line as UTF-16, however, will have no 
problem opening the file.

Allin Cottrell

Re: filenames on MS Windows

[Ethan A M. <me...@uw...>]

On Saturday, 31 October 2020 17:36:03 PST Allin Cottrell wrote:
> On Sun, 25 Oct 2020, Allin Cottrell wrote:
> 
> > I tried googling this but didn't find an answer -- sorry if I should have 
> > just tried harder! My question is: can gnuplot on Windows handle a unicode 
> > filename argument passed in UTF-16? As in
> >
> > path/to/wgnuplot.exe <UTF-16 input filename>
> 
> OK, that question was under-researched, but now I've done my 
> homework. Sorry, this is a bit long but I hope I can arouse some 
> interest in the topic.

I don't have any direct insight into this issue other than to note
that the filesytem itself may be an issue.  The standard Windows
filesystems impose an encoding on filenames, whereas linux filesystems
are agnostic to encoding; any null-terminated byte sequence not
containing '/' is a legal file name.  

The following entry from the R developer blog is of interest

   https://developer.r-project.org/Blog/public/2020/05/02/utf-8-support-on-windows/

I gather from the discussion there that Windows-10 can be made to
support UTF-8 as a native encoding, calling it "extended ASCII".
In that mode R (and I suppose gnuplot) can use the existing generic
linux code paths rather than multiple layers of text conversion.

	Ethan



> 
> Why bother with UTF-16 filename arguments? Nowadays a fair number of 
> Windows users construct paths (directory names or filenames) which 
> are "out of codepage" -- that is, unicode names which cannot be 
> represented in the (retro) "system codepage", which is typically 
> just an 8-bit encoding. Since Windows has supported unicode since NT 
> came out, it's a reasonable expectation that any filename one can 
> construct on the platform should be accessible via any program of 
> interest. But a program that restricts itself to the "ANSI" form of 
> filenames simply cannot access files with out-of-codepage paths. 
> (Sane modern OSes don't have this problem because they use UTF-8 
> throughout.)
> 
> So what about gnuplot? I may be wrong but it seems to me that 
> gnuplot on Windows is stuck with "ANSI" filenames at present. Even 
> with UNICODE and _UNICODE defined when compiling the program, the 
> command-line arguments are retrieved in winmain.c using either _argv 
> or __argv (depending on the compiler), and these get the ANSI-form 
> arguments (as opposed to __wargv which gets the arguments in UTF-16 
> form).
> 
> It would be easy to swap out __argv for __wargv but by itself this 
> would be very disruptive. The subsequent code in winmain.c, and then 
> the code in plot.c (gnu_main) to which the args array is passed, all 
> assumes the elements of argv are plain "char *", not "wide char" 
> arrays. Handling UTF-16, which is chock-full of NUL bytes, would 
> require lots of messy "ifdefs".
> 
> I have a proposal for fixing this. I realise it may not be 
> acceptable as it stands but maybe someone else might want to take it 
> up. I'm attaching patches for src/win/winmain.c and src/misc.c for 
> reference but here I'll try to explain the strategy.
> 
> 1) In winmain.c, grab the command-line arguments as UTF-16 but 
> immediately convert them to UTF-8, so they can handled by the 
> regular string.h APIs, both here and in plot.c (gnu_main).
> 
> 2) When we actually go to open a command-line file argument 
> (loadpath_fopen, in misc.c, called from gnu_main), we first try 
> opening the file using the filename as-is, but it that fails (and 
> the filename validates as UTF-8) we convert it to UTF-16 and try 
> again.
> 
> Since UTF-8 is a superset of ASCII, ASCII filename arguments should 
> pass through transparently. Within-codepage non-ASCII filenames 
> should get converted back to UTF-16 and opened OK. And the bonus is 
> that out-of-codepage arguments should also be converted and opened 
> OK.
> 
> I've tested this on Windows 10, with the system codepage set to 
> Windows 1252 ("Western Europe"), and have successfully opened files 
> with names in Russian and Greek. (I think this should also work if 
> the user has the system codepage set to UTF-8 (65001), which is a 
> "beta" option on Windows.)
> 
> My implementation uses GLib APIs (nice and simple) to convert from 
> UTF-16 to UTF-8 and back again (if needed). GLib is required anyway 
> if one is building the Cairo-based terminals. I suppose one could 
> use native Windows APIs to the same purpose but I suspect it would 
> be a lot more bother.
> 
> In my test setup this whole deal is triggered by the CFLAGS define
> 
> -DWIDE_ARGS
> 
> which is respected only when building for Windows -- and admittedly 
> has only been tested when cross-compiling for Windows from Linux 
> using Mingw-w64. In my mingw Makefile, I have:
> 
> WIDE_ARGS = 1
> 
> ...
> 
> ifdef WIDE_ARGS
>    CFLAGS += -DWIDE_ARGS
>    CFLAGS += $(shell pkg-config --cflags glib-2.0)
> endif
> 
> --
> Allin Cottrell
> Department of Economics
> Wake Forest University

Re: filenames on MS Windows

[Allin C. <cot...@wf...>]

On Sun, 25 Oct 2020, Allin Cottrell wrote:

> I tried googling this but didn't find an answer -- sorry if I should have 
> just tried harder! My question is: can gnuplot on Windows handle a unicode 
> filename argument passed in UTF-16? As in
>
> path/to/wgnuplot.exe <UTF-16 input filename>

OK, that question was under-researched, but now I've done my 
homework. Sorry, this is a bit long but I hope I can arouse some 
interest in the topic.

Why bother with UTF-16 filename arguments? Nowadays a fair number of 
Windows users construct paths (directory names or filenames) which 
are "out of codepage" -- that is, unicode names which cannot be 
represented in the (retro) "system codepage", which is typically 
just an 8-bit encoding. Since Windows has supported unicode since NT 
came out, it's a reasonable expectation that any filename one can 
construct on the platform should be accessible via any program of 
interest. But a program that restricts itself to the "ANSI" form of 
filenames simply cannot access files with out-of-codepage paths. 
(Sane modern OSes don't have this problem because they use UTF-8 
throughout.)

So what about gnuplot? I may be wrong but it seems to me that 
gnuplot on Windows is stuck with "ANSI" filenames at present. Even 
with UNICODE and _UNICODE defined when compiling the program, the 
command-line arguments are retrieved in winmain.c using either _argv 
or __argv (depending on the compiler), and these get the ANSI-form 
arguments (as opposed to __wargv which gets the arguments in UTF-16 
form).

It would be easy to swap out __argv for __wargv but by itself this 
would be very disruptive. The subsequent code in winmain.c, and then 
the code in plot.c (gnu_main) to which the args array is passed, all 
assumes the elements of argv are plain "char *", not "wide char" 
arrays. Handling UTF-16, which is chock-full of NUL bytes, would 
require lots of messy "ifdefs".

I have a proposal for fixing this. I realise it may not be 
acceptable as it stands but maybe someone else might want to take it 
up. I'm attaching patches for src/win/winmain.c and src/misc.c for 
reference but here I'll try to explain the strategy.

1) In winmain.c, grab the command-line arguments as UTF-16 but 
immediately convert them to UTF-8, so they can handled by the 
regular string.h APIs, both here and in plot.c (gnu_main).

2) When we actually go to open a command-line file argument 
(loadpath_fopen, in misc.c, called from gnu_main), we first try 
opening the file using the filename as-is, but it that fails (and 
the filename validates as UTF-8) we convert it to UTF-16 and try 
again.

Since UTF-8 is a superset of ASCII, ASCII filename arguments should 
pass through transparently. Within-codepage non-ASCII filenames 
should get converted back to UTF-16 and opened OK. And the bonus is 
that out-of-codepage arguments should also be converted and opened 
OK.

I've tested this on Windows 10, with the system codepage set to 
Windows 1252 ("Western Europe"), and have successfully opened files 
with names in Russian and Greek. (I think this should also work if 
the user has the system codepage set to UTF-8 (65001), which is a 
"beta" option on Windows.)

My implementation uses GLib APIs (nice and simple) to convert from 
UTF-16 to UTF-8 and back again (if needed). GLib is required anyway 
if one is building the Cairo-based terminals. I suppose one could 
use native Windows APIs to the same purpose but I suspect it would 
be a lot more bother.

In my test setup this whole deal is triggered by the CFLAGS define

-DWIDE_ARGS

which is respected only when building for Windows -- and admittedly 
has only been tested when cross-compiling for Windows from Linux 
using Mingw-w64. In my mingw Makefile, I have:

WIDE_ARGS = 1

...

ifdef WIDE_ARGS
   CFLAGS += -DWIDE_ARGS
   CFLAGS += $(shell pkg-config --cflags glib-2.0)
endif

--
Allin Cottrell
Department of Economics
Wake Forest University

Re: with pointcolor

[Ethan A M. <me...@uw...>]

On Thursday, 29 October 2020 13:26:38 PDT Dima Kogan wrote:
> Hi. I just discovered that there's no "pointcolor" property. So this
> doesn't work:
> 
>   plot x with points pt 6 ps 2 pointcolor "red"
> 
> The "right" way to do it is this:
> 
>   plot x with points pt 6 ps 2 linecolor "red"
> 
> Which is really surprising. And it means that when plotting with
> "linespoints" we can't use separate colors for the line and for the
> points. I'm not convinced that we do want to make separate line/point
> colors possible, but can "pointcolor" at the very least become an alias
> for "linecolor"?

There is nothing special about points.

Plot elements are drawn using the current pen, and the keyword
for specifying the color of the current pen is "linecolor".
Actually, prior to version 4.something the only options were
"linetype n" or "linestyle n".  The "linecolor" keyword was
added as a generic ad hoc specification that didn't require
pre-defined a linetype or linestyle.  I suppose it could have
been "pencolor" or "currentcolor" or even just "color", but
what we have is "linecolor".

So  plot ... with boxes linecolor "blue"    # not 'boxcolor'
    plot ... with impulses linecolor "blue" # not 'impulsecolor'
    plot ... with fsteps linecolor "blue"   # not "stepcolor'
    plot ... with points linecolor "blue"   # not 'pointcolor'


	Ethan


> 
> Is there some historical reason for this?
> 
> Thanks!
> dima

with pointcolor

[Dima K. <gn...@di...>]

Hi. I just discovered that there's no "pointcolor" property. So this
doesn't work:

  plot x with points pt 6 ps 2 pointcolor "red"

The "right" way to do it is this:

  plot x with points pt 6 ps 2 linecolor "red"

Which is really surprising. And it means that when plotting with
"linespoints" we can't use separate colors for the line and for the
points. I'm not convinced that we do want to make separate line/point
colors possible, but can "pointcolor" at the very least become an alias
for "linecolor"?

Is there some historical reason for this?

Thanks!
dima

filenames on MS Windows

[Allin C. <cot...@wf...>]

I tried googling this but didn't find an answer -- sorry if I should 
have just tried harder! My question is: can gnuplot on Windows 
handle a unicode filename argument passed in UTF-16? As in

path/to/wgnuplot.exe <UTF-16 input filename>

TIA.

-- 
Allin Cottrell
Department of Economics
Wake Forest University

merge for datablocks

[Daniel M. <mon...@ku...>]

Hi,

I just read about the patch for a "merge" command 
(https://sourceforge.net/p/gnuplot/patches/615/) that would allow 
merging the results of two plot commands into a single datablock that 
could then be used for a subsequent plot that needed both results. I 
don't think this was added to gnuplot. I tried "merge" in my current 
gnuplot version (5.2.8) and I get an "invalid command" reply.

It sounds it would be useful for combining data from different 
datablocks or files.
My use case for this would be creating a plot of the percentage of 
points in each interval that have a specific values from another column.

For example, for a file called "datafile.txt", containing values between 
0 and 1 in COL1 and a binary label in COL2:
COL1 COL2
0.1    0
0.2    1
0.1    1
0.33    0
0.4    1
1.0    0
0.9    0
0.12    1
0.13    1
0.13    0
0.89    1
0.88    0
....

A "merge" command would help to plot the following curve:

- percentage of points having COL1 between [0.0, 0.1] and COL2==1
- percentage of points have COL1 in the range [0.1,0.2] and COL2==1
- percentage of points have COL1 in the range [0.2,0.3] and COL2==1
and so on.

Currently, I am able to achieve this with "table" and "smooth frequency":
--------------------------------------------------------------------------------
# define function for binning data
binwidth = 0.1
bin(val) = binwidth * floor(val/binwidth)

# save total counts in each bin
set table "fileA"
plot "datafile.txt" using (bin(column("COL1"))):(1.0) smooth frequency 
with points

# save counts in each bin with a specific label
set table "fileB"
plot "datafile.txt" using (bin(column("COL1"))):(column("COL2") > 0 ? 
1.0 : 0.0) smooth frequency with points
unset table

# plot the percentage of dots with label "1" in each bin of width 0.1
plot "< paste fileA fileB" using 1:($5/$2) with linespoints
--------------------------------------------------------------------------------


Clearly this could be achieved pre-processing the data externally, but 
it looks like that the proposed "merge" command for datablocks would be 
a useful tool here.
It would simplify this without the need for writing data to files and 
reading it back.


Thank you,

__________________________________
Daniel Montezano
Center for Computational Biology
University of Kansas
Lawrence, KS 66045

Re: jitter documentation

[Allin C. <cot...@wf...>]

On Thu, 15 Oct 2020, Ethan A Merritt wrote:

> On Thursday, 15 October 2020 12:26:22 PDT Allin Cottrell wrote:
>> On Thu, 15 Oct 2020, Ethan A Merritt wrote:
>>
>>> On Thursday, 15 October 2020 08:02:27 PDT Allin Cottrell wrote:
>>>> Maybe I'm missing something, but isn't the documentation for
>>>> "jitter" backwards with respect to the swarm/square choice?
>>>>
>>>> The text reads thus: "The default jittering operation displaces
>>>> points only along x. This produces a distinctive pattern sometimes
>>>> called a "bee swarm plot". The optional keyword square adjusts the y
>>>> coordinate of displaced points in addition to their x coordinate so
>>>> that the points lie in distinct layers..."
>>>
>>> The text is correct as written.  Perhaps the attached figure,
>>> combining two plots from jitter.dem, will clarify.
>>>
>>> Left panel:
>>> 	original data, randomly distributed on y, all x values the same
>>> Center panel:
>>> 	"beeswarm" result from displacing points along x
>>> 	if they would otherwise overlap.
>>> 	Points are still randomly distributed on y.
>>> Right panel:
>>> 	"square" plot uses the x displacements that would
>>> 	generate a beeswarm plot, and adds an y displacement
>>> 	that is effectively a floor(y) operation where the unit of
>>> 	the floor operation is the "overlap" parameter to jitter.
>>>
>>>>
>>>> In the demo and in my own usage it seems the default is in fact to
>>>> displace in both the x and y dimensions while "square" limits the
>>>> scatter to the x dimension. (Except that in some cases I'm not
>>>> getting any y displacement with either choice, but that's another
>>>> issue.)
>>
>> Thanks, Ethan. I think I get it now. But this is potentially quite
>> confusing -- to understand exactly what jitter is doing one really
>> has to look closely at the y data in numerical form. So 'swarm'
>> preserves the original y values and just shoves points sideways to
>> get them off each other, while 'square' will also regularize the y
>> values to get the points into straight rows (more or less).
>
> Bee swarm plots were new to me.  I came across one in a paper I was
> reading and made a note to myself to look into it.
> The resulting gnuplot implementation was guided by what R does.
>
> A salient feature of bee swarm plots is that the jitter operation
> is reversible. If you consider any single point you can reconstruct
> its original [x,y] coordinates by projecting back onto the corresponding
> discrete x value.
>
> The "square" option loses this.  It is essentially a representation
> of binned data with a small number of points in each bin.  There is
> nothing to distinguish two points in the same bin from each other,
> as their original coordinates have been lost.
>
> As the number of points becomes large, both options are inferior
> to a violin plot.  The violinplot demo compares them and also
> shows a Gaussian jitter.
>
>> And then, given a pile-up of data points at some discrete {x,y}
>> value, there's no option to nudge them apart in both dimensions to
>> form a cloud?
>
> This sounds nice, but if x and y are both continuous I don't think
> it is a well-defined operation.  For a small-ish number of points you
> could define an energy function that has of a steep penalty gradient
> for overlap and then minimize the total energy by monte carlo.
> That rapidly becomes compute-intensive as the number of points
> increases.
>
> For x and y discrete there's a better option, sometimes called a
> bubble plot.  At each discrete [x,y], draw a circle with size
> proportional to the number of points that piled up there.
> The "size" is a sore point, however.  radius or area?
> I have not given much thought to whether that can be done with
> existing options in gnuplot or whether it would require new code
> or an external data processing stage.

Thanks again. All very clear.

It seems that what I was after in the doubly discrete case is really 
a bubble plot (and I'd vote for area!).

--
Allin Cottrell
Department of Economics
Wake Forest University

Re: jitter documentation

[Ethan A M. <me...@uw...>]

On Thursday, 15 October 2020 12:26:22 PDT Allin Cottrell wrote:
> On Thu, 15 Oct 2020, Ethan A Merritt wrote:
> 
> > On Thursday, 15 October 2020 08:02:27 PDT Allin Cottrell wrote:
> >> Maybe I'm missing something, but isn't the documentation for
> >> "jitter" backwards with respect to the swarm/square choice?
> >>
> >> The text reads thus: "The default jittering operation displaces
> >> points only along x. This produces a distinctive pattern sometimes
> >> called a "bee swarm plot". The optional keyword square adjusts the y
> >> coordinate of displaced points in addition to their x coordinate so
> >> that the points lie in distinct layers..."
> >
> > The text is correct as written.  Perhaps the attached figure,
> > combining two plots from jitter.dem, will clarify.
> >
> > Left panel:
> > 	original data, randomly distributed on y, all x values the same
> > Center panel:
> > 	"beeswarm" result from displacing points along x
> > 	if they would otherwise overlap.
> > 	Points are still randomly distributed on y.
> > Right panel:
> > 	"square" plot uses the x displacements that would
> > 	generate a beeswarm plot, and adds an y displacement
> > 	that is effectively a floor(y) operation where the unit of
> > 	the floor operation is the "overlap" parameter to jitter.
> >
> >>
> >> In the demo and in my own usage it seems the default is in fact to
> >> displace in both the x and y dimensions while "square" limits the
> >> scatter to the x dimension. (Except that in some cases I'm not
> >> getting any y displacement with either choice, but that's another
> >> issue.)
> 
> Thanks, Ethan. I think I get it now. But this is potentially quite 
> confusing -- to understand exactly what jitter is doing one really 
> has to look closely at the y data in numerical form. So 'swarm' 
> preserves the original y values and just shoves points sideways to 
> get them off each other, while 'square' will also regularize the y 
> values to get the points into straight rows (more or less).

Bee swarm plots were new to me.  I came across one in a paper I was
reading and made a note to myself to look into it.
The resulting gnuplot implementation was guided by what R does.

A salient feature of bee swarm plots is that the jitter operation
is reversible. If you consider any single point you can reconstruct
its original [x,y] coordinates by projecting back onto the corresponding
discrete x value.

The "square" option loses this.  It is essentially a representation
of binned data with a small number of points in each bin.  There is
nothing to distinguish two points in the same bin from each other,
as their original coordinates have been lost.

As the number of points becomes large, both options are inferior
to a violin plot.  The violinplot demo compares them and also
shows a Gaussian jitter.

> And then, given a pile-up of data points at some discrete {x,y}
> value, there's no option to nudge them apart in both dimensions to 
> form a cloud? 

This sounds nice, but if x and y are both continuous I don't think
it is a well-defined operation.  For a small-ish number of points you
could define an energy function that has of a steep penalty gradient
for overlap and then minimize the total energy by monte carlo.
That rapidly becomes compute-intensive as the number of points
increases.

For x and y discrete there's a better option, sometimes called a
bubble plot.  At each discrete [x,y], draw a circle with size
proportional to the number of points that piled up there.
The "size" is a sore point, however.  radius or area?
I have not given much thought to whether that can be done with
existing options in gnuplot or whether it would require new code
or an external data processing stage.

	cheers,
		Ethan

Re: jitter documentation

[Allin C. <cot...@wf...>]

On Thu, 15 Oct 2020, Ethan A Merritt wrote:

> On Thursday, 15 October 2020 08:02:27 PDT Allin Cottrell wrote:
>> Maybe I'm missing something, but isn't the documentation for
>> "jitter" backwards with respect to the swarm/square choice?
>>
>> The text reads thus: "The default jittering operation displaces
>> points only along x. This produces a distinctive pattern sometimes
>> called a "bee swarm plot". The optional keyword square adjusts the y
>> coordinate of displaced points in addition to their x coordinate so
>> that the points lie in distinct layers..."
>
> The text is correct as written.  Perhaps the attached figure,
> combining two plots from jitter.dem, will clarify.
>
> Left panel:
> 	original data, randomly distributed on y, all x values the same
> Center panel:
> 	"beeswarm" result from displacing points along x
> 	if they would otherwise overlap.
> 	Points are still randomly distributed on y.
> Right panel:
> 	"square" plot uses the x displacements that would
> 	generate a beeswarm plot, and adds an y displacement
> 	that is effectively a floor(y) operation where the unit of
> 	the floor operation is the "overlap" parameter to jitter.
>
>>
>> In the demo and in my own usage it seems the default is in fact to
>> displace in both the x and y dimensions while "square" limits the
>> scatter to the x dimension. (Except that in some cases I'm not
>> getting any y displacement with either choice, but that's another
>> issue.)

Thanks, Ethan. I think I get it now. But this is potentially quite 
confusing -- to understand exactly what jitter is doing one really 
has to look closely at the y data in numerical form. So 'swarm' 
preserves the original y values and just shoves points sideways to 
get them off each other, while 'square' will also regularize the y 
values to get the points into straight rows (more or less).

And then, given a pile-up of data points at some discrete {x,y}
value, there's no option to nudge them apart in both dimensions to 
form a cloud? I had the idea that was what "bee swarm" was about, 
but I guess I was mistaken.

--
Allin Cottrell
Department of Economics
Wake Forest University

Re: jitter documentation

[Ethan A M. <me...@uw...>]

On Thursday, 15 October 2020 08:02:27 PDT Allin Cottrell wrote:
> Maybe I'm missing something, but isn't the documentation for 
> "jitter" backwards with respect to the swarm/square choice?
> 
> The text reads thus: "The default jittering operation displaces 
> points only along x. This produces a distinctive pattern sometimes 
> called a "bee swarm plot". The optional keyword square adjusts the y 
> coordinate of displaced points in addition to their x coordinate so 
> that the points lie in distinct layers..."

The text is correct as written.  Perhaps the attached figure,
combining two plots from jitter.dem, will clarify.

Left panel:
	original data, randomly distributed on y, all x values the same
Center panel:
	"beeswarm" result from displacing points along x
	if they would otherwise overlap.
	Points are still randomly distributed on y.
Right panel:
	"square" plot uses the x displacements that would
	generate a beeswarm plot, and adds an y displacement
	that is effectively a floor(y) operation where the unit of
	the floor operation is the "overlap" parameter to jitter.

> 
> In the demo and in my own usage it seems the default is in fact to 
> displace in both the x and y dimensions while "square" limits the 
> scatter to the x dimension. (Except that in some cases I'm not 
> getting any y displacement with either choice, but that's another 
> issue.)

(figure attached)

	Ethan

bee swarm behavior

[Allin C. <cot...@wf...>]

I have a "jitter" test case with an integer-valued x variable. If I 
create a continuous y variable and plot with jitter, the swarm and 
square options give different plots (scatter in both dimensions with 
swarm, x dimension only with square). Fine (except it's the opposite 
of what "help jitter" says).

However, if I make the y variable integer-valued swarm and square 
produce the same plot, with displacement along x only. Is that 
intended? It seems contrary to the general intent. (I might mention: 
'vertical' produces y scatter OK in this case, but of course no x 
scatter.)

-- 
Allin Cottrell
Department of Economics
Wake Forest University

jitter documentation

[Allin C. <cot...@wf...>]

Maybe I'm missing something, but isn't the documentation for 
"jitter" backwards with respect to the swarm/square choice?

The text reads thus: "The default jittering operation displaces 
points only along x. This produces a distinctive pattern sometimes 
called a "bee swarm plot". The optional keyword square adjusts the y 
coordinate of displaced points in addition to their x coordinate so 
that the points lie in distinct layers..."

In the demo and in my own usage it seems the default is in fact to 
displace in both the x and y dimensions while "square" limits the 
scatter to the x dimension. (Except that in some cases I'm not 
getting any y displacement with either choice, but that's another 
issue.)

-- 
Allin Cottrell
Department of Economics
Wake Forest University

New body of code in development version: complex special functions

[Ethan A M. <me...@uw...>]

Since splitting off the stable branch for version 5.4 a year ago,
I have been working on new directions for development.
One of these is the expansion of support for complex functions
and in particular for complex special functions.
These are generally incomplete or altogether missing in other packages,
so it is an area where gnuplot can fill a gap.

This work is now merged into the main git repository.
Demos for some of the additions are in the on-line collection:

    http:/gnuplot.info/demo_5.5/

You can exercise them interactively in a new demo

    load 'special_functions.dem'

Here is a summary from the documentation.
The full manual contains additional information about the implementations
with references.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
This version of gnuplot provides a larger set of complex-valued functions
and updated versions of functions that were present in earlier versions. 

*  Updated incomplete gamma function with improved domain and precision. 
   Complex arguments accepted. 
*  Updated incomplete beta function with improved domain and precision. 
*  New function for the inverse incomplete gamma function. 
*  New function for the inverse incomplete beta function. 
*  New complex function LambertW(z,k) returns the kth branch of multivalued 
   function W_k(z). 
   Note that the older function lambertw(x) = real(LambertW( real(z), 0 )). 
*  New complex function lnGamma(z). 
   Note that existing function lgamma(x) = real(lnGamma(real(z)). 
*  Synchrotron function F(x). 
*  acosh(z) domain extended to cover negative real axis. 
*  asin(z) asinh(z) improved precision for complex arguments. 
*  Complex function conj(z) returns the complex conjugate of z. 
*  Predefined variable I = sqrt(-1) = {0,1} for convenience.

Additional special functions are supported if a suitable external
library is found at build time.
 *  Complex Bessel functions Iν(z), Jν(z), Kν(z), Yν(z) of order ν (real)
    with complex argument z.
 *  Complex Hankel functions H1ν(z), H2ν(z) of order ν with complex z.
 *  Complex Airy functions Ai(z), Bi(z).
 *  Complex exponential integral of order n.
 *  Fresnel integrals C(x) and S(x).

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%


There is room for further development.
Please add comments and suggestions!

- Existing built-in functions, old and new, are inconsistent about how
  they return very large values (floating overflow)
    Some return VERYLARGE (system-dependent but generally 9.e+307)
    Some return NaN
    Some return +Inf
  It would be nice to agree on a uniform convention and on how
  to handle it while plotting.

- Support for complex igamma(z) in the domain real(z) < 0 

- Support for complex elliptic integrals

- Modify "fit" to automatically handle complex-valued functions

- Support is dependent on external libraries found during configuration
  (libcerf libopenspecfun libamos).  Many linux distros have packages
  for libcerf and libopenspecfun.  I don't know how available they are
  for Windows or OSX. Much of the code in these libraries originally
  came from netlib and could be included in the gnuplot repository if
  we wanted to do so.

	Ethan

Re: 5.4.0: Mac OS X version available

[Ethan A M. <me...@uw...>]

On Wednesday, 23 September 2020 18:39:55 PDT Erik Luijten wrote:
> Dear Ethan et al.,
> 
> A Mac OS X package for version 5.4.0 is now available at
> https://csml-wiki.northwestern.edu/index.php/Binary_versions_of_Gnuplot_for_OS_X
> 
> If you encounter any problem, please let me know.
> 
> There is one small compilation issue that *seems* to be due to the way that
> src/Makefile is constructed. On the more recent versions of OS X, the X11
> libraries and header files are in /opt/X11, which ./configure does not
> detect. This is fine, since I can use the options '--x-include=/opt/X11/include
> --x-libraries=/opt/X11/lib' to specify this. However, the X11 library
> location does not get used in the compilation of gnuplot (only for
> gnuplot_x11, via the XLIBS variable), even though the main executable needs
> the Xpm library. Manually adding -L/opt/X11/lib to LDFLAGS in src/Makefile
> fixes this, but should not be necessary.
> 
> Regards,
> 
> Erik

Does the patch below fix it?

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
diff --git a/src/Makefile.am b/src/Makefile.am
index 0b261ea47..3a8b9b499 100644
--- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -69,6 +69,7 @@ pkglibexec_PROGRAMS += gnuplot_x11
 gnuplot_x11_SOURCES = gplt_x11.c gplt_x11.h gpexecute.c gpexecute.h mousecmn.h version.c ver
sion.h
 XLIBS = @LIBRARIES_FOR_X@
 gnuplot_x11_LDADD = getcolor_x11.o $(XLIBS)
+gnuplot_LDADD += $(XLIBS)
 endif
 
 getcolor_x11.o: getcolor.c
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

	Ethan

Re: [PATCH] cairolatex: respect transparency setting

[Ethan M. <eam...@gm...>]

On Thursday, 24 September 2020 11:30:11 PDT Henri Menke wrote:
> Dear Ehtan,
> 
> I just wanted to ask what is the status of this patch?  It doesn't
> change the default behaviour but makes the notransparent option actually
> do something.  If you want I can write sentence in the documentation
> about the layer ordering.

As I understand it, you want this not because of a problem in the gnuplot
latex output but rather because of a bug/misfeature/quirk of some downstream
program that fails to validate the pdf file produced by running pdflatex on
the gnuplot output.  Do I have that right?

The thing is, if a user selects the "notransparent" option with your patch
applied then the output from gnuplot is incorrect.   
It loses the "back" layer of text, including the default axis tic labels.
This can be partially worked around by 
     set tics front
     set grid front
but then the axis borders and tics in a 3D plot incorrectly occlude the 
plotted surface.  This has the paradoxical effect of making the
"notransparent" option cause a pm3d surface to look transparent!

Therefore I do not like the patch as it stands.  In order to produce
correct 3D output further changes to the program would be required
to separate the handling of border text and lines.

Would it not be better to document that for some purposes
it may be helpful to post-process the png image?

       mogrify -flatten myfigure-inc.png

I think that achieves exactly the same result (including the problematic 3D border)
at the cost of one extra step.  But I could be wrong; maybe your validation program
still doesn't like it eve after flattening?

]    cheers,

	Ethan


> 
> Cheers, Henri
> 
> On 21/08/20, 13:34, Henri Menke wrote:
> > Currently the cairolatex terminal does not respect transparency options
> > given in the terminal setting.  In this minimal example the
> > notransparent option is ignored:
> > 
> >     set terminal cairolatex png notransparent
> >     set output "test.tex"
> >     plot sin(x)
> > 
> > Running this through gnuplot and then using `file' reveals that the
> > alpha channel is still present:
> > 
> >     $ gnuplot test.gnuplot && file test.png
> >     test.png: PNG image data, 1500 x 900, 8-bit/color RGBA, non-interlace
> > 
> > With this patch the transparency setting will be honoured.  The current
> > default setting of implicitly assuming transparency is unaltered.
> > ---
> >  term/cairo.trm | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/term/cairo.trm b/term/cairo.trm
> > index ee748dfe2..3ef76ecc6 100644
> > --- a/term/cairo.trm
> > +++ b/term/cairo.trm
> > @@ -849,7 +849,7 @@ void cairotrm_graphics()
> >  	plot.background.g = cairo_params->background.g;
> >  	plot.background.b = cairo_params->background.b;
> >  	gp_cairo_set_background(cairo_params->background);
> > -	if (ISCAIROLATEX || cairo_params->transparent)
> > +	if (cairo_params->transparent)
> >  		gp_cairo_clear_background(&plot);
> >  	else
> >  		gp_cairo_solid_background(&plot);
> 
> 
> _______________________________________________
> gnuplot-beta mailing list
> gnu...@li...
> Membership management via: https://lists.sourceforge.net/lists/listinfo/gnuplot-beta
> 

Re: [PATCH] cairolatex: respect transparency setting

[Henri M. <hen...@gm...>]

Dear Ehtan,

I just wanted to ask what is the status of this patch?  It doesn't
change the default behaviour but makes the notransparent option actually
do something.  If you want I can write sentence in the documentation
about the layer ordering.

Cheers, Henri

On 21/08/20, 13:34, Henri Menke wrote:
> Currently the cairolatex terminal does not respect transparency options
> given in the terminal setting.  In this minimal example the
> notransparent option is ignored:
> 
>     set terminal cairolatex png notransparent
>     set output "test.tex"
>     plot sin(x)
> 
> Running this through gnuplot and then using `file' reveals that the
> alpha channel is still present:
> 
>     $ gnuplot test.gnuplot && file test.png
>     test.png: PNG image data, 1500 x 900, 8-bit/color RGBA, non-interlace
> 
> With this patch the transparency setting will be honoured.  The current
> default setting of implicitly assuming transparency is unaltered.
> ---
>  term/cairo.trm | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/term/cairo.trm b/term/cairo.trm
> index ee748dfe2..3ef76ecc6 100644
> --- a/term/cairo.trm
> +++ b/term/cairo.trm
> @@ -849,7 +849,7 @@ void cairotrm_graphics()
>  	plot.background.g = cairo_params->background.g;
>  	plot.background.b = cairo_params->background.b;
>  	gp_cairo_set_background(cairo_params->background);
> -	if (ISCAIROLATEX || cairo_params->transparent)
> +	if (cairo_params->transparent)
>  		gp_cairo_clear_background(&plot);
>  	else
>  		gp_cairo_solid_background(&plot);
> -- 
> 2.28.0
> 

5.4.0: Mac OS X version available

[Erik L. <eri...@gm...>]

Dear Ethan et al.,

A Mac OS X package for version 5.4.0 is now available at
https://csml-wiki.northwestern.edu/index.php/Binary_versions_of_Gnuplot_for_OS_X

If you encounter any problem, please let me know.

There is one small compilation issue that *seems* to be due to the way that
src/Makefile is constructed. On the more recent versions of OS X, the X11
libraries and header files are in /opt/X11, which ./configure does not
detect. This is fine, since I can use the options '--x-include=/opt/X11/include
--x-libraries=/opt/X11/lib' to specify this. However, the X11 library
location does not get used in the compilation of gnuplot (only for
gnuplot_x11, via the XLIBS variable), even though the main executable needs
the Xpm library. Manually adding -L/opt/X11/lib to LDFLAGS in src/Makefile
fixes this, but should not be necessary.

Regards,

Erik


On Thu, Jul 16, 2020 at 2:28 PM Ethan A Merritt <me...@uw...> wrote:

> Gnuplot version 5.4 - first release
>
> I have placed a source tarball for 5.4 on SourceForge
>
>
> https://sf.net/projects/gnuplot/files/gnuplot/5.4.0/gnuplot-5.4.0.tar.gz
>
> Release Notes with new feature list
>
>    http://gnuplot.info/ReleaseNotes_5_4.html
>
> Version 5.4.0 is the start of a new "major release" series.
> It supersedes version 5.2, first released in September 2017.
>
> I realize that there may be unresolved issues with building Windows
> binaries
> for Windows 10, but I decided that it is not reasonable to hold up release
> any
> longer for issues that appear to be more related to system configuration
> than to
> the gnuplot source.  If necessary we can put out a Windows-specific
> patchlevel 1.
> As always, I am totally reliant on and extremely grateful to the volunteers
> who put together Windows binaries from a source release package.
>
>         cheers, and happy plotting
>
>                 Ethan
>
>
>
>
>
>
>
>
> _______________________________________________
> gnuplot-beta mailing list
> gnu...@li...
> Membership management via:
> https://lists.sourceforge.net/lists/listinfo/gnuplot-beta
>

Re: New Release package - gnuplot version 5.4

[Karl-Friedrich R. <mai...@gm...>]

Many thanks!

I noticed the new feature "Help/Edit (w)gnuplot.ini" in the menu.
Next to the editor (works fine), it additionally opens a black text
window titled "gnuplot 5.4 patchlevel rc1" (?!?) with a blinking
cursor in it, but anything typed there lands in the regular gnuplot
shell window. Closing that window kills wgnuplot.exe.

   Karl


Am 19.09.2020 um 18:00 schrieb Bastian Märkisch:
> Dear all,
>
> You can now find Windows binaries of the 5.4.0 release for testing on SourceForge:
>
>   https://sf.net/projects/gnuplot/files/gnuplot/testing/
>
> Please give them a try.
>
>   Bastian

Aw: New Release package - gnuplot version 5.4

[Bastian M. <bma...@we...>]

Dear all,

You can now find Windows binaries of the 5.4.0 release for testing on SourceForge:

  https://sf.net/projects/gnuplot/files/gnuplot/testing/

Please give them a try.

  Bastian


> Gesendet: Donnerstag, 16. Juli 2020 um 21:25 Uhr
> Von: "Ethan A Merritt" <me...@uw...>
> An: gnu...@li...
> Betreff: New Release package - gnuplot version 5.4
>
> Gnuplot version 5.4 - first release
>
> I have placed a source tarball for 5.4 on SourceForge
>
>    https://sf.net/projects/gnuplot/files/gnuplot/5.4.0/gnuplot-5.4.0.tar.gz
>
> Release Notes with new feature list
>
>    http://gnuplot.info/ReleaseNotes_5_4.html
>
> Version 5.4.0 is the start of a new "major release" series.
> It supersedes version 5.2, first released in September 2017.
>
> I realize that there may be unresolved issues with building Windows binaries
> for Windows 10, but I decided that it is not reasonable to hold up release any
> longer for issues that appear to be more related to system configuration than to
> the gnuplot source.  If necessary we can put out a Windows-specific patchlevel 1.
> As always, I am totally reliant on and extremely grateful to the volunteers
> who put together Windows binaries from a source release package.
>
>         cheers, and happy plotting
>
>                 Ethan
>
>
>
>
>
>
>
>
> _______________________________________________
> gnuplot-beta mailing list
> gnu...@li...
> Membership management via: https://lists.sourceforge.net/lists/listinfo/gnuplot-beta
>

Re: Strange autoscaling with rgbimage and inverted yranges

[Ethan A M. <me...@uw...>]

On Sunday, 13 September 2020 13:17:59 PDT Dima Kogan wrote:
> Hi. I do observe your sequence work at you state:
> 
> > 1) start gnuplot
> > 2) load script
> > 3) zoom
> > 4) 'u'
> >    plot is now incorrectly scaled
> > 5) 'n'
> >    shows zoom state from (3)
> > 6) 'p'
> >    correctly unzooms to original plot
> 
> Step 6 may be 'u' or 'p'. The reason things don't look the same after
> steps 4 and 6 is the value of
> 
>   { axis_array[FIRST_Y_AXIS].min, axis_array[FIRST_Y_AXIS].max }
> 
> at the start of the refresh_bounds() call. This function assumes
> min<max. And if we're reversing some axis, this is done at the very end
> of refresh_bounds() with the axis_check_range() calls.
> 
> In step 3, refresh_bounds() sees {min,max} = {4000,-500}
> In step 6, refresh_bounds() sees {min,max} = {1850,1400}
> 
> This is dependent on the actual zoom bounds. Both of these have the
> unexpected min>max, obviously, but in step 6 we get lucky, and it ends
> up doing the right thing anyway, in this case.

OK.
But no autoscaling at all should be necessary during zoom or unzoom.
That is true even for nonvolatile data, but is doubly true for volatile data.
The whole point of zoom is that the bounds are being chosen
interactively by the mouse, not by the content of the plot.

My thought is to add a flag somewhere that zoom/unzoom is in progress,
and skip all autoscaling operations if it is set.  In fact there already is a flag
"inside_zoom" but it isn't used for much.  I have not yet inventoried all the
places in the code that would need to be changed.  

	Ethan

Re: Strange autoscaling with rgbimage and inverted yranges

[Dima K. <gn...@di...>]

Hi. I do observe your sequence work at you state:

> 1) start gnuplot
> 2) load script
> 3) zoom
> 4) 'u'
>    plot is now incorrectly scaled
> 5) 'n'
>    shows zoom state from (3)
> 6) 'p'
>    correctly unzooms to original plot

Step 6 may be 'u' or 'p'. The reason things don't look the same after
steps 4 and 6 is the value of

  { axis_array[FIRST_Y_AXIS].min, axis_array[FIRST_Y_AXIS].max }

at the start of the refresh_bounds() call. This function assumes
min<max. And if we're reversing some axis, this is done at the very end
of refresh_bounds() with the axis_check_range() calls.

In step 3, refresh_bounds() sees {min,max} = {4000,-500}
In step 6, refresh_bounds() sees {min,max} = {1850,1400}

This is dependent on the actual zoom bounds. Both of these have the
unexpected min>max, obviously, but in step 6 we get lucky, and it ends
up doing the right thing anyway, in this case.

Specifically the diverging behavior comes from

  refresh_bounds()          calls
  process_image()           calls
  STORE_AND_UPDATE_RANGE()  calls
  store_and_update_range()

The logic in store_and_update_range() is effectively:

    if ( curval > axis->max && curval >= axis->min )
      axis->max = curval;

Which is correct only if axis->min < axis->max

I'm not entirely sure what the values of axis->min, axis->max should be
at the start of refresh_bounds(), but reordering them makes this thing
work correctly. Patch (usable for this specific test case only):

  diff --git a/src/plot2d.c b/src/plot2d.c
  index b2877eb89..7e4e57259 100644
  --- a/src/plot2d.c
  +++ b/src/plot2d.c
  @@ -299,6 +299,17 @@ refresh_bounds(struct curve_points *first_plot, int nplots)
       struct curve_points *this_plot = first_plot;
       int iplot;		/* plot index */

  +
  +    if(axis_array[FIRST_Y_AXIS].min > axis_array[FIRST_Y_AXIS].max)
  +    {
  +        double t = axis_array[FIRST_Y_AXIS].min;
  +        axis_array[FIRST_Y_AXIS].min = axis_array[FIRST_Y_AXIS].max;
  +        axis_array[FIRST_Y_AXIS].max = t;
  +    }
  +
  +
  +
  +
       for (iplot = 0;  iplot < nplots; iplot++, this_plot = this_plot->next) {
          int i;		/* point index */
          struct axis *x_axis = &axis_array[this_plot->x_axis];

Applying this patch makes the test case work, even before any of your
patches from yesterday. Clearly, this should be more general, but I'm
hazy on the details, so it'd be better if you finish this, probably.

Thanks!