gnuplot-beta — Developers and beta testers of new gnuplot code

First release candidate for gnuplot 6

[Ethan A M. <me...@uw...>]

The "testing" directory on SourceForge now has a tarball for gnuplot-6-0-rc1

   https://sourceforge.net/projects/gnuplot/files/gnuplot/testing/gnuplot-6.0.rc1.tar.gz

This is a first release candidate for version 6.0.

Nine years ago we went through three release candidates and six months of
testing before releasing 5.0.  I expect that finalizing version 6.0 will
go faster, but we will see.   Compared to the v4->v5 transition,
I think v5->v6 introduces a larger number of new features but fewer
changes that may affect existing scripts or user expectations.



		Release Notes
		=============

Version 6.0 is the start of a new stable release series.
Work on gnuplot 6 has proceeded in parallel with the incremental updates to
version 5.4 over the past three years.  The new version introduces extensions
to the gnuplot command language, support for new output protocols, and
additional plotting styles.  Backwards compatibility is given high priority
in gnuplot development, so users should find no significant changes required
for techniques or code they are currently using with gnuplot 5.

Your feedback from testing of this release candidate will contribute to
the success of a full 6.0 release expected later in 2023.

	Features introduced in version 6
	================================

For a detailed list of new features, with illustrations, see
  http://gnuplot.info/docs_6.0/NewFeatures.html

For more example plots see
  http://gnuplot.info/demo_6.0/

- Function blocks and scoped variables
- Larger collection of special and complex-valued functions
- New plot styles
        o 2D plot style `with surface` works in 2D polar coordinates to produce a
          solid-fill gridded representation of the plane. This is analogous to the
          use of dgrid3d and pm3d to produce a 3D gridded surface.
        o 2D plot style `with sectors` renders one annular segment ("sector") for
          each line of input data. This style can generate pie and donut charts,
          windrose charts, and a polar equivalent to sparse-matrix heatmaps.
        o 2D plot style `with lines` now has a filter option `sharpen`.
          This filter detects spikes in a function plot that would be missed or
          under-represented due to coarse sampling.  It adds an additional
          sampling point at the location of each such peak.
- Hulls, masks, and smoothing
        o A cluster of 2D points can be replaced by a bounding polygon ("hull").
          Both convex hulls and concave hulls (χ-shapes) are supported.
        o Any hull or other closed path can be used as a mask to display only
          selected regions of a pm3d surface or image plot.
        o New smoothing option "smooth path" can be used on 2D and 3D curves
          that are not monotonic on x or y.  This allows smoothing of hulls.
- Named palettes
        o The current palette can be saved to a named colormap for future us.
        o A predefined palette named "viridis" is provided.
        o Plots can specify a previously saved palette by name.
          This permits the use of multiple palettes in a single plot command.
        o Named palettes can be edited to contain an alpha channel.
- New built-in functions and array operations
        o palette(z) returns the current RGB palette color mapping for z.
        o rgbcolor("name") returns the 32bit ARGB value for a named color.
        o index(Array, element) returns the first index i for which Array[i]
          is equal to element.
        o split("string", "separator") unpacks the fields in a string into
          an array of strings.
        o join(array, "separator") is the complement to split().
          It concatenates the elements of a string array into a single string.
        o `stats <non-existent file>` yields a testable value with no error;
          useful to avoid errors or warnings in scripts.
- Program control flow
        o New syntax if {...} else if {...} else {...}
        o XDG base directory conventions for configuration files are supported.
        o `unset warnings` suppresses output of warning messages to the console.
        o The `fit` command is protected by exception handling. Control always
          returns to the next line of input even in the case of fit errors.
          On return FIT_ERROR is non-zero if an error occurred.
        o "Watchpoints"  are target values associated with individual plots
          in a graph. As that plot is drawn, each component line segment is
          monitored to see if its endpoints bracket the target value of a
          watchpoint coordinate (x, y, or z) or function f(x,y).
          If a match is found, the [x,y] coordinates of the match point are
          saved for later use.  Possible uses include
                - find the intersection points of two curves
                - find zeros of a function
                - find and notate where a dependent variable or function f(x,y)
                  crosses a threshold value
                - use the mouse to track values along multiple plots simultaneously
- New terminals and terminal options
        o Terminals that display graphics in the same window as text entry now
          support pseudo-mousing; i.e. they respond to arrow keys and other
          hot-key bindings during "pause mouse".
        o New terminals kittygd and kittycairo provide in-window graphics for
          terminal emulators that support the kitty protocol.
        o New terminal webp generates a single frame or an animation sequence
          using webp encoding. Frames are generated using pngcairo,
          then encoded through the WebPAnimEncoder API.
        o New terminal block for text-mode pseudo-graphics uses Unicode block
          or Braille characters to offer improved resolution compared to the
          dumb or caca terminals.
        o latex terminals standalone mode updated to work with texlive2023
- Miscellaneous other new features
        o Time unit settings for major and minor axis tics.  For example,
          minor tic marks can be placed at exactly one month intervals.
        o The character sequence $# in a using specifier evaluates to the total
          number of columns available in the current line of data.
          "plot FOO using 0:(column($# - 1))" plots the last-but-one field of each row.
        o keyword binvalue=avg plots the average, rather than the sum, of binned data.
        o "set colorbox bottom" places the color box underneath the plot.
        o "set pm3d spotlight" adds a user-controlled spotlight to the lighting model.
        o New key layout options to force specific width or number of columns.
          Automatic positioning of the key on the page can be manually tweaked
          by giving an offset.
        o "set isotropic" adjusts the axis scaling in both 2D and 3D plots such
          that x, y, and z axes all have the same scale.
        o Text rotation angles are not limited to integral degree values.
        o Data-driven color assignments in plot style "histograms".


    happy gnuplotting

	Ethan

Re: SVG pixelation (was pdfcairo terminal seams between bars in the colorbox)

[Ethan A M. <me...@uw...>]

On Thursday, 13 July 2023 09:26:25 PDT Dima Kogan wrote:
> Dima Kogan <gn...@di...> writes:
> 
> > Most non-browser applications I've seen use rsvg: bug report here
> >
> >   https://urldefense.com/v3/__https://gitlab.gnome.org/GNOME/librsvg/-/issues/985__;!!K-Hz7m0Vt54!k39HeEQ9zt5kuFFtV54jBNfoBJvfqILRmzHarDU6bxyPm-2JXE1ruwmzZkCnARQ0hIggFA3zaJa59Jm0EojHJg$ 
> 
> Just an FYI. The rsvg upstream has fixed the issue, and closed the bug
> report (allegedly; I couldn't easily try it). But if it's fixed, then
> most non-browser client applications will show our pixelated images.

Except, apparently, on MacOS.
See discussion on 
   https://sourceforge.net/p/gnuplot/feature-requests/558/
   https://www.userlandia.com/home/2021/11/the-mystery-of-mac-os-mangled-image-interpolation-implementation

I'm in the process of combining suggestions from
Hiroki Motoyoshi (problems with MacOS)
and Alexander Täschner (problems with MSVC)
to hopefully end up with something that works for everyone.

    thanks

	Ethan

Re: SVG pixelation (was pdfcairo terminal seams between bars in the colorbox)

[Dima K. <gn...@di...>]

Dima Kogan <gn...@di...> writes:

> Most non-browser applications I've seen use rsvg: bug report here
>
>   https://gitlab.gnome.org/GNOME/librsvg/-/issues/985

Just an FYI. The rsvg upstream has fixed the issue, and closed the bug
report (allegedly; I couldn't easily try it). But if it's fixed, then
most non-browser client applications will show our pixelated images.

Re: strange multiplot behavior with wxt

[Allin C. <cot...@wf...>]

On Thu, 29 Jun 2023, Allin Cottrell wrote:

> On Wed, 28 Jun 2023, Ethan A Merritt wrote:
>
>> Valgrind finds a test of an uninitialized flag "hidden" that could
>> plausibly be involved here.
>> I have corrected that by setting the flag to FALSE in the one place
>> that I could see initialization omitted, but I do not actually
>> understand why that code path is triggered by this test case (it is
>> the response to a "refresh" command).  That makes valgrind happy, so
>> maybe it will fix your display also.  I'll leave the tracker issue
>> "open" for now.
>
> Thanks, Ethan! I've built gnuplot 6.1.0 from gnuplot-main git and tested on 
> Fedora: it works fine. I'll also try on Arch with more recent wx, but it 
> looks like you've nailed the problem.

I can confirm that the fix also works for wx 3.2.2.

Allin Cottrell

Re: strange multiplot behavior with wxt

[Allin C. <cot...@wf...>]

On Wed, 28 Jun 2023, Ethan A Merritt wrote:

> Valgrind finds a test of an uninitialized flag "hidden" that could
> plausibly be involved here.
> I have corrected that by setting the flag to FALSE in the one place
> that I could see initialization omitted, but I do not actually
> understand why that code path is triggered by this test case (it is
> the response to a "refresh" command).  That makes valgrind happy, so
> maybe it will fix your display also.  I'll leave the tracker issue
> "open" for now.

Thanks, Ethan! I've built gnuplot 6.1.0 from gnuplot-main git and 
tested on Fedora: it works fine. I'll also try on Arch with more 
recent wx, but it looks like you've nailed the problem.

Just wondering: are you likely to cherry-pick this fix into 
branch-5-4-stable?

Allin Cottrell

>
> On Wed, Jun 28, 2023 at 10:31 AM Allin Cottrell <cot...@wf...> wrote:
>>
>> On Wed, 28 Jun 2023, Ethan A Merritt wrote:
>>
>>> On Wednesday, 28 June 2023 05:50:15 PDT Allin Cottrell wrote:
>>>> I'm puzzled by a multiplot example where the output seems to be
>>>> non-deterministic -- though the fault may be in my script
>>>> (strange_multi.plt, attached).
>>>>
>>>> I'm on Linux, using gnuplot 5.4.3 and also 5.4.8, using
>>>> the wxt terminal for screen display with the command line
>>>>
>>>> gnuplot strange_multi.plt -persist
>>>
>>> I have kept source and executables around for many versions of gnuplot.
>>> I can reproduce your "missing bars in first plot" symptom with old
>>> executables for 5.4.0 through 5.4.6, but not with 5.4.8 or the
>>> development branch.  Any given executable seems to be 100% reproducible
>>> in either failing or succeeding.
>>>
>>> But here's the thing - if I rebuild 5.4.3 from the same original
>>> source, I get a new executable that does not exhibit the problem.
>>> So my suspicion is that the difference is the version of the
>>> wx- shared libraries the binary is linked against.
>>>
>>> old gnuplot_5.4.3 binary (built May 2022)
>>> [~/temp] ldd /usr/local/bin/gnuplot_5.4.3 | grep wx
>>>        libwx_gtk3u_xrc-3.1.so.5 => /lib64/libwx_gtk3u_xrc-3.1.so.5 (0x00007fc634b09000)
>>>        libwx_gtk3u_html-3.1.so.5 => /lib64/libwx_gtk3u_html-3.1.so.5 (0x00007fc634a25000)
>>>        libwx_gtk3u_qa-3.1.so.5 => /lib64/libwx_gtk3u_qa-3.1.so.5 (0x00007fc6349f6000)
>>>        libwx_gtk3u_core-3.1.so.5 => /lib64/libwx_gtk3u_core-3.1.so.5 (0x00007fc634000000)
>>>        libwx_baseu_xml-3.1.so.5 => /lib64/libwx_baseu_xml-3.1.so.5 (0x00007fc6349e2000)
>>>        libwx_baseu_net-3.1.so.5 => /lib64/libwx_baseu_net-3.1.so.5 (0x00007fc63497f000)
>>>        libwx_baseu-3.1.so.5 => /lib64/libwx_baseu-3.1.so.5 (0x00007fc633c00000)
>>>
>>> new gnuplot_5.4.3 binary (built today)
>>> [~/temp] ldd ~/cvs/gnuplot-5.4.3/src/gnuplot | grep wx
>>>        libwx_gtk3u_xrc-3.2.so.0 => /lib64/libwx_gtk3u_xrc-3.2.so.0 (0x00007f3cedc64000)
>>>        libwx_gtk3u_html-3.2.so.0 => /lib64/libwx_gtk3u_html-3.2.so.0 (0x00007f3cedb83000)
>>>        libwx_gtk3u_qa-3.2.so.0 => /lib64/libwx_gtk3u_qa-3.2.so.0 (0x00007f3cedb54000)
>>>        libwx_gtk3u_core-3.2.so.0 => /lib64/libwx_gtk3u_core-3.2.so.0 (0x00007f3ced200000)
>>>        libwx_baseu_xml-3.2.so.0 => /lib64/libwx_baseu_xml-3.2.so.0 (0x00007f3cedb42000)
>>>        libwx_baseu_net-3.2.so.0 => /lib64/libwx_baseu_net-3.2.so.0 (0x00007f3cedade000)
>>>        libwx_baseu-3.2.so.0 => /lib64/libwx_baseu-3.2.so.0 (0x00007f3cece00000)
>>>
>>> But it's only a suspicion. [...]
>>
>> I have a little more information to contribute. I'm testing my
>> example on two machines. The one I'm sitting at runs Fedora 36,
>> with gnuplot 5.4.3 and wx 3.0.5 (per wx-config). The linkage is
>>
>> myrtle:~$ ldd /usr/bin/gnuplot | grep wx
>>          libwx_gtk3u_core-3.0.so.0 => /lib64/libwx_gtk3u_core-3.0.so.0 (0x00007fa5bfc00000)
>>          libwx_baseu-3.0.so.0 => /lib64/libwx_baseu-3.0.so.0 (0x00007fa5bf800000)
>>
>> On this machine I'm seeing the non-deterministic behavior.
>>
>> The other machine I'm accessing remotely via ssh. It runs current
>> Arch Linux with gnuplot 5.4.8 and wx 3.2.2, linkage:
>>
>> robroy:~$ ldd /usr/bin/gnuplot | grep wx
>>          libwx_gtk3u_core-3.2.so.0 => /usr/lib/libwx_gtk3u_core-3.2.so.0 (0x00007f5447a00000)
>>          libwx_baseu-3.2.so.0 => /usr/lib/libwx_baseu-3.2.so.0 (0x00007f5447600000)
>>
>> On this more up-to-date system I now think I was wrong to say the
>> behavior is non-deterministic. The plot appears somewhat slowly over
>> the remote connection, and here's the reproducible sequence I'm
>> seeing over 2 or 3 seconds:
>>
>> * an empty (black) window appears
>> * the canvas goes to white
>> * the first subplot appears, correctly
>> * the second subplot appears, correctly
>> * the data portion of the first subplot is wiped out to white
>>
>> Might this help with diagnosis?
>>
>> Allin Cottrell

Re: strange multiplot behavior with wxt

[Ethan A M. <me...@uw...>]

Valgrind finds a test of an uninitialized flag "hidden" that could
plausibly be involved here.
I have corrected that by setting the flag to FALSE in the one place
that I could see initialization omitted, but I do not actually
understand why that code path is triggered by this test case (it is
the response to a "refresh" command).  That makes valgrind happy, so
maybe it will fix your display also.  I'll leave the tracker issue
"open" for now.


On Wed, Jun 28, 2023 at 10:31 AM Allin Cottrell <cot...@wf...> wrote:
>
> On Wed, 28 Jun 2023, Ethan A Merritt wrote:
>
> > On Wednesday, 28 June 2023 05:50:15 PDT Allin Cottrell wrote:
> >> I'm puzzled by a multiplot example where the output seems to be
> >> non-deterministic -- though the fault may be in my script
> >> (strange_multi.plt, attached).
> >>
> >> I'm on Linux, using gnuplot 5.4.3 and also 5.4.8, using
> >> the wxt terminal for screen display with the command line
> >>
> >> gnuplot strange_multi.plt -persist
> >
> > I have kept source and executables around for many versions of gnuplot.
> > I can reproduce your "missing bars in first plot" symptom with old
> > executables for 5.4.0 through 5.4.6, but not with 5.4.8 or the
> > development branch.  Any given executable seems to be 100% reproducible
> > in either failing or succeeding.
> >
> > But here's the thing - if I rebuild 5.4.3 from the same original
> > source, I get a new executable that does not exhibit the problem.
> > So my suspicion is that the difference is the version of the
> > wx- shared libraries the binary is linked against.
> >
> > old gnuplot_5.4.3 binary (built May 2022)
> > [~/temp] ldd /usr/local/bin/gnuplot_5.4.3 | grep wx
> >        libwx_gtk3u_xrc-3.1.so.5 => /lib64/libwx_gtk3u_xrc-3.1.so.5 (0x00007fc634b09000)
> >        libwx_gtk3u_html-3.1.so.5 => /lib64/libwx_gtk3u_html-3.1.so.5 (0x00007fc634a25000)
> >        libwx_gtk3u_qa-3.1.so.5 => /lib64/libwx_gtk3u_qa-3.1.so.5 (0x00007fc6349f6000)
> >        libwx_gtk3u_core-3.1.so.5 => /lib64/libwx_gtk3u_core-3.1.so.5 (0x00007fc634000000)
> >        libwx_baseu_xml-3.1.so.5 => /lib64/libwx_baseu_xml-3.1.so.5 (0x00007fc6349e2000)
> >        libwx_baseu_net-3.1.so.5 => /lib64/libwx_baseu_net-3.1.so.5 (0x00007fc63497f000)
> >        libwx_baseu-3.1.so.5 => /lib64/libwx_baseu-3.1.so.5 (0x00007fc633c00000)
> >
> > new gnuplot_5.4.3 binary (built today)
> > [~/temp] ldd ~/cvs/gnuplot-5.4.3/src/gnuplot | grep wx
> >        libwx_gtk3u_xrc-3.2.so.0 => /lib64/libwx_gtk3u_xrc-3.2.so.0 (0x00007f3cedc64000)
> >        libwx_gtk3u_html-3.2.so.0 => /lib64/libwx_gtk3u_html-3.2.so.0 (0x00007f3cedb83000)
> >        libwx_gtk3u_qa-3.2.so.0 => /lib64/libwx_gtk3u_qa-3.2.so.0 (0x00007f3cedb54000)
> >        libwx_gtk3u_core-3.2.so.0 => /lib64/libwx_gtk3u_core-3.2.so.0 (0x00007f3ced200000)
> >        libwx_baseu_xml-3.2.so.0 => /lib64/libwx_baseu_xml-3.2.so.0 (0x00007f3cedb42000)
> >        libwx_baseu_net-3.2.so.0 => /lib64/libwx_baseu_net-3.2.so.0 (0x00007f3cedade000)
> >        libwx_baseu-3.2.so.0 => /lib64/libwx_baseu-3.2.so.0 (0x00007f3cece00000)
> >
> > But it's only a suspicion. [...]
>
> I have a little more information to contribute. I'm testing my
> example on two machines. The one I'm sitting at runs Fedora 36,
> with gnuplot 5.4.3 and wx 3.0.5 (per wx-config). The linkage is
>
> myrtle:~$ ldd /usr/bin/gnuplot | grep wx
>          libwx_gtk3u_core-3.0.so.0 => /lib64/libwx_gtk3u_core-3.0.so.0 (0x00007fa5bfc00000)
>          libwx_baseu-3.0.so.0 => /lib64/libwx_baseu-3.0.so.0 (0x00007fa5bf800000)
>
> On this machine I'm seeing the non-deterministic behavior.
>
> The other machine I'm accessing remotely via ssh. It runs current
> Arch Linux with gnuplot 5.4.8 and wx 3.2.2, linkage:
>
> robroy:~$ ldd /usr/bin/gnuplot | grep wx
>          libwx_gtk3u_core-3.2.so.0 => /usr/lib/libwx_gtk3u_core-3.2.so.0 (0x00007f5447a00000)
>          libwx_baseu-3.2.so.0 => /usr/lib/libwx_baseu-3.2.so.0 (0x00007f5447600000)
>
> On this more up-to-date system I now think I was wrong to say the
> behavior is non-deterministic. The plot appears somewhat slowly over
> the remote connection, and here's the reproducible sequence I'm
> seeing over 2 or 3 seconds:
>
> * an empty (black) window appears
> * the canvas goes to white
> * the first subplot appears, correctly
> * the second subplot appears, correctly
> * the data portion of the first subplot is wiped out to white
>
> Might this help with diagnosis?
>
> Allin Cottrell



-- 
Ethan A Merritt
Department of Biochemistry
University of Washington, Seattle

Re: strange multiplot behavior with wxt

[Allin C. <cot...@wf...>]

On Wed, 28 Jun 2023, Ethan A Merritt wrote:

> On Wednesday, 28 June 2023 05:50:15 PDT Allin Cottrell wrote:
>> I'm puzzled by a multiplot example where the output seems to be
>> non-deterministic -- though the fault may be in my script
>> (strange_multi.plt, attached).
>>
>> I'm on Linux, using gnuplot 5.4.3 and also 5.4.8, using
>> the wxt terminal for screen display with the command line
>>
>> gnuplot strange_multi.plt -persist
>
> I have kept source and executables around for many versions of gnuplot.
> I can reproduce your "missing bars in first plot" symptom with old
> executables for 5.4.0 through 5.4.6, but not with 5.4.8 or the
> development branch.  Any given executable seems to be 100% reproducible
> in either failing or succeeding.
>
> But here's the thing - if I rebuild 5.4.3 from the same original
> source, I get a new executable that does not exhibit the problem.
> So my suspicion is that the difference is the version of the
> wx- shared libraries the binary is linked against.
>
> old gnuplot_5.4.3 binary (built May 2022)
> [~/temp] ldd /usr/local/bin/gnuplot_5.4.3 | grep wx
>        libwx_gtk3u_xrc-3.1.so.5 => /lib64/libwx_gtk3u_xrc-3.1.so.5 (0x00007fc634b09000)
>        libwx_gtk3u_html-3.1.so.5 => /lib64/libwx_gtk3u_html-3.1.so.5 (0x00007fc634a25000)
>        libwx_gtk3u_qa-3.1.so.5 => /lib64/libwx_gtk3u_qa-3.1.so.5 (0x00007fc6349f6000)
>        libwx_gtk3u_core-3.1.so.5 => /lib64/libwx_gtk3u_core-3.1.so.5 (0x00007fc634000000)
>        libwx_baseu_xml-3.1.so.5 => /lib64/libwx_baseu_xml-3.1.so.5 (0x00007fc6349e2000)
>        libwx_baseu_net-3.1.so.5 => /lib64/libwx_baseu_net-3.1.so.5 (0x00007fc63497f000)
>        libwx_baseu-3.1.so.5 => /lib64/libwx_baseu-3.1.so.5 (0x00007fc633c00000)
>
> new gnuplot_5.4.3 binary (built today)
> [~/temp] ldd ~/cvs/gnuplot-5.4.3/src/gnuplot | grep wx
>        libwx_gtk3u_xrc-3.2.so.0 => /lib64/libwx_gtk3u_xrc-3.2.so.0 (0x00007f3cedc64000)
>        libwx_gtk3u_html-3.2.so.0 => /lib64/libwx_gtk3u_html-3.2.so.0 (0x00007f3cedb83000)
>        libwx_gtk3u_qa-3.2.so.0 => /lib64/libwx_gtk3u_qa-3.2.so.0 (0x00007f3cedb54000)
>        libwx_gtk3u_core-3.2.so.0 => /lib64/libwx_gtk3u_core-3.2.so.0 (0x00007f3ced200000)
>        libwx_baseu_xml-3.2.so.0 => /lib64/libwx_baseu_xml-3.2.so.0 (0x00007f3cedb42000)
>        libwx_baseu_net-3.2.so.0 => /lib64/libwx_baseu_net-3.2.so.0 (0x00007f3cedade000)
>        libwx_baseu-3.2.so.0 => /lib64/libwx_baseu-3.2.so.0 (0x00007f3cece00000)
>
> But it's only a suspicion. [...]

I have a little more information to contribute. I'm testing my 
example on two machines. The one I'm sitting at runs Fedora 36,
with gnuplot 5.4.3 and wx 3.0.5 (per wx-config). The linkage is

myrtle:~$ ldd /usr/bin/gnuplot | grep wx
         libwx_gtk3u_core-3.0.so.0 => /lib64/libwx_gtk3u_core-3.0.so.0 (0x00007fa5bfc00000)
         libwx_baseu-3.0.so.0 => /lib64/libwx_baseu-3.0.so.0 (0x00007fa5bf800000)

On this machine I'm seeing the non-deterministic behavior.

The other machine I'm accessing remotely via ssh. It runs current 
Arch Linux with gnuplot 5.4.8 and wx 3.2.2, linkage:

robroy:~$ ldd /usr/bin/gnuplot | grep wx
         libwx_gtk3u_core-3.2.so.0 => /usr/lib/libwx_gtk3u_core-3.2.so.0 (0x00007f5447a00000)
         libwx_baseu-3.2.so.0 => /usr/lib/libwx_baseu-3.2.so.0 (0x00007f5447600000)

On this more up-to-date system I now think I was wrong to say the 
behavior is non-deterministic. The plot appears somewhat slowly over 
the remote connection, and here's the reproducible sequence I'm 
seeing over 2 or 3 seconds:

* an empty (black) window appears
* the canvas goes to white
* the first subplot appears, correctly
* the second subplot appears, correctly
* the data portion of the first subplot is wiped out to white

Might this help with diagnosis?

Allin Cottrell

Re: strange multiplot behavior with wxt

[Ethan A M. <me...@uw...>]

On Wednesday, 28 June 2023 05:50:15 PDT Allin Cottrell wrote:
> I'm puzzled by a multiplot example where the output seems to be 
> non-deterministic -- though the fault may be in my script 
> (strange_multi.plt, attached).
> 
> I'm on Linux, using gnuplot 5.4.3 and also 5.4.8, using 
> the wxt terminal for screen display with the command line
> 
> gnuplot strange_multi.plt -persist

I have kept source and executables around for many versions of gnuplot.
I can reproduce your "missing bars in first plot" symptom with old
executables for 5.4.0 through 5.4.6, but not with 5.4.8 or the
development branch.  Any given executable seems to be 100% reproducible
in either failing or succeeding.

But here's the thing - if I rebuild 5.4.3 from the same original
source, I get a new executable that does not exhibit the problem.
So my suspicion is that the difference is the version of the 
wx- shared libraries the binary is linked against.

old gnuplot_5.4.3 binary (built May 2022)
 [~/temp] ldd /usr/local/bin/gnuplot_5.4.3 | grep wx
        libwx_gtk3u_xrc-3.1.so.5 => /lib64/libwx_gtk3u_xrc-3.1.so.5 (0x00007fc634b09000)
        libwx_gtk3u_html-3.1.so.5 => /lib64/libwx_gtk3u_html-3.1.so.5 (0x00007fc634a25000)
        libwx_gtk3u_qa-3.1.so.5 => /lib64/libwx_gtk3u_qa-3.1.so.5 (0x00007fc6349f6000)
        libwx_gtk3u_core-3.1.so.5 => /lib64/libwx_gtk3u_core-3.1.so.5 (0x00007fc634000000)
        libwx_baseu_xml-3.1.so.5 => /lib64/libwx_baseu_xml-3.1.so.5 (0x00007fc6349e2000)
        libwx_baseu_net-3.1.so.5 => /lib64/libwx_baseu_net-3.1.so.5 (0x00007fc63497f000)
        libwx_baseu-3.1.so.5 => /lib64/libwx_baseu-3.1.so.5 (0x00007fc633c00000)

new gnuplot_5.4.3 binary (built today)
[~/temp] ldd ~/cvs/gnuplot-5.4.3/src/gnuplot | grep wx
        libwx_gtk3u_xrc-3.2.so.0 => /lib64/libwx_gtk3u_xrc-3.2.so.0 (0x00007f3cedc64000)
        libwx_gtk3u_html-3.2.so.0 => /lib64/libwx_gtk3u_html-3.2.so.0 (0x00007f3cedb83000)
        libwx_gtk3u_qa-3.2.so.0 => /lib64/libwx_gtk3u_qa-3.2.so.0 (0x00007f3cedb54000)
        libwx_gtk3u_core-3.2.so.0 => /lib64/libwx_gtk3u_core-3.2.so.0 (0x00007f3ced200000)
        libwx_baseu_xml-3.2.so.0 => /lib64/libwx_baseu_xml-3.2.so.0 (0x00007f3cedb42000)
        libwx_baseu_net-3.2.so.0 => /lib64/libwx_baseu_net-3.2.so.0 (0x00007f3cedade000)
        libwx_baseu-3.2.so.0 => /lib64/libwx_baseu-3.2.so.0 (0x00007f3cece00000)

But it's only a suspicion.  It could instead be a difference in some
other set of libraries, or in the compiler version used.  Or it could be
a timing issue that either is or is not triggered by generated code.
Perhaps the generated code for your semi-reproducible case hits right
on the edge of that timing. 
In any case I don't think the difference indicates a change in the gnuplot
source code.

Having said all that, if someone pins down the actual point of failure
or timing issue, perhaps we could code around it.   

	Ethan

> On some invocations the frequency bars appear in the first sub-plot, 
> on other invocations they do not appear, the upper plot shows no 
> data. But if I write the plot to file using, say, pngcairo or 
> pdfcairo, the first sub-plot is always OK.
> 
> Am I misusing multiplot or might there be a bug here?
> 
> 

strange multiplot behavior with wxt

[Allin C. <cot...@wf...>]

I'm puzzled by a multiplot example where the output seems to be 
non-deterministic -- though the fault may be in my script 
(strange_multi.plt, attached).

I'm on Linux, using gnuplot 5.4.3 and also 5.4.8, using 
the wxt terminal for screen display with the command line

gnuplot strange_multi.plt -persist

On some invocations the frequency bars appear in the first sub-plot, 
on other invocations they do not appear, the upper plot shows no 
data. But if I write the plot to file using, say, pngcairo or 
pdfcairo, the first sub-plot is always OK.

Am I misusing multiplot or might there be a bug here?

-- 
Allin Cottrell
Department of Economics
Wake Forest University

Re: Re: Commit [36a21d]

[Tatsuro M. <tma...@ya...>]

Ethan
Thanks for the reply.

>  Under Windows + mingw it is
This is also true on cygwin.
 
Under Windows + mingw and cygwin it is

Tatsuro
> ----- Original Message -----
> 
> From: "Ethan A Merritt" <me...@uw...>
> To: "Tatsuro MATSUOKA" <tma...@ya...>
> Cc: "beta" <gnu...@li...>
> Date: 2023/06/25 日 00:57
> Subject: Re: Commit [36a21d]
> 
> 
> Thank you for catching that.  The "-fPIC" flag should have been added
> to the previous example command, the one for linux.
> 
>   Ethan
> 
> On Fri, Jun 23, 2023 at 11:55 PM Tatsuro MATSUOKA via gnuplot-beta
> <gnu...@li...> wrote:
> >
> > I found in Commit [36a21d]
> > +++ b/demo/plugin/README
> > @@ -23,12 +23,12 @@
> >
> >  Under Windows + mingw it is
> >
> > -    gcc -shared -lm -o newprogram.dll newprogram.c
> > +    gcc -shared -lm -fPIC -o newprogram.dll newprogram.c
> >
> > As far as I know, -fPIC is not effective on WIndows.
> > https://urldefense.com/v3/__https://bugs.webkit.org/show_bug.cgi?id=163949__;!!K-Hz7m0Vt54!jBF_aoqeTsaFf72wifSDQ-GopXPU9-4IpIwvRSENh-r0xtrJ-MD4LVRY3HP5A_6Br1lExVzif4elmnlMcXpyHjzUyma7UQ$
> >
> > Tatsuro
> >
> >
> >
> > _______________________________________________
> > gnuplot-beta mailing list
> > gnu...@li...
> > Membership management via: https://urldefense.com/v3/__https://lists.sourceforge.net/lists/listinfo/gnuplot-beta__;!!K-Hz7m0Vt54!jBF_aoqeTsaFf72wifSDQ-GopXPU9-4IpIwvRSENh-r0xtrJ-MD4LVRY3HP5A_6Br1lExVzif4elmnlMcXpyHjxbN-9g9w$
> 
> 
> 
> -- 
> Ethan A Merritt
> Department of Biochemistry
> University of Washington, Seattle

Re: Commit [36a21d]

[Ethan A M. <me...@uw...>]

Thank you for catching that.  The "-fPIC" flag should have been added
to the previous example command, the one for linux.

  Ethan

On Fri, Jun 23, 2023 at 11:55 PM Tatsuro MATSUOKA via gnuplot-beta
<gnu...@li...> wrote:
>
> I found in Commit [36a21d]
> +++ b/demo/plugin/README
> @@ -23,12 +23,12 @@
>
>  Under Windows + mingw it is
>
> -    gcc -shared -lm -o newprogram.dll newprogram.c
> +    gcc -shared -lm -fPIC -o newprogram.dll newprogram.c
>
> As far as I know, -fPIC is not effective on WIndows.
> https://urldefense.com/v3/__https://bugs.webkit.org/show_bug.cgi?id=163949__;!!K-Hz7m0Vt54!jBF_aoqeTsaFf72wifSDQ-GopXPU9-4IpIwvRSENh-r0xtrJ-MD4LVRY3HP5A_6Br1lExVzif4elmnlMcXpyHjzUyma7UQ$
>
> Tatsuro
>
>
>
> _______________________________________________
> gnuplot-beta mailing list
> gnu...@li...
> Membership management via: https://urldefense.com/v3/__https://lists.sourceforge.net/lists/listinfo/gnuplot-beta__;!!K-Hz7m0Vt54!jBF_aoqeTsaFf72wifSDQ-GopXPU9-4IpIwvRSENh-r0xtrJ-MD4LVRY3HP5A_6Br1lExVzif4elmnlMcXpyHjxbN-9g9w$



-- 
Ethan A Merritt
Department of Biochemistry
University of Washington, Seattle

Commit [36a21d]

[Tatsuro M. <tma...@ya...>]

I found in Commit [36a21d]
+++ b/demo/plugin/README
@@ -23,12 +23,12 @@
 
 Under Windows + mingw it is
 
-    gcc -shared -lm -o newprogram.dll newprogram.c
+    gcc -shared -lm -fPIC -o newprogram.dll newprogram.c

As far as I know, -fPIC is not effective on WIndows.
https://bugs.webkit.org/show_bug.cgi?id=163949

Tatsuro

Re: SVG pixelation (was pdfcairo terminal seams between bars in the colorbox)

[Dima K. <gn...@di...>]

Ethan A Merritt <me...@uw...> writes:

> I looked into it a bit further.
>
> The thing is, the two hinting properties you found are not part of the
> SVG standard (version 2).  They are instead CSS style hints to the browser.
> Chrome and Firefox apparently dive into an included SVG file and extract
> the CSS style hints from there. SVG-aware programs that are not browsers,
> like inkscape or gimp, could do the same but the motivation for doing
> so would come from imitating Chrome or Firefox rather than adhering to
> the SVG standard.  Inkscape does, but gimp doesn't. 
>
>> I'll file feature-request bug reports for those two projects.
>> But the patch should probably still go in: supporting firefox and chrome
>> is much better than supporting nothing.

> For me personally, the support from inkscape is just as important as
> from the browsers.  I use it to prepare figures for publication.
> And it's still true that you can work around this
> by using "with image pixels" for gnuplot heat maps. 
>
> What about lobbying the SVG working group to add an equivalent
> to the SVG spec?  All the various support libraries would then
> eventually fall in line.  As of SVG 2.0 you only get a choice
> between "optimizeSpeed" and "optimizeQuality", both of which
> are so vague as to be useless for our purpose.
>
>     https://svgwg.org/svg2-draft/painting.html#ImageRendering

That's interesting. Not sure what it means for us in practical terms.
The patch in this thread should cover most browsers. Most non-browser
applications I've seen use rsvg: bug report here

  https://gitlab.gnome.org/GNOME/librsvg/-/issues/985

I guess QT has its own implementation too; I ran out of steam, and
haven't pestered them. I don't know what libreoffice does, but it does
the right thing on my machine. And I don't know how to lobby the SVG
working group.

I would merge the patch in this thread, maybe file a bug report against
QSvg, and move on.

Re: SVG pixelation (was pdfcairo terminal seams between bars in the colorbox)

[Ethan A M. <me...@uw...>]

On Sunday, 18 June 2023 23:55:07 PDT Dima Kogan wrote:
> Ethan A Merritt <me...@uw...> writes:
> 
> > As tested here those attributes are indeed honored by firefox and chrome.
> > Mixed success on other svg handling programs.
> 
> I'm guessing that adding support for either image-rendering=crips-edge
> or image-rendering=pixelated into librsvg and QSvg would make all of
> these work. But it looks like these two libraries simply don't support
> either option yet.

I looked into it a bit further.

The thing is, the two hinting properties you found are not part of the
SVG standard (version 2).  They are instead CSS style hints to the browser.
Chrome and Firefox apparently dive into an included SVG file and extract
the CSS style hints from there. SVG-aware programs that are not browsers,
like inkscape or gimp, could do the same but the motivation for doing
so would come from imitating Chrome or Firefox rather than adhering to
the SVG standard.  Inkscape does, but gimp doesn't. 

> I'll file feature-request bug reports for those two projects.
> But the patch should probably still go in: supporting firefox and chrome
> is much better than supporting nothing.

For me personally, the support from inkscape is just as important as
from the browsers.  I use it to prepare figures for publication.
And it's still true that you can work around this
by using "with image pixels" for gnuplot heat maps. 

What about lobbying the SVG working group to add an equivalent
to the SVG spec?  All the various support libraries would then
eventually fall in line.  As of SVG 2.0 you only get a choice
between "optimizeSpeed" and "optimizeQuality", both of which
are so vague as to be useless for our purpose.

    https://svgwg.org/svg2-draft/painting.html#ImageRendering

Re: SVG pixelation (was pdfcairo terminal seams between bars in the colorbox)

[Dima K. <gn...@di...>]

Ethan A Merritt <me...@uw...> writes:

> As tested here those attributes are indeed honored by firefox and chrome.
> Mixed success on other svg handling programs.

I'm guessing that adding support for either image-rendering=crips-edge
or image-rendering=pixelated into librsvg and QSvg would make all of
these work. But it looks like these two libraries simply don't support
either option yet.

I'll file feature-request bug reports for those two projects.

But the patch should probably still go in: supporting firefox and chrome
is much better than supporting nothing.

SVG pixelation (was pdfcairo terminal seams between bars in the colorbox)

[Ethan A M. <me...@uw...>]

> The svg terminal currently produces images with no image-rendering
> annotations. And most browsers render images with bilinear interpolation
> by default, which makes low-res images look very wrong.

> My friend figured out how to get svg to work
> in both firefox and chrome, and I'm attaching a patch that does that.
> The description is in the patch.

[snip patch that adds attributes
 <image image-rendering='crisp-edges' style='image-rendering: pixelated' ...
]

Thanks.  That's a good start.

As tested here those attributes are indeed honored by firefox and chrome.
Mixed success on other svg handling programs.

Works:
   chrome
   firefox
   inkscape/inkview
   ImageMagick

Fails:
   konqueror
   LibreOffice / soffice
   gwenview
   gimp

Unknown (did not try):
   eog
   geeqie

	Ethan

Re: pdfcairo terminal; seams between bars in the colorbox

[Dima K. <gn...@di...>]

Thanks for merging those. My friend figured out how to get svg to work
in both firefox and chrome, and I'm attaching a patch that does that.
The description is in the patch.

Re: pdfcairo terminal; seams between bars in the colorbox

[Ethan A M. <me...@uw...>]

OK, it's pushed to the development branch for 6.1.

I revised the top-level criterion for choosing the new method that
uses term->image rather than the old method that uses term->fille_polygon.
Commit message:

    Rather than selecting the colorbox-as-image code for all terminals
    that support term->image, make it dependent on a terminal flag.
    Instead of
        if (term->image) ...
    use
        if ((term->flags & TERM_COLORBOX_IMAGE))
    
    The svg, domterm, and canvas terminals definitely do not want this flag.
    The pdfcairo terminal definitely does want it.
    
    For now the flag is set by these terminals
            epscairo cairolatex pdfcairo qt
    
    Other terminals can be added later by setting TERM_COLOR_BOX_IMAGE in the
    corresponding driver source.  Possible candidates
            tikz wxt

The PostScript-based terminals are not affected since colorbox generation
code is done in the PostScript header code rather than in the core C code.

Thanks for working on this.

    Ethan

On Thursday, 15 June 2023 22:13:47 PDT Ethan A Merritt wrote:
> On Thursday, 15 June 2023 15:39:22 PDT Dima Kogan wrote:
> > 
> >   dima@shorty:~/projects/gnuplot$ git show-branch HEAD origin/master
> >   ! [HEAD] smooth colorbox: render as a bitmap if the terminal supports it
> >    ! [origin/master] Do not report terminal type to user until after initialization is complete
> >   --
> >   +  [HEAD] smooth colorbox: render as a bitmap if the terminal supports it
> >   +  [HEAD^] no-op cleanup: consolidated next-colorbox-slice logic
> >   +  [HEAD~2] no-op cleanup: consolidated draw-colorbox-slice logic
> >   +  [HEAD~3] no-op cleanup: consolidated colorbox bounds logic
> >   +  [HEAD~4] no-op cleanup: consolidated steps-in-colorbox logic
> >    + [origin/master] Do not report terminal type to user until after initialization is complete
> >    + [origin/master^] VMS cleanup
> >   ++ [HEAD~5] qt:  qt4 was apparently weak at int->double promotion
> > 
> > I'm attaching the patches to this email.
> 
> I like it so far.
> I will try to think of more tests to run.
> 
> > > OK. The other alternative is to make this specific to cairopdf. Unless
> > > you know of other terminals that suffer from the same problem?
> 
> [snip descroption of svg problems with small number of image pixels]
> > So if we
> > can fix this, I'd say we should do the colorbox-image thing whenever
> > available. But if we can't, maybe limiting it to only cairopdf (or NOT
> > svg) is the answer.
> > 
> > Let me ask a web-dev friend about how to make svg work right in all
> > browsers. There's probably a way.
> > 
> > The current set of attached patches does NOT limit itself to cairopdf
> 
> I think it benefits all the cairo terminals, or at least doesn't hurt.
> Beyond that, well - that's what I am trying to test.
> I will add a FIXME comment.
>  
> > Ah yes. This is exactly what I mentioned above. I don't remember any
> > terminal other than svg having this problem. But I certainly haven't
> > touched all of them.
> 
> The HTML5 canvas terminal has the same problem: browsers apply
> inter-pixel interpolation, details of which are poorly documented.
> 
>    cheers
>    Ethan
> 
> 
> 


-- 
Ethan A Merritt
Biomolecular Structure Center,  K-428 Health Sciences Bldg
MS 357742,   University of Washington, Seattle 98195-7742

Re: pdfcairo terminal; seams between bars in the colorbox

[Petr M. <mi...@ph...>]

> Oh, and two more notes, where the old code had off-by-one errors
>
> - There old code was set up to overlap each colorbox by extending the
>  bottom/right edge of each slice: GPMIN(xy_to,xy2+1)

Please try also "set term postscript color" output to see what your pdf 
viewers do there. These "write stripes issue" and antialiasing was an issue 
two decades ago. It depended whether you have switched on or off antialising 
in the pdf/ps viewer. Hm, I see that current viewer do not have such an option 
any longer (even though ghostscript can be run with different options), so you 
cannot check the problem directly in the viewer.

What you can try as the lowest-level approach is

gs a.pdf
gs -dDOINTERPOLATE a.pdf

Pdf itself can manipulate antialiasing directly, such as 
3 0 obj
<</AntiAlias false>>
but that's probably possible to insert directy in the poppler or 
cairo library?

> - The old code was computing the colors for each slice like this:
>
>    for (i = 0; i < steps; i++)
>      gray = i / (double)steps;
>
>  This is another off-by-one error. The colors in the colorbox spanned
>  [0,1-1/steps] instead of [0,1]. My patch fixes this one.
>
>  There's some funky logic in quanize_gray() that maybe is trying to
>  deal with this in the limited-max-colors case, but it's not commented,
>  and applies to only this case. Code:
>
>    qgray = floor(gray * sm_palette.use_maxcolors)
> 		        / (sm_palette.use_maxcolors-1);

The colorbox code must work correctly for limited number of colors, try
 	set palette maxcolors 4
 	splot x*y with pm3d
Thus I think the code is correct, if you see exactly 4 color boxes in the 
colorbox.

---
Petr Mikulik

Re: pdfcairo terminal; seams between bars in the colorbox

[Ethan A M. <me...@uw...>]

On Thursday, 15 June 2023 15:39:22 PDT Dima Kogan wrote:
> 
>   dima@shorty:~/projects/gnuplot$ git show-branch HEAD origin/master
>   ! [HEAD] smooth colorbox: render as a bitmap if the terminal supports it
>    ! [origin/master] Do not report terminal type to user until after initialization is complete
>   --
>   +  [HEAD] smooth colorbox: render as a bitmap if the terminal supports it
>   +  [HEAD^] no-op cleanup: consolidated next-colorbox-slice logic
>   +  [HEAD~2] no-op cleanup: consolidated draw-colorbox-slice logic
>   +  [HEAD~3] no-op cleanup: consolidated colorbox bounds logic
>   +  [HEAD~4] no-op cleanup: consolidated steps-in-colorbox logic
>    + [origin/master] Do not report terminal type to user until after initialization is complete
>    + [origin/master^] VMS cleanup
>   ++ [HEAD~5] qt:  qt4 was apparently weak at int->double promotion
> 
> I'm attaching the patches to this email.

I like it so far.
I will try to think of more tests to run.

> > OK. The other alternative is to make this specific to cairopdf. Unless
> > you know of other terminals that suffer from the same problem?

[snip descroption of svg problems with small number of image pixels]
> So if we
> can fix this, I'd say we should do the colorbox-image thing whenever
> available. But if we can't, maybe limiting it to only cairopdf (or NOT
> svg) is the answer.
> 
> Let me ask a web-dev friend about how to make svg work right in all
> browsers. There's probably a way.
> 
> The current set of attached patches does NOT limit itself to cairopdf

I think it benefits all the cairo terminals, or at least doesn't hurt.
Beyond that, well - that's what I am trying to test.
I will add a FIXME comment.
 
> Ah yes. This is exactly what I mentioned above. I don't remember any
> terminal other than svg having this problem. But I certainly haven't
> touched all of them.

The HTML5 canvas terminal has the same problem: browsers apply
inter-pixel interpolation, details of which are poorly documented.

   cheers
   Ethan

Re: pdfcairo terminal; seams between bars in the colorbox

[Dima K. <gn...@di...>]

Ethan A Merritt <me...@uw...> writes:

>> The patches I'm proposing are here:
>> 
>>   https://github.com/dkogan/gnuplot/tree/colorbox_smooth
>
> I do not know how to generate the relevant patches via github. It
> says: "master and colorbox_smooth are entirely different commit
> histories" Could you please send me the relevant patch series as
> generated by "git format-patch"?

I'm only two patches out-of-date:

  dima@shorty:~/projects/gnuplot$ git show-branch HEAD origin/master
  ! [HEAD] smooth colorbox: render as a bitmap if the terminal supports it
   ! [origin/master] Do not report terminal type to user until after initialization is complete
  --
  +  [HEAD] smooth colorbox: render as a bitmap if the terminal supports it
  +  [HEAD^] no-op cleanup: consolidated next-colorbox-slice logic
  +  [HEAD~2] no-op cleanup: consolidated draw-colorbox-slice logic
  +  [HEAD~3] no-op cleanup: consolidated colorbox bounds logic
  +  [HEAD~4] no-op cleanup: consolidated steps-in-colorbox logic
   + [origin/master] Do not report terminal type to user until after initialization is complete
   + [origin/master^] VMS cleanup
  ++ [HEAD~5] qt:  qt4 was apparently weak at int->double promotion

I'm attaching the patches to this email.


> OK. The other alternative is to make this specific to cairopdf. Unless
> you know of other terminals that suffer from the same problem?

Actually.... You're right. It's only semi-related to THIS discussion,
but let me bring that up.

The svg terminal currently produces images with no image-rendering
annotations. And most browsers render images with bilinear interpolation
by default, which makes low-res images look very wrong. I looked at this
a while back, and, there was no browser-independent way to turn that
off. And I think there still isn't. I'm attaching an svg of this gnuplot
script made WITH the new patches AND by manually adding
image-rendering="crisp-edges" to it:

  set output "/tmp/tst-colorbox-log.svg"
  set terminal svg
  set palette maxcolors 5
  set logscale cb
  set cbrange [1:10]
  plot 5 with points palette

If you open that in firefox, it should look good, but if you open it in
chrome, it won't. To make it work in chrome, "crisp-edges" needs to
become "pixelated". I don't know how to make it work with both. So if we
can fix this, I'd say we should do the colorbox-image thing whenever
available. But if we can't, maybe limiting it to only cairopdf (or NOT
svg) is the answer.

Let me ask a web-dev friend about how to make svg work right in all
browsers. There's probably a way.

The current set of attached patches does NOT limit itself to cairopdf


>> I don't see this. I'm looking at
>> 
>>   https://gnuplot.sourceforge.net/demo/heatmaps.html
>
> That is the png version; it doesn't show the effect I'm talking about.
> Have a look at the svg version
>     https://gnuplot.sourceforge.net/demo_svg/heatmaps.html
>
> In particular the plot with title
>     Compare "image" and "image pixels"

Ah yes. This is exactly what I mentioned above. I don't remember any
terminal other than svg having this problem. But I certainly haven't
touched all of them.

Re: pdfcairo terminal; seams between bars in the colorbox

[Ethan A M. <me...@uw...>]

On Wednesday, 14 June 2023 10:43:26 PDT Dima Kogan wrote:
> Hi Ethan. Thanks for looking at this.
> 
> If possible I'd like to get the colorbox-as-bitmap logic integrated. I'm
> all for filing bugs in popper and mupdf and all those, but I don't have
> an understanding of what's going on there at all, or if these are even
> bugs. So hopefully we can get another implementation of gnuplot colorbox
> rendering we're happy with.
> 
> The patches I'm proposing are here:
> 
>   https://github.com/dkogan/gnuplot/tree/colorbox_smooth

I do not know how to generate the relevant patches via github.
It says:  "master and colorbox_smooth are entirely different commit histories"
Could you please send me the relevant patch series as generated by
"git format-patch"?
  
> > This is already possible in current gnuplot, although to be fair it is not
> > hooked up automatically to colorbox generation and maybe not well
> > enough documented.   See "named_palettes.dem" for an example.
> > In a nutshell:
> > 1) Define your palette ("set palette ...")
> > 2) Save it as a colormap ("set colormap new NAME")
> > 3) Use the colormap as a pixmap
> >    ("set pixmap 1 colormap NAME at screen 0.92, 0.3  size screen 0.03, 0.4")
> 
> OK. Yes. I have confirmed that this uses a bitmap for the colorbox. I'm
> not sure what bearing this has. Are you saying the code that does this
> should be consolidated with the new code that does the bitmap colorbox?

Mostly I meant that the good/bad effects of this approach can be tested
using the code already there.  

> > However this approach has at least two limitations,
> > whether done via colormap+pixmap or via your proposed new code.
> > 1) Not all terminals support it
> 
> The patches above fall back on the discrete slices if term->image==NULL

OK.  The other alternative is to make this specific to cairopdf.
Unless you know of other terminals that suffer from the same problem?

> > 2) It does not handle nonlinear color mappings,
> >    e.g. "set log cb"
> 
> I just tested it. It looks the same before/after. The logscale affects
> the tic marks on the colorbox, not the color box image.

You are right.  I was thinking of something else, but never mind.

> > There is a third issue also that would apply to using it for svg or
> > canvas output. See the 7th plot in heatmaps.dem. The display programs
> > for these modes apply an averaging/blurring function across image
> > pixels, which means that discrete palette colors would be muddied or
> > mangled entirely if represented by image pixels rather than by
> > separate solid color rectangles.
> 
> I don't see this. I'm looking at
> 
>   https://gnuplot.sourceforge.net/demo/heatmaps.html

That is the png version; it doesn't show the effect I'm talking about.
Have a look at the svg version
    https://gnuplot.sourceforge.net/demo_svg/heatmaps.html

In particular the plot with title
    Compare "image" and "image pixels"

> Can you try the patches?

If you send them as patches, yeah.
Or upload them to the patch tracker
    https://sourceforge.net/p/gnuplot/patches/

	Ethan

Re: pdfcairo terminal; seams between bars in the colorbox

[Dima K. <gn...@di...>]

Hi Ethan. Thanks for looking at this.

If possible I'd like to get the colorbox-as-bitmap logic integrated. I'm
all for filing bugs in popper and mupdf and all those, but I don't have
an understanding of what's going on there at all, or if these are even
bugs. So hopefully we can get another implementation of gnuplot colorbox
rendering we're happy with.

The patches I'm proposing are here:

  https://github.com/dkogan/gnuplot/tree/colorbox_smooth

This is the "colorbox_smooth" branch of that repo. There are several
no-op cleanup patches that consolidate duplicated logic. And one patch
to use the colorbox-as-bitmap path if term->image is available. If it's
not available we fall back to the existing implementation.

I think this works. I tested it with horizontal/vertical colorboxes and
logscales. Before/after patch behavior is the same, except for the
anti-aliasing artifacts we're trying to get rid of.

More notes inline.


Ethan A Merritt <me...@uw...> writes:

> On Sunday, 11 June 2023 16:22:01 PDT Dima Kogan wrote:

> This is already possible in current gnuplot, although to be fair it is not
> hooked up automatically to colorbox generation and maybe not well
> enough documented.   See "named_palettes.dem" for an example.
> In a nutshell:
> 1) Define your palette ("set palette ...")
> 2) Save it as a colormap ("set colormap new NAME")
> 3) Use the colormap as a pixmap
>    ("set pixmap 1 colormap NAME at screen 0.92, 0.3  size screen 0.03, 0.4")

OK. Yes. I have confirmed that this uses a bitmap for the colorbox. I'm
not sure what bearing this has. Are you saying the code that does this
should be consolidated with the new code that does the bitmap colorbox?


> However this approach has at least two limitations,
> whether done via colormap+pixmap or via your proposed new code.
> 1) Not all terminals support it

The patches above fall back on the discrete slices if term->image==NULL


> 2) It does not handle nonlinear color mappings,
>    e.g. "set log cb"

I just tested it. It looks the same before/after. The logscale affects
the tic marks on the colorbox, not the color box image.


> There is a third issue also that would apply to using it for svg or
> canvas output. See the 7th plot in heatmaps.dem. The display programs
> for these modes apply an averaging/blurring function across image
> pixels, which means that discrete palette colors would be muddied or
> mangled entirely if represented by image pixels rather than by
> separate solid color rectangles.

I don't see this. I'm looking at

  https://gnuplot.sourceforge.net/demo/heatmaps.html

The 7th plot is the last one. I don't think that's what you mean. If I
run the "headtmaps.dem" script with the current gnuplot in git (the
website should match!) I see a different sequence. The 7th one is the
one with the two images without any colorboxes at all. But they all look
right in any case.

Can you try the patches?

Thanks

Re: pdfcairo terminal; seams between bars in the colorbox

[Ethan A M. <me...@uw...>]

New information:

I found a chunk of code in cairo terminal driver from way back when
that was intended to deal with an old version of the cairo library
(2010 or earlier).

good news:
The one line patch below fixes or at least improves the banding
artifact in pdf output for almost all plots I have tried.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
diff --git a/term/cairo.trm b/term/cairo.trm
index 9df11cc79..8ab9698cb 100644
--- a/term/cairo.trm
+++ b/term/cairo.trm
@@ -775,7 +775,7 @@ void cairotrm_init()
 		/* disable OPERATOR_SATURATE, not implemented in cairo pdf backend.
 		 * Unfortunately the can result in noticeable seams between adjacent
 		 * polygons. */
-		plot.polygons_saturate = FALSE;
+		plot.polygons_saturate = TRUE;
 		/* Empirical correction to make pdf output look more like wxt and png */
 		plot.dashlength /= 2;
 	} else if (!strcmp(term->name,"pngcairo") || !strcmp(term->name, "kittycairo")) {
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

bad news:
It degrades text rendering in the pdf output.  I don't know why.
For at least one plot (walls.dem) it introduces a much worse artifact.
I don't know why on that one either.

There are other places in the cairo.trm code that were also
added apparently to accommodate this old cairo libary.
I will try to trace back what changed in cairo and why the
gnuplot terminal driver was trying to work around it.
The bad text rendering side effect is a show-stopper at the moment.
If I can disentangle that from the change to polygon rendering
I would be inclined to make the change in the gnuplot development
source and work on the resulting artifact in walls.dem separately.

I am dubious about the large block of code in gp_cairo.c that begins 
  /* this is meant to test Full-Scene-Anti-Aliasing by supersampling,
   * in association with CAIRO_ANTIALIAS_NONE a few lines below
   */
Before the change to plot.polygons_saturate it was never executed for
pdf output. Now it is.  Perhaps it can be replaced by something simpler?
The "supersampling" idea was never implemented, so maybe this code
never did work correctly.  On the other hand it is also used by
the wxt terminal and I haven't noticed problems there, so ....?

Anyhow, if you have any ideas or can find a way to remove the
text-rendering side effect that would be great.

   cheers,
      Ethan

Re: pdfcairo terminal; seams between bars in the colorbox

[Dima K. <gn...@di...>]

Ethan A Merritt <me...@uw...> writes:

> I do not see any such problem in the output from either pdflatex or lualatex as run here.
>
>     [~/temp] pdflatex --version 
>     pdfTeX 3.141592653-2.6-1.40.24 (TeX Live 2022/Mageia)
>     [~/temp] lualatex --version 
>     This is LuaHBTeX, Version 1.15.0 (TeX Live 2022/Mageia) 
>     Development id: 7509
>
> Output from pdflatex attached.

My bad. I gave you a pdf link in the last email that showed the issue.
But since then I've been working on my presentation, and in that process
re-generated my figures with the patched gnuplot, so the link in the
last email now points to a fixed image. This is what you used in your
attachment just now, and that's why it looks nice. You can confirm like
this:

  dima@shorty:~/projects/gnuplot$ pdfimages -list /tmp/dima.pdf

  page   num  type   width height color comp bpc  enc interp  object ID x-ppi y-ppi size ratio
  --------------------------------------------------------------------------------------------
     1     0 image      60    40  rgb     3   8  image  no        42  0    25    29 3602B  50%
     1     1 image       1   128  rgb     3   8  image  no        43  0     8    96  395B 103%

The 1x128 bitmap embedded in the .pdf is the colorbar, and its presence
indicates a patched gnuplot.

I'm attaching the pre-fix pdf file. I suspect you will see issues with
it if you pdflatex and the okular the output.


> So again, I do not think the problem is in gnuplot but rather in the
> pdf processing stage that comes after that.

Maybe. But if the issues in the surrounding tools are so pervasive that
things are broken more often than not, I'd be interested in tweaking
gnuplot to work well in the world that we have.


> But I'm not all that hopeful; variations of this sort of aliasing
> glitch persisted for years (decades?) in ghostview / gv despite it
> being a known and well-characterized bug

I haven't been fighting it for as long, so I remain optimistic :)

You raised some great points in your other email. I'll need a few days
to look at those.

Thanks much.