gnuplot-beta — Developers and beta testers of new gnuplot code

Re: strange multiplot behavior with wxt

[Allin C. <cot...@wf...>]

On Wed, 28 Jun 2023, Ethan A Merritt wrote:

> Valgrind finds a test of an uninitialized flag "hidden" that could
> plausibly be involved here.
> I have corrected that by setting the flag to FALSE in the one place
> that I could see initialization omitted, but I do not actually
> understand why that code path is triggered by this test case (it is
> the response to a "refresh" command).  That makes valgrind happy, so
> maybe it will fix your display also.  I'll leave the tracker issue
> "open" for now.

Thanks, Ethan! I've built gnuplot 6.1.0 from gnuplot-main git and 
tested on Fedora: it works fine. I'll also try on Arch with more 
recent wx, but it looks like you've nailed the problem.

Just wondering: are you likely to cherry-pick this fix into 
branch-5-4-stable?

Allin Cottrell

>
> On Wed, Jun 28, 2023 at 10:31 AM Allin Cottrell <cot...@wf...> wrote:
>>
>> On Wed, 28 Jun 2023, Ethan A Merritt wrote:
>>
>>> On Wednesday, 28 June 2023 05:50:15 PDT Allin Cottrell wrote:
>>>> I'm puzzled by a multiplot example where the output seems to be
>>>> non-deterministic -- though the fault may be in my script
>>>> (strange_multi.plt, attached).
>>>>
>>>> I'm on Linux, using gnuplot 5.4.3 and also 5.4.8, using
>>>> the wxt terminal for screen display with the command line
>>>>
>>>> gnuplot strange_multi.plt -persist
>>>
>>> I have kept source and executables around for many versions of gnuplot.
>>> I can reproduce your "missing bars in first plot" symptom with old
>>> executables for 5.4.0 through 5.4.6, but not with 5.4.8 or the
>>> development branch.  Any given executable seems to be 100% reproducible
>>> in either failing or succeeding.
>>>
>>> But here's the thing - if I rebuild 5.4.3 from the same original
>>> source, I get a new executable that does not exhibit the problem.
>>> So my suspicion is that the difference is the version of the
>>> wx- shared libraries the binary is linked against.
>>>
>>> old gnuplot_5.4.3 binary (built May 2022)
>>> [~/temp] ldd /usr/local/bin/gnuplot_5.4.3 | grep wx
>>>        libwx_gtk3u_xrc-3.1.so.5 => /lib64/libwx_gtk3u_xrc-3.1.so.5 (0x00007fc634b09000)
>>>        libwx_gtk3u_html-3.1.so.5 => /lib64/libwx_gtk3u_html-3.1.so.5 (0x00007fc634a25000)
>>>        libwx_gtk3u_qa-3.1.so.5 => /lib64/libwx_gtk3u_qa-3.1.so.5 (0x00007fc6349f6000)
>>>        libwx_gtk3u_core-3.1.so.5 => /lib64/libwx_gtk3u_core-3.1.so.5 (0x00007fc634000000)
>>>        libwx_baseu_xml-3.1.so.5 => /lib64/libwx_baseu_xml-3.1.so.5 (0x00007fc6349e2000)
>>>        libwx_baseu_net-3.1.so.5 => /lib64/libwx_baseu_net-3.1.so.5 (0x00007fc63497f000)
>>>        libwx_baseu-3.1.so.5 => /lib64/libwx_baseu-3.1.so.5 (0x00007fc633c00000)
>>>
>>> new gnuplot_5.4.3 binary (built today)
>>> [~/temp] ldd ~/cvs/gnuplot-5.4.3/src/gnuplot | grep wx
>>>        libwx_gtk3u_xrc-3.2.so.0 => /lib64/libwx_gtk3u_xrc-3.2.so.0 (0x00007f3cedc64000)
>>>        libwx_gtk3u_html-3.2.so.0 => /lib64/libwx_gtk3u_html-3.2.so.0 (0x00007f3cedb83000)
>>>        libwx_gtk3u_qa-3.2.so.0 => /lib64/libwx_gtk3u_qa-3.2.so.0 (0x00007f3cedb54000)
>>>        libwx_gtk3u_core-3.2.so.0 => /lib64/libwx_gtk3u_core-3.2.so.0 (0x00007f3ced200000)
>>>        libwx_baseu_xml-3.2.so.0 => /lib64/libwx_baseu_xml-3.2.so.0 (0x00007f3cedb42000)
>>>        libwx_baseu_net-3.2.so.0 => /lib64/libwx_baseu_net-3.2.so.0 (0x00007f3cedade000)
>>>        libwx_baseu-3.2.so.0 => /lib64/libwx_baseu-3.2.so.0 (0x00007f3cece00000)
>>>
>>> But it's only a suspicion. [...]
>>
>> I have a little more information to contribute. I'm testing my
>> example on two machines. The one I'm sitting at runs Fedora 36,
>> with gnuplot 5.4.3 and wx 3.0.5 (per wx-config). The linkage is
>>
>> myrtle:~$ ldd /usr/bin/gnuplot | grep wx
>>          libwx_gtk3u_core-3.0.so.0 => /lib64/libwx_gtk3u_core-3.0.so.0 (0x00007fa5bfc00000)
>>          libwx_baseu-3.0.so.0 => /lib64/libwx_baseu-3.0.so.0 (0x00007fa5bf800000)
>>
>> On this machine I'm seeing the non-deterministic behavior.
>>
>> The other machine I'm accessing remotely via ssh. It runs current
>> Arch Linux with gnuplot 5.4.8 and wx 3.2.2, linkage:
>>
>> robroy:~$ ldd /usr/bin/gnuplot | grep wx
>>          libwx_gtk3u_core-3.2.so.0 => /usr/lib/libwx_gtk3u_core-3.2.so.0 (0x00007f5447a00000)
>>          libwx_baseu-3.2.so.0 => /usr/lib/libwx_baseu-3.2.so.0 (0x00007f5447600000)
>>
>> On this more up-to-date system I now think I was wrong to say the
>> behavior is non-deterministic. The plot appears somewhat slowly over
>> the remote connection, and here's the reproducible sequence I'm
>> seeing over 2 or 3 seconds:
>>
>> * an empty (black) window appears
>> * the canvas goes to white
>> * the first subplot appears, correctly
>> * the second subplot appears, correctly
>> * the data portion of the first subplot is wiped out to white
>>
>> Might this help with diagnosis?
>>
>> Allin Cottrell

Re: strange multiplot behavior with wxt

[Ethan A M. <me...@uw...>]

Valgrind finds a test of an uninitialized flag "hidden" that could
plausibly be involved here.
I have corrected that by setting the flag to FALSE in the one place
that I could see initialization omitted, but I do not actually
understand why that code path is triggered by this test case (it is
the response to a "refresh" command).  That makes valgrind happy, so
maybe it will fix your display also.  I'll leave the tracker issue
"open" for now.


On Wed, Jun 28, 2023 at 10:31 AM Allin Cottrell <cot...@wf...> wrote:
>
> On Wed, 28 Jun 2023, Ethan A Merritt wrote:
>
> > On Wednesday, 28 June 2023 05:50:15 PDT Allin Cottrell wrote:
> >> I'm puzzled by a multiplot example where the output seems to be
> >> non-deterministic -- though the fault may be in my script
> >> (strange_multi.plt, attached).
> >>
> >> I'm on Linux, using gnuplot 5.4.3 and also 5.4.8, using
> >> the wxt terminal for screen display with the command line
> >>
> >> gnuplot strange_multi.plt -persist
> >
> > I have kept source and executables around for many versions of gnuplot.
> > I can reproduce your "missing bars in first plot" symptom with old
> > executables for 5.4.0 through 5.4.6, but not with 5.4.8 or the
> > development branch.  Any given executable seems to be 100% reproducible
> > in either failing or succeeding.
> >
> > But here's the thing - if I rebuild 5.4.3 from the same original
> > source, I get a new executable that does not exhibit the problem.
> > So my suspicion is that the difference is the version of the
> > wx- shared libraries the binary is linked against.
> >
> > old gnuplot_5.4.3 binary (built May 2022)
> > [~/temp] ldd /usr/local/bin/gnuplot_5.4.3 | grep wx
> >        libwx_gtk3u_xrc-3.1.so.5 => /lib64/libwx_gtk3u_xrc-3.1.so.5 (0x00007fc634b09000)
> >        libwx_gtk3u_html-3.1.so.5 => /lib64/libwx_gtk3u_html-3.1.so.5 (0x00007fc634a25000)
> >        libwx_gtk3u_qa-3.1.so.5 => /lib64/libwx_gtk3u_qa-3.1.so.5 (0x00007fc6349f6000)
> >        libwx_gtk3u_core-3.1.so.5 => /lib64/libwx_gtk3u_core-3.1.so.5 (0x00007fc634000000)
> >        libwx_baseu_xml-3.1.so.5 => /lib64/libwx_baseu_xml-3.1.so.5 (0x00007fc6349e2000)
> >        libwx_baseu_net-3.1.so.5 => /lib64/libwx_baseu_net-3.1.so.5 (0x00007fc63497f000)
> >        libwx_baseu-3.1.so.5 => /lib64/libwx_baseu-3.1.so.5 (0x00007fc633c00000)
> >
> > new gnuplot_5.4.3 binary (built today)
> > [~/temp] ldd ~/cvs/gnuplot-5.4.3/src/gnuplot | grep wx
> >        libwx_gtk3u_xrc-3.2.so.0 => /lib64/libwx_gtk3u_xrc-3.2.so.0 (0x00007f3cedc64000)
> >        libwx_gtk3u_html-3.2.so.0 => /lib64/libwx_gtk3u_html-3.2.so.0 (0x00007f3cedb83000)
> >        libwx_gtk3u_qa-3.2.so.0 => /lib64/libwx_gtk3u_qa-3.2.so.0 (0x00007f3cedb54000)
> >        libwx_gtk3u_core-3.2.so.0 => /lib64/libwx_gtk3u_core-3.2.so.0 (0x00007f3ced200000)
> >        libwx_baseu_xml-3.2.so.0 => /lib64/libwx_baseu_xml-3.2.so.0 (0x00007f3cedb42000)
> >        libwx_baseu_net-3.2.so.0 => /lib64/libwx_baseu_net-3.2.so.0 (0x00007f3cedade000)
> >        libwx_baseu-3.2.so.0 => /lib64/libwx_baseu-3.2.so.0 (0x00007f3cece00000)
> >
> > But it's only a suspicion. [...]
>
> I have a little more information to contribute. I'm testing my
> example on two machines. The one I'm sitting at runs Fedora 36,
> with gnuplot 5.4.3 and wx 3.0.5 (per wx-config). The linkage is
>
> myrtle:~$ ldd /usr/bin/gnuplot | grep wx
>          libwx_gtk3u_core-3.0.so.0 => /lib64/libwx_gtk3u_core-3.0.so.0 (0x00007fa5bfc00000)
>          libwx_baseu-3.0.so.0 => /lib64/libwx_baseu-3.0.so.0 (0x00007fa5bf800000)
>
> On this machine I'm seeing the non-deterministic behavior.
>
> The other machine I'm accessing remotely via ssh. It runs current
> Arch Linux with gnuplot 5.4.8 and wx 3.2.2, linkage:
>
> robroy:~$ ldd /usr/bin/gnuplot | grep wx
>          libwx_gtk3u_core-3.2.so.0 => /usr/lib/libwx_gtk3u_core-3.2.so.0 (0x00007f5447a00000)
>          libwx_baseu-3.2.so.0 => /usr/lib/libwx_baseu-3.2.so.0 (0x00007f5447600000)
>
> On this more up-to-date system I now think I was wrong to say the
> behavior is non-deterministic. The plot appears somewhat slowly over
> the remote connection, and here's the reproducible sequence I'm
> seeing over 2 or 3 seconds:
>
> * an empty (black) window appears
> * the canvas goes to white
> * the first subplot appears, correctly
> * the second subplot appears, correctly
> * the data portion of the first subplot is wiped out to white
>
> Might this help with diagnosis?
>
> Allin Cottrell



-- 
Ethan A Merritt
Department of Biochemistry
University of Washington, Seattle

Re: strange multiplot behavior with wxt

[Allin C. <cot...@wf...>]

On Wed, 28 Jun 2023, Ethan A Merritt wrote:

> On Wednesday, 28 June 2023 05:50:15 PDT Allin Cottrell wrote:
>> I'm puzzled by a multiplot example where the output seems to be
>> non-deterministic -- though the fault may be in my script
>> (strange_multi.plt, attached).
>>
>> I'm on Linux, using gnuplot 5.4.3 and also 5.4.8, using
>> the wxt terminal for screen display with the command line
>>
>> gnuplot strange_multi.plt -persist
>
> I have kept source and executables around for many versions of gnuplot.
> I can reproduce your "missing bars in first plot" symptom with old
> executables for 5.4.0 through 5.4.6, but not with 5.4.8 or the
> development branch.  Any given executable seems to be 100% reproducible
> in either failing or succeeding.
>
> But here's the thing - if I rebuild 5.4.3 from the same original
> source, I get a new executable that does not exhibit the problem.
> So my suspicion is that the difference is the version of the
> wx- shared libraries the binary is linked against.
>
> old gnuplot_5.4.3 binary (built May 2022)
> [~/temp] ldd /usr/local/bin/gnuplot_5.4.3 | grep wx
>        libwx_gtk3u_xrc-3.1.so.5 => /lib64/libwx_gtk3u_xrc-3.1.so.5 (0x00007fc634b09000)
>        libwx_gtk3u_html-3.1.so.5 => /lib64/libwx_gtk3u_html-3.1.so.5 (0x00007fc634a25000)
>        libwx_gtk3u_qa-3.1.so.5 => /lib64/libwx_gtk3u_qa-3.1.so.5 (0x00007fc6349f6000)
>        libwx_gtk3u_core-3.1.so.5 => /lib64/libwx_gtk3u_core-3.1.so.5 (0x00007fc634000000)
>        libwx_baseu_xml-3.1.so.5 => /lib64/libwx_baseu_xml-3.1.so.5 (0x00007fc6349e2000)
>        libwx_baseu_net-3.1.so.5 => /lib64/libwx_baseu_net-3.1.so.5 (0x00007fc63497f000)
>        libwx_baseu-3.1.so.5 => /lib64/libwx_baseu-3.1.so.5 (0x00007fc633c00000)
>
> new gnuplot_5.4.3 binary (built today)
> [~/temp] ldd ~/cvs/gnuplot-5.4.3/src/gnuplot | grep wx
>        libwx_gtk3u_xrc-3.2.so.0 => /lib64/libwx_gtk3u_xrc-3.2.so.0 (0x00007f3cedc64000)
>        libwx_gtk3u_html-3.2.so.0 => /lib64/libwx_gtk3u_html-3.2.so.0 (0x00007f3cedb83000)
>        libwx_gtk3u_qa-3.2.so.0 => /lib64/libwx_gtk3u_qa-3.2.so.0 (0x00007f3cedb54000)
>        libwx_gtk3u_core-3.2.so.0 => /lib64/libwx_gtk3u_core-3.2.so.0 (0x00007f3ced200000)
>        libwx_baseu_xml-3.2.so.0 => /lib64/libwx_baseu_xml-3.2.so.0 (0x00007f3cedb42000)
>        libwx_baseu_net-3.2.so.0 => /lib64/libwx_baseu_net-3.2.so.0 (0x00007f3cedade000)
>        libwx_baseu-3.2.so.0 => /lib64/libwx_baseu-3.2.so.0 (0x00007f3cece00000)
>
> But it's only a suspicion. [...]

I have a little more information to contribute. I'm testing my 
example on two machines. The one I'm sitting at runs Fedora 36,
with gnuplot 5.4.3 and wx 3.0.5 (per wx-config). The linkage is

myrtle:~$ ldd /usr/bin/gnuplot | grep wx
         libwx_gtk3u_core-3.0.so.0 => /lib64/libwx_gtk3u_core-3.0.so.0 (0x00007fa5bfc00000)
         libwx_baseu-3.0.so.0 => /lib64/libwx_baseu-3.0.so.0 (0x00007fa5bf800000)

On this machine I'm seeing the non-deterministic behavior.

The other machine I'm accessing remotely via ssh. It runs current 
Arch Linux with gnuplot 5.4.8 and wx 3.2.2, linkage:

robroy:~$ ldd /usr/bin/gnuplot | grep wx
         libwx_gtk3u_core-3.2.so.0 => /usr/lib/libwx_gtk3u_core-3.2.so.0 (0x00007f5447a00000)
         libwx_baseu-3.2.so.0 => /usr/lib/libwx_baseu-3.2.so.0 (0x00007f5447600000)

On this more up-to-date system I now think I was wrong to say the 
behavior is non-deterministic. The plot appears somewhat slowly over 
the remote connection, and here's the reproducible sequence I'm 
seeing over 2 or 3 seconds:

* an empty (black) window appears
* the canvas goes to white
* the first subplot appears, correctly
* the second subplot appears, correctly
* the data portion of the first subplot is wiped out to white

Might this help with diagnosis?

Allin Cottrell

Re: strange multiplot behavior with wxt

[Ethan A M. <me...@uw...>]

On Wednesday, 28 June 2023 05:50:15 PDT Allin Cottrell wrote:
> I'm puzzled by a multiplot example where the output seems to be 
> non-deterministic -- though the fault may be in my script 
> (strange_multi.plt, attached).
> 
> I'm on Linux, using gnuplot 5.4.3 and also 5.4.8, using 
> the wxt terminal for screen display with the command line
> 
> gnuplot strange_multi.plt -persist

I have kept source and executables around for many versions of gnuplot.
I can reproduce your "missing bars in first plot" symptom with old
executables for 5.4.0 through 5.4.6, but not with 5.4.8 or the
development branch.  Any given executable seems to be 100% reproducible
in either failing or succeeding.

But here's the thing - if I rebuild 5.4.3 from the same original
source, I get a new executable that does not exhibit the problem.
So my suspicion is that the difference is the version of the 
wx- shared libraries the binary is linked against.

old gnuplot_5.4.3 binary (built May 2022)
 [~/temp] ldd /usr/local/bin/gnuplot_5.4.3 | grep wx
        libwx_gtk3u_xrc-3.1.so.5 => /lib64/libwx_gtk3u_xrc-3.1.so.5 (0x00007fc634b09000)
        libwx_gtk3u_html-3.1.so.5 => /lib64/libwx_gtk3u_html-3.1.so.5 (0x00007fc634a25000)
        libwx_gtk3u_qa-3.1.so.5 => /lib64/libwx_gtk3u_qa-3.1.so.5 (0x00007fc6349f6000)
        libwx_gtk3u_core-3.1.so.5 => /lib64/libwx_gtk3u_core-3.1.so.5 (0x00007fc634000000)
        libwx_baseu_xml-3.1.so.5 => /lib64/libwx_baseu_xml-3.1.so.5 (0x00007fc6349e2000)
        libwx_baseu_net-3.1.so.5 => /lib64/libwx_baseu_net-3.1.so.5 (0x00007fc63497f000)
        libwx_baseu-3.1.so.5 => /lib64/libwx_baseu-3.1.so.5 (0x00007fc633c00000)

new gnuplot_5.4.3 binary (built today)
[~/temp] ldd ~/cvs/gnuplot-5.4.3/src/gnuplot | grep wx
        libwx_gtk3u_xrc-3.2.so.0 => /lib64/libwx_gtk3u_xrc-3.2.so.0 (0x00007f3cedc64000)
        libwx_gtk3u_html-3.2.so.0 => /lib64/libwx_gtk3u_html-3.2.so.0 (0x00007f3cedb83000)
        libwx_gtk3u_qa-3.2.so.0 => /lib64/libwx_gtk3u_qa-3.2.so.0 (0x00007f3cedb54000)
        libwx_gtk3u_core-3.2.so.0 => /lib64/libwx_gtk3u_core-3.2.so.0 (0x00007f3ced200000)
        libwx_baseu_xml-3.2.so.0 => /lib64/libwx_baseu_xml-3.2.so.0 (0x00007f3cedb42000)
        libwx_baseu_net-3.2.so.0 => /lib64/libwx_baseu_net-3.2.so.0 (0x00007f3cedade000)
        libwx_baseu-3.2.so.0 => /lib64/libwx_baseu-3.2.so.0 (0x00007f3cece00000)

But it's only a suspicion.  It could instead be a difference in some
other set of libraries, or in the compiler version used.  Or it could be
a timing issue that either is or is not triggered by generated code.
Perhaps the generated code for your semi-reproducible case hits right
on the edge of that timing. 
In any case I don't think the difference indicates a change in the gnuplot
source code.

Having said all that, if someone pins down the actual point of failure
or timing issue, perhaps we could code around it.   

	Ethan

> On some invocations the frequency bars appear in the first sub-plot, 
> on other invocations they do not appear, the upper plot shows no 
> data. But if I write the plot to file using, say, pngcairo or 
> pdfcairo, the first sub-plot is always OK.
> 
> Am I misusing multiplot or might there be a bug here?
> 
> 

strange multiplot behavior with wxt

[Allin C. <cot...@wf...>]

I'm puzzled by a multiplot example where the output seems to be 
non-deterministic -- though the fault may be in my script 
(strange_multi.plt, attached).

I'm on Linux, using gnuplot 5.4.3 and also 5.4.8, using 
the wxt terminal for screen display with the command line

gnuplot strange_multi.plt -persist

On some invocations the frequency bars appear in the first sub-plot, 
on other invocations they do not appear, the upper plot shows no 
data. But if I write the plot to file using, say, pngcairo or 
pdfcairo, the first sub-plot is always OK.

Am I misusing multiplot or might there be a bug here?

-- 
Allin Cottrell
Department of Economics
Wake Forest University

Re: Re: Commit [36a21d]

[Tatsuro M. <tma...@ya...>]

Ethan
Thanks for the reply.

>  Under Windows + mingw it is
This is also true on cygwin.
 
Under Windows + mingw and cygwin it is

Tatsuro
> ----- Original Message -----
> 
> From: "Ethan A Merritt" <me...@uw...>
> To: "Tatsuro MATSUOKA" <tma...@ya...>
> Cc: "beta" <gnu...@li...>
> Date: 2023/06/25 日 00:57
> Subject: Re: Commit [36a21d]
> 
> 
> Thank you for catching that.  The "-fPIC" flag should have been added
> to the previous example command, the one for linux.
> 
>   Ethan
> 
> On Fri, Jun 23, 2023 at 11:55 PM Tatsuro MATSUOKA via gnuplot-beta
> <gnu...@li...> wrote:
> >
> > I found in Commit [36a21d]
> > +++ b/demo/plugin/README
> > @@ -23,12 +23,12 @@
> >
> >  Under Windows + mingw it is
> >
> > -    gcc -shared -lm -o newprogram.dll newprogram.c
> > +    gcc -shared -lm -fPIC -o newprogram.dll newprogram.c
> >
> > As far as I know, -fPIC is not effective on WIndows.
> > https://urldefense.com/v3/__https://bugs.webkit.org/show_bug.cgi?id=163949__;!!K-Hz7m0Vt54!jBF_aoqeTsaFf72wifSDQ-GopXPU9-4IpIwvRSENh-r0xtrJ-MD4LVRY3HP5A_6Br1lExVzif4elmnlMcXpyHjzUyma7UQ$
> >
> > Tatsuro
> >
> >
> >
> > _______________________________________________
> > gnuplot-beta mailing list
> > gnu...@li...
> > Membership management via: https://urldefense.com/v3/__https://lists.sourceforge.net/lists/listinfo/gnuplot-beta__;!!K-Hz7m0Vt54!jBF_aoqeTsaFf72wifSDQ-GopXPU9-4IpIwvRSENh-r0xtrJ-MD4LVRY3HP5A_6Br1lExVzif4elmnlMcXpyHjxbN-9g9w$
> 
> 
> 
> -- 
> Ethan A Merritt
> Department of Biochemistry
> University of Washington, Seattle

Re: Commit [36a21d]

[Ethan A M. <me...@uw...>]

Thank you for catching that.  The "-fPIC" flag should have been added
to the previous example command, the one for linux.

  Ethan

On Fri, Jun 23, 2023 at 11:55 PM Tatsuro MATSUOKA via gnuplot-beta
<gnu...@li...> wrote:
>
> I found in Commit [36a21d]
> +++ b/demo/plugin/README
> @@ -23,12 +23,12 @@
>
>  Under Windows + mingw it is
>
> -    gcc -shared -lm -o newprogram.dll newprogram.c
> +    gcc -shared -lm -fPIC -o newprogram.dll newprogram.c
>
> As far as I know, -fPIC is not effective on WIndows.
> https://urldefense.com/v3/__https://bugs.webkit.org/show_bug.cgi?id=163949__;!!K-Hz7m0Vt54!jBF_aoqeTsaFf72wifSDQ-GopXPU9-4IpIwvRSENh-r0xtrJ-MD4LVRY3HP5A_6Br1lExVzif4elmnlMcXpyHjzUyma7UQ$
>
> Tatsuro
>
>
>
> _______________________________________________
> gnuplot-beta mailing list
> gnu...@li...
> Membership management via: https://urldefense.com/v3/__https://lists.sourceforge.net/lists/listinfo/gnuplot-beta__;!!K-Hz7m0Vt54!jBF_aoqeTsaFf72wifSDQ-GopXPU9-4IpIwvRSENh-r0xtrJ-MD4LVRY3HP5A_6Br1lExVzif4elmnlMcXpyHjxbN-9g9w$



-- 
Ethan A Merritt
Department of Biochemistry
University of Washington, Seattle

Commit [36a21d]

[Tatsuro M. <tma...@ya...>]

I found in Commit [36a21d]
+++ b/demo/plugin/README
@@ -23,12 +23,12 @@
 
 Under Windows + mingw it is
 
-    gcc -shared -lm -o newprogram.dll newprogram.c
+    gcc -shared -lm -fPIC -o newprogram.dll newprogram.c

As far as I know, -fPIC is not effective on WIndows.
https://bugs.webkit.org/show_bug.cgi?id=163949

Tatsuro

Re: SVG pixelation (was pdfcairo terminal seams between bars in the colorbox)

[Dima K. <gn...@di...>]

Ethan A Merritt <me...@uw...> writes:

> I looked into it a bit further.
>
> The thing is, the two hinting properties you found are not part of the
> SVG standard (version 2).  They are instead CSS style hints to the browser.
> Chrome and Firefox apparently dive into an included SVG file and extract
> the CSS style hints from there. SVG-aware programs that are not browsers,
> like inkscape or gimp, could do the same but the motivation for doing
> so would come from imitating Chrome or Firefox rather than adhering to
> the SVG standard.  Inkscape does, but gimp doesn't. 
>
>> I'll file feature-request bug reports for those two projects.
>> But the patch should probably still go in: supporting firefox and chrome
>> is much better than supporting nothing.

> For me personally, the support from inkscape is just as important as
> from the browsers.  I use it to prepare figures for publication.
> And it's still true that you can work around this
> by using "with image pixels" for gnuplot heat maps. 
>
> What about lobbying the SVG working group to add an equivalent
> to the SVG spec?  All the various support libraries would then
> eventually fall in line.  As of SVG 2.0 you only get a choice
> between "optimizeSpeed" and "optimizeQuality", both of which
> are so vague as to be useless for our purpose.
>
>     https://svgwg.org/svg2-draft/painting.html#ImageRendering

That's interesting. Not sure what it means for us in practical terms.
The patch in this thread should cover most browsers. Most non-browser
applications I've seen use rsvg: bug report here

  https://gitlab.gnome.org/GNOME/librsvg/-/issues/985

I guess QT has its own implementation too; I ran out of steam, and
haven't pestered them. I don't know what libreoffice does, but it does
the right thing on my machine. And I don't know how to lobby the SVG
working group.

I would merge the patch in this thread, maybe file a bug report against
QSvg, and move on.

Re: SVG pixelation (was pdfcairo terminal seams between bars in the colorbox)

[Ethan A M. <me...@uw...>]

On Sunday, 18 June 2023 23:55:07 PDT Dima Kogan wrote:
> Ethan A Merritt <me...@uw...> writes:
> 
> > As tested here those attributes are indeed honored by firefox and chrome.
> > Mixed success on other svg handling programs.
> 
> I'm guessing that adding support for either image-rendering=crips-edge
> or image-rendering=pixelated into librsvg and QSvg would make all of
> these work. But it looks like these two libraries simply don't support
> either option yet.

I looked into it a bit further.

The thing is, the two hinting properties you found are not part of the
SVG standard (version 2).  They are instead CSS style hints to the browser.
Chrome and Firefox apparently dive into an included SVG file and extract
the CSS style hints from there. SVG-aware programs that are not browsers,
like inkscape or gimp, could do the same but the motivation for doing
so would come from imitating Chrome or Firefox rather than adhering to
the SVG standard.  Inkscape does, but gimp doesn't. 

> I'll file feature-request bug reports for those two projects.
> But the patch should probably still go in: supporting firefox and chrome
> is much better than supporting nothing.

For me personally, the support from inkscape is just as important as
from the browsers.  I use it to prepare figures for publication.
And it's still true that you can work around this
by using "with image pixels" for gnuplot heat maps. 

What about lobbying the SVG working group to add an equivalent
to the SVG spec?  All the various support libraries would then
eventually fall in line.  As of SVG 2.0 you only get a choice
between "optimizeSpeed" and "optimizeQuality", both of which
are so vague as to be useless for our purpose.

    https://svgwg.org/svg2-draft/painting.html#ImageRendering

Re: SVG pixelation (was pdfcairo terminal seams between bars in the colorbox)

[Dima K. <gn...@di...>]

Ethan A Merritt <me...@uw...> writes:

> As tested here those attributes are indeed honored by firefox and chrome.
> Mixed success on other svg handling programs.

I'm guessing that adding support for either image-rendering=crips-edge
or image-rendering=pixelated into librsvg and QSvg would make all of
these work. But it looks like these two libraries simply don't support
either option yet.

I'll file feature-request bug reports for those two projects.

But the patch should probably still go in: supporting firefox and chrome
is much better than supporting nothing.

SVG pixelation (was pdfcairo terminal seams between bars in the colorbox)

[Ethan A M. <me...@uw...>]

> The svg terminal currently produces images with no image-rendering
> annotations. And most browsers render images with bilinear interpolation
> by default, which makes low-res images look very wrong.

> My friend figured out how to get svg to work
> in both firefox and chrome, and I'm attaching a patch that does that.
> The description is in the patch.

[snip patch that adds attributes
 <image image-rendering='crisp-edges' style='image-rendering: pixelated' ...
]

Thanks.  That's a good start.

As tested here those attributes are indeed honored by firefox and chrome.
Mixed success on other svg handling programs.

Works:
   chrome
   firefox
   inkscape/inkview
   ImageMagick

Fails:
   konqueror
   LibreOffice / soffice
   gwenview
   gimp

Unknown (did not try):
   eog
   geeqie

	Ethan

Re: pdfcairo terminal; seams between bars in the colorbox

[Dima K. <gn...@di...>]

Thanks for merging those. My friend figured out how to get svg to work
in both firefox and chrome, and I'm attaching a patch that does that.
The description is in the patch.

Re: pdfcairo terminal; seams between bars in the colorbox

[Ethan A M. <me...@uw...>]

OK, it's pushed to the development branch for 6.1.

I revised the top-level criterion for choosing the new method that
uses term->image rather than the old method that uses term->fille_polygon.
Commit message:

    Rather than selecting the colorbox-as-image code for all terminals
    that support term->image, make it dependent on a terminal flag.
    Instead of
        if (term->image) ...
    use
        if ((term->flags & TERM_COLORBOX_IMAGE))
    
    The svg, domterm, and canvas terminals definitely do not want this flag.
    The pdfcairo terminal definitely does want it.
    
    For now the flag is set by these terminals
            epscairo cairolatex pdfcairo qt
    
    Other terminals can be added later by setting TERM_COLOR_BOX_IMAGE in the
    corresponding driver source.  Possible candidates
            tikz wxt

The PostScript-based terminals are not affected since colorbox generation
code is done in the PostScript header code rather than in the core C code.

Thanks for working on this.

    Ethan

On Thursday, 15 June 2023 22:13:47 PDT Ethan A Merritt wrote:
> On Thursday, 15 June 2023 15:39:22 PDT Dima Kogan wrote:
> > 
> >   dima@shorty:~/projects/gnuplot$ git show-branch HEAD origin/master
> >   ! [HEAD] smooth colorbox: render as a bitmap if the terminal supports it
> >    ! [origin/master] Do not report terminal type to user until after initialization is complete
> >   --
> >   +  [HEAD] smooth colorbox: render as a bitmap if the terminal supports it
> >   +  [HEAD^] no-op cleanup: consolidated next-colorbox-slice logic
> >   +  [HEAD~2] no-op cleanup: consolidated draw-colorbox-slice logic
> >   +  [HEAD~3] no-op cleanup: consolidated colorbox bounds logic
> >   +  [HEAD~4] no-op cleanup: consolidated steps-in-colorbox logic
> >    + [origin/master] Do not report terminal type to user until after initialization is complete
> >    + [origin/master^] VMS cleanup
> >   ++ [HEAD~5] qt:  qt4 was apparently weak at int->double promotion
> > 
> > I'm attaching the patches to this email.
> 
> I like it so far.
> I will try to think of more tests to run.
> 
> > > OK. The other alternative is to make this specific to cairopdf. Unless
> > > you know of other terminals that suffer from the same problem?
> 
> [snip descroption of svg problems with small number of image pixels]
> > So if we
> > can fix this, I'd say we should do the colorbox-image thing whenever
> > available. But if we can't, maybe limiting it to only cairopdf (or NOT
> > svg) is the answer.
> > 
> > Let me ask a web-dev friend about how to make svg work right in all
> > browsers. There's probably a way.
> > 
> > The current set of attached patches does NOT limit itself to cairopdf
> 
> I think it benefits all the cairo terminals, or at least doesn't hurt.
> Beyond that, well - that's what I am trying to test.
> I will add a FIXME comment.
>  
> > Ah yes. This is exactly what I mentioned above. I don't remember any
> > terminal other than svg having this problem. But I certainly haven't
> > touched all of them.
> 
> The HTML5 canvas terminal has the same problem: browsers apply
> inter-pixel interpolation, details of which are poorly documented.
> 
>    cheers
>    Ethan
> 
> 
> 


-- 
Ethan A Merritt
Biomolecular Structure Center,  K-428 Health Sciences Bldg
MS 357742,   University of Washington, Seattle 98195-7742

Re: pdfcairo terminal; seams between bars in the colorbox

[Petr M. <mi...@ph...>]

> Oh, and two more notes, where the old code had off-by-one errors
>
> - There old code was set up to overlap each colorbox by extending the
>  bottom/right edge of each slice: GPMIN(xy_to,xy2+1)

Please try also "set term postscript color" output to see what your pdf 
viewers do there. These "write stripes issue" and antialiasing was an issue 
two decades ago. It depended whether you have switched on or off antialising 
in the pdf/ps viewer. Hm, I see that current viewer do not have such an option 
any longer (even though ghostscript can be run with different options), so you 
cannot check the problem directly in the viewer.

What you can try as the lowest-level approach is

gs a.pdf
gs -dDOINTERPOLATE a.pdf

Pdf itself can manipulate antialiasing directly, such as 
3 0 obj
<</AntiAlias false>>
but that's probably possible to insert directy in the poppler or 
cairo library?

> - The old code was computing the colors for each slice like this:
>
>    for (i = 0; i < steps; i++)
>      gray = i / (double)steps;
>
>  This is another off-by-one error. The colors in the colorbox spanned
>  [0,1-1/steps] instead of [0,1]. My patch fixes this one.
>
>  There's some funky logic in quanize_gray() that maybe is trying to
>  deal with this in the limited-max-colors case, but it's not commented,
>  and applies to only this case. Code:
>
>    qgray = floor(gray * sm_palette.use_maxcolors)
> 		        / (sm_palette.use_maxcolors-1);

The colorbox code must work correctly for limited number of colors, try
 	set palette maxcolors 4
 	splot x*y with pm3d
Thus I think the code is correct, if you see exactly 4 color boxes in the 
colorbox.

---
Petr Mikulik

Re: pdfcairo terminal; seams between bars in the colorbox

[Ethan A M. <me...@uw...>]

On Thursday, 15 June 2023 15:39:22 PDT Dima Kogan wrote:
> 
>   dima@shorty:~/projects/gnuplot$ git show-branch HEAD origin/master
>   ! [HEAD] smooth colorbox: render as a bitmap if the terminal supports it
>    ! [origin/master] Do not report terminal type to user until after initialization is complete
>   --
>   +  [HEAD] smooth colorbox: render as a bitmap if the terminal supports it
>   +  [HEAD^] no-op cleanup: consolidated next-colorbox-slice logic
>   +  [HEAD~2] no-op cleanup: consolidated draw-colorbox-slice logic
>   +  [HEAD~3] no-op cleanup: consolidated colorbox bounds logic
>   +  [HEAD~4] no-op cleanup: consolidated steps-in-colorbox logic
>    + [origin/master] Do not report terminal type to user until after initialization is complete
>    + [origin/master^] VMS cleanup
>   ++ [HEAD~5] qt:  qt4 was apparently weak at int->double promotion
> 
> I'm attaching the patches to this email.

I like it so far.
I will try to think of more tests to run.

> > OK. The other alternative is to make this specific to cairopdf. Unless
> > you know of other terminals that suffer from the same problem?

[snip descroption of svg problems with small number of image pixels]
> So if we
> can fix this, I'd say we should do the colorbox-image thing whenever
> available. But if we can't, maybe limiting it to only cairopdf (or NOT
> svg) is the answer.
> 
> Let me ask a web-dev friend about how to make svg work right in all
> browsers. There's probably a way.
> 
> The current set of attached patches does NOT limit itself to cairopdf

I think it benefits all the cairo terminals, or at least doesn't hurt.
Beyond that, well - that's what I am trying to test.
I will add a FIXME comment.
 
> Ah yes. This is exactly what I mentioned above. I don't remember any
> terminal other than svg having this problem. But I certainly haven't
> touched all of them.

The HTML5 canvas terminal has the same problem: browsers apply
inter-pixel interpolation, details of which are poorly documented.

   cheers
   Ethan

Re: pdfcairo terminal; seams between bars in the colorbox

[Dima K. <gn...@di...>]

Ethan A Merritt <me...@uw...> writes:

>> The patches I'm proposing are here:
>> 
>>   https://github.com/dkogan/gnuplot/tree/colorbox_smooth
>
> I do not know how to generate the relevant patches via github. It
> says: "master and colorbox_smooth are entirely different commit
> histories" Could you please send me the relevant patch series as
> generated by "git format-patch"?

I'm only two patches out-of-date:

  dima@shorty:~/projects/gnuplot$ git show-branch HEAD origin/master
  ! [HEAD] smooth colorbox: render as a bitmap if the terminal supports it
   ! [origin/master] Do not report terminal type to user until after initialization is complete
  --
  +  [HEAD] smooth colorbox: render as a bitmap if the terminal supports it
  +  [HEAD^] no-op cleanup: consolidated next-colorbox-slice logic
  +  [HEAD~2] no-op cleanup: consolidated draw-colorbox-slice logic
  +  [HEAD~3] no-op cleanup: consolidated colorbox bounds logic
  +  [HEAD~4] no-op cleanup: consolidated steps-in-colorbox logic
   + [origin/master] Do not report terminal type to user until after initialization is complete
   + [origin/master^] VMS cleanup
  ++ [HEAD~5] qt:  qt4 was apparently weak at int->double promotion

I'm attaching the patches to this email.


> OK. The other alternative is to make this specific to cairopdf. Unless
> you know of other terminals that suffer from the same problem?

Actually.... You're right. It's only semi-related to THIS discussion,
but let me bring that up.

The svg terminal currently produces images with no image-rendering
annotations. And most browsers render images with bilinear interpolation
by default, which makes low-res images look very wrong. I looked at this
a while back, and, there was no browser-independent way to turn that
off. And I think there still isn't. I'm attaching an svg of this gnuplot
script made WITH the new patches AND by manually adding
image-rendering="crisp-edges" to it:

  set output "/tmp/tst-colorbox-log.svg"
  set terminal svg
  set palette maxcolors 5
  set logscale cb
  set cbrange [1:10]
  plot 5 with points palette

If you open that in firefox, it should look good, but if you open it in
chrome, it won't. To make it work in chrome, "crisp-edges" needs to
become "pixelated". I don't know how to make it work with both. So if we
can fix this, I'd say we should do the colorbox-image thing whenever
available. But if we can't, maybe limiting it to only cairopdf (or NOT
svg) is the answer.

Let me ask a web-dev friend about how to make svg work right in all
browsers. There's probably a way.

The current set of attached patches does NOT limit itself to cairopdf


>> I don't see this. I'm looking at
>> 
>>   https://gnuplot.sourceforge.net/demo/heatmaps.html
>
> That is the png version; it doesn't show the effect I'm talking about.
> Have a look at the svg version
>     https://gnuplot.sourceforge.net/demo_svg/heatmaps.html
>
> In particular the plot with title
>     Compare "image" and "image pixels"

Ah yes. This is exactly what I mentioned above. I don't remember any
terminal other than svg having this problem. But I certainly haven't
touched all of them.

Re: pdfcairo terminal; seams between bars in the colorbox

[Ethan A M. <me...@uw...>]

On Wednesday, 14 June 2023 10:43:26 PDT Dima Kogan wrote:
> Hi Ethan. Thanks for looking at this.
> 
> If possible I'd like to get the colorbox-as-bitmap logic integrated. I'm
> all for filing bugs in popper and mupdf and all those, but I don't have
> an understanding of what's going on there at all, or if these are even
> bugs. So hopefully we can get another implementation of gnuplot colorbox
> rendering we're happy with.
> 
> The patches I'm proposing are here:
> 
>   https://github.com/dkogan/gnuplot/tree/colorbox_smooth

I do not know how to generate the relevant patches via github.
It says:  "master and colorbox_smooth are entirely different commit histories"
Could you please send me the relevant patch series as generated by
"git format-patch"?
  
> > This is already possible in current gnuplot, although to be fair it is not
> > hooked up automatically to colorbox generation and maybe not well
> > enough documented.   See "named_palettes.dem" for an example.
> > In a nutshell:
> > 1) Define your palette ("set palette ...")
> > 2) Save it as a colormap ("set colormap new NAME")
> > 3) Use the colormap as a pixmap
> >    ("set pixmap 1 colormap NAME at screen 0.92, 0.3  size screen 0.03, 0.4")
> 
> OK. Yes. I have confirmed that this uses a bitmap for the colorbox. I'm
> not sure what bearing this has. Are you saying the code that does this
> should be consolidated with the new code that does the bitmap colorbox?

Mostly I meant that the good/bad effects of this approach can be tested
using the code already there.  

> > However this approach has at least two limitations,
> > whether done via colormap+pixmap or via your proposed new code.
> > 1) Not all terminals support it
> 
> The patches above fall back on the discrete slices if term->image==NULL

OK.  The other alternative is to make this specific to cairopdf.
Unless you know of other terminals that suffer from the same problem?

> > 2) It does not handle nonlinear color mappings,
> >    e.g. "set log cb"
> 
> I just tested it. It looks the same before/after. The logscale affects
> the tic marks on the colorbox, not the color box image.

You are right.  I was thinking of something else, but never mind.

> > There is a third issue also that would apply to using it for svg or
> > canvas output. See the 7th plot in heatmaps.dem. The display programs
> > for these modes apply an averaging/blurring function across image
> > pixels, which means that discrete palette colors would be muddied or
> > mangled entirely if represented by image pixels rather than by
> > separate solid color rectangles.
> 
> I don't see this. I'm looking at
> 
>   https://gnuplot.sourceforge.net/demo/heatmaps.html

That is the png version; it doesn't show the effect I'm talking about.
Have a look at the svg version
    https://gnuplot.sourceforge.net/demo_svg/heatmaps.html

In particular the plot with title
    Compare "image" and "image pixels"

> Can you try the patches?

If you send them as patches, yeah.
Or upload them to the patch tracker
    https://sourceforge.net/p/gnuplot/patches/

	Ethan

Re: pdfcairo terminal; seams between bars in the colorbox

[Dima K. <gn...@di...>]

Hi Ethan. Thanks for looking at this.

If possible I'd like to get the colorbox-as-bitmap logic integrated. I'm
all for filing bugs in popper and mupdf and all those, but I don't have
an understanding of what's going on there at all, or if these are even
bugs. So hopefully we can get another implementation of gnuplot colorbox
rendering we're happy with.

The patches I'm proposing are here:

  https://github.com/dkogan/gnuplot/tree/colorbox_smooth

This is the "colorbox_smooth" branch of that repo. There are several
no-op cleanup patches that consolidate duplicated logic. And one patch
to use the colorbox-as-bitmap path if term->image is available. If it's
not available we fall back to the existing implementation.

I think this works. I tested it with horizontal/vertical colorboxes and
logscales. Before/after patch behavior is the same, except for the
anti-aliasing artifacts we're trying to get rid of.

More notes inline.


Ethan A Merritt <me...@uw...> writes:

> On Sunday, 11 June 2023 16:22:01 PDT Dima Kogan wrote:

> This is already possible in current gnuplot, although to be fair it is not
> hooked up automatically to colorbox generation and maybe not well
> enough documented.   See "named_palettes.dem" for an example.
> In a nutshell:
> 1) Define your palette ("set palette ...")
> 2) Save it as a colormap ("set colormap new NAME")
> 3) Use the colormap as a pixmap
>    ("set pixmap 1 colormap NAME at screen 0.92, 0.3  size screen 0.03, 0.4")

OK. Yes. I have confirmed that this uses a bitmap for the colorbox. I'm
not sure what bearing this has. Are you saying the code that does this
should be consolidated with the new code that does the bitmap colorbox?


> However this approach has at least two limitations,
> whether done via colormap+pixmap or via your proposed new code.
> 1) Not all terminals support it

The patches above fall back on the discrete slices if term->image==NULL


> 2) It does not handle nonlinear color mappings,
>    e.g. "set log cb"

I just tested it. It looks the same before/after. The logscale affects
the tic marks on the colorbox, not the color box image.


> There is a third issue also that would apply to using it for svg or
> canvas output. See the 7th plot in heatmaps.dem. The display programs
> for these modes apply an averaging/blurring function across image
> pixels, which means that discrete palette colors would be muddied or
> mangled entirely if represented by image pixels rather than by
> separate solid color rectangles.

I don't see this. I'm looking at

  https://gnuplot.sourceforge.net/demo/heatmaps.html

The 7th plot is the last one. I don't think that's what you mean. If I
run the "headtmaps.dem" script with the current gnuplot in git (the
website should match!) I see a different sequence. The 7th one is the
one with the two images without any colorboxes at all. But they all look
right in any case.

Can you try the patches?

Thanks

Re: pdfcairo terminal; seams between bars in the colorbox

[Ethan A M. <me...@uw...>]

New information:

I found a chunk of code in cairo terminal driver from way back when
that was intended to deal with an old version of the cairo library
(2010 or earlier).

good news:
The one line patch below fixes or at least improves the banding
artifact in pdf output for almost all plots I have tried.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
diff --git a/term/cairo.trm b/term/cairo.trm
index 9df11cc79..8ab9698cb 100644
--- a/term/cairo.trm
+++ b/term/cairo.trm
@@ -775,7 +775,7 @@ void cairotrm_init()
 		/* disable OPERATOR_SATURATE, not implemented in cairo pdf backend.
 		 * Unfortunately the can result in noticeable seams between adjacent
 		 * polygons. */
-		plot.polygons_saturate = FALSE;
+		plot.polygons_saturate = TRUE;
 		/* Empirical correction to make pdf output look more like wxt and png */
 		plot.dashlength /= 2;
 	} else if (!strcmp(term->name,"pngcairo") || !strcmp(term->name, "kittycairo")) {
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

bad news:
It degrades text rendering in the pdf output.  I don't know why.
For at least one plot (walls.dem) it introduces a much worse artifact.
I don't know why on that one either.

There are other places in the cairo.trm code that were also
added apparently to accommodate this old cairo libary.
I will try to trace back what changed in cairo and why the
gnuplot terminal driver was trying to work around it.
The bad text rendering side effect is a show-stopper at the moment.
If I can disentangle that from the change to polygon rendering
I would be inclined to make the change in the gnuplot development
source and work on the resulting artifact in walls.dem separately.

I am dubious about the large block of code in gp_cairo.c that begins 
  /* this is meant to test Full-Scene-Anti-Aliasing by supersampling,
   * in association with CAIRO_ANTIALIAS_NONE a few lines below
   */
Before the change to plot.polygons_saturate it was never executed for
pdf output. Now it is.  Perhaps it can be replaced by something simpler?
The "supersampling" idea was never implemented, so maybe this code
never did work correctly.  On the other hand it is also used by
the wxt terminal and I haven't noticed problems there, so ....?

Anyhow, if you have any ideas or can find a way to remove the
text-rendering side effect that would be great.

   cheers,
      Ethan

Re: pdfcairo terminal; seams between bars in the colorbox

[Dima K. <gn...@di...>]

Ethan A Merritt <me...@uw...> writes:

> I do not see any such problem in the output from either pdflatex or lualatex as run here.
>
>     [~/temp] pdflatex --version 
>     pdfTeX 3.141592653-2.6-1.40.24 (TeX Live 2022/Mageia)
>     [~/temp] lualatex --version 
>     This is LuaHBTeX, Version 1.15.0 (TeX Live 2022/Mageia) 
>     Development id: 7509
>
> Output from pdflatex attached.

My bad. I gave you a pdf link in the last email that showed the issue.
But since then I've been working on my presentation, and in that process
re-generated my figures with the patched gnuplot, so the link in the
last email now points to a fixed image. This is what you used in your
attachment just now, and that's why it looks nice. You can confirm like
this:

  dima@shorty:~/projects/gnuplot$ pdfimages -list /tmp/dima.pdf

  page   num  type   width height color comp bpc  enc interp  object ID x-ppi y-ppi size ratio
  --------------------------------------------------------------------------------------------
     1     0 image      60    40  rgb     3   8  image  no        42  0    25    29 3602B  50%
     1     1 image       1   128  rgb     3   8  image  no        43  0     8    96  395B 103%

The 1x128 bitmap embedded in the .pdf is the colorbar, and its presence
indicates a patched gnuplot.

I'm attaching the pre-fix pdf file. I suspect you will see issues with
it if you pdflatex and the okular the output.


> So again, I do not think the problem is in gnuplot but rather in the
> pdf processing stage that comes after that.

Maybe. But if the issues in the surrounding tools are so pervasive that
things are broken more often than not, I'd be interested in tweaking
gnuplot to work well in the world that we have.


> But I'm not all that hopeful; variations of this sort of aliasing
> glitch persisted for years (decades?) in ghostview / gv despite it
> being a known and well-characterized bug

I haven't been fighting it for as long, so I remain optimistic :)

You raised some great points in your other email. I'll need a few days
to look at those.

Thanks much.

Re: pdfcairo terminal; seams between bars in the colorbox

[Ethan A M. <me...@uw...>]

On Sunday, 11 June 2023 16:22:01 PDT Dima Kogan wrote:
> Ethan A Merritt <me...@uw...> writes:
> 
> > I may not understand what you are suggesting.
> 
> Hi. I guess I wasn't clear. The suggestion was that instead of creating
> 128 discrete bars with different colors we create a 128x1 bitmap and
> draw it as an image. The thought was that this would work because images
> look correct in pdfcairo, without the ugly border lines.

This is already possible in current gnuplot, although to be fair it is not
hooked up automatically to colorbox generation and maybe not well
enough documented.   See "named_palettes.dem" for an example.
In a nutshell:
1) Define your palette ("set palette ...")
2) Save it as a colormap ("set colormap new NAME")
3) Use the colormap as a pixmap
   ("set pixmap 1 colormap NAME at screen 0.92, 0.3  size screen 0.03, 0.4")

Because it is not hooked up automatically to colorbox generation,
you would have to position the pixmap on top of the colorbox.
I admit that would be a nuisance.  Maybe you would like to look into
adding it as a colorbox option?

However this approach has at least two limitations,
whether done via colormap+pixmap or via your proposed new code.
1) Not all terminals support it
2) It does not handle nonlinear color mappings,
   e.g. "set log cb"

There is a third issue also that would apply to using it for
svg or canvas output.  See the 7th plot in heatmaps.dem.
The display programs for these modes apply an averaging/blurring
function across image pixels, which means that discrete palette
colors would be muddied or mangled entirely if represented by
image pixels rather than by separate solid color rectangles.

For these reasons I do not think this approach is suitable as
a replacement for the current colorbox code.

On the other hand if you can figure out some change to the
cairo + pdf pathway that avoids triggering bugs in poppler or
other external pdf-handling programs, that would be a good thing.
Of course fixing or at least filing bug reports for those
external programs would also be a good thing.

    Ethan

> 
> I implemented this in the attached patch, and it works and looks good. I
> tried several gnuplot terminals (x11, qt, wxt, pdfcairo, png, svg), and
> it looks like it works there too. I haven't found anything that this
> breaks. The script I used to test:
> 
>   plot 'demo/blutux.rgb' binary array=(128,128) flipy format='%uchar' with rgbimage, 5 with points palette
> 
> I don't understand the gnuplot codebase deeply, so I might have missed
> things. Some questions:
> 
> - I patched draw_inside_colorbox_bitmap_smooth() only. There are other
>   flavors of this function (look where it is called), and I don't know
>   if the others should be updated in this way as well
> 
> - I have an extra "gray = 1 - gray;" that I don't understand. It makes
>   things correct, though
> 
> - The patch uses rgb unconditionally. I that going to break something in
>   some terminal?
> 
> - I copied most of the logic from the old function without an
>   understanding of what it does. Am I handling these properly?
> 
>     - sm_palette.use_maxcolors
>     - sm_palette.gradient_num
>     - sm_palette.positive
> 
> - If I "set colorbox horizontal" then the colorbox stays in the same
>   spot with the same shape, but I change the color sequence to move
>   horizontally. This happens even before my patch. I'm here:
> 
>     ae7dfa3f3..: Ethan A Merritt 2023-06-09 qt:  qt4 was apparently weak at int->double promotion
> 
>   Shouldn't the colorbox move below or above the plot, and reorient
>   itself? Or is it the user's job to set the geometry?
> 
> This makes my PDFs look nice. It'd be good if some version of this patch
> was merged.
> 
> Thanks
> 

Re: pdfcairo terminal; seams between bars in the colorbox

[Ethan A M. <me...@uw...>]

On Saturday, 10 June 2023 17:24:37 PDT Dima Kogan wrote:
> 
> Actually, okular doesn't fully work either.
> 
> I'm currently writing a presentation where I'm using pdflatex and I'm
> including gnuplot figures. In that context okular shows the bands also.
> Maybe pdflatex is also buggy, but if mupdf AND evince AND pdflatex have
> this bug, maybe we should try to work around it. gv is still ok.
> 
> You can see this for yourself probably. I downloaded this figure that
> was generated by gnuplot:
> 
>   https://urldefense.com/v3/__http://mrcal.secretsauce.net/external/figures/diff/diff-radius0-heatmap-splined-opencv8.pdf__;!!K-Hz7m0Vt54!lfFwuvLqC5ID1P35DtMpTPxIqwIhHjrPNe0d_k_Uqz17nVeq07BR5n-UHuapjVlxgaL_-jJNDFaxrT6V_3nGHg$ 
> 
> And ran "pdflatex tst.tex" where tst.tex is:
> 
>   \documentclass[presentation]{beamer}
>   \begin{document}
>   \includegraphics[keepaspectratio,width=0.9\textwidth,height=0.7\textheight]{/tmp/diff-radius0-heatmap-splined-opencv8.pdf}
>   \end{document}
> 
> The result is attached. I see the bands even with okular.

I do not see any such problem in the output from either pdflatex or lualatex as run here.

    [~/temp] pdflatex --version 
    pdfTeX 3.141592653-2.6-1.40.24 (TeX Live 2022/Mageia)
    [~/temp] lualatex --version 
    This is LuaHBTeX, Version 1.15.0 (TeX Live 2022/Mageia) 
    Development id: 7509
So again, I do not think the problem is in gnuplot but rather in the
pdf processing stage that comes after that.

... Which is not to say it is impossible to change gnuplot in such
a way that it does not trigger bugs in other programs.
But I'm not all that hopeful;  variations of this sort of aliasing glitch
persisted for years (decades?) in ghostview / gv despite it being a known
and well-characterized bug.

Output from pdflatex attached.

    Ethan

Re: gnuplot crashes if GNUPLOT_DRIVER_DIR is invalid

[Ethan A M. <me...@uw...>]

On Sunday, 11 June 2023 20:03:02 PDT Dima Kogan wrote:> 
> Reproducible like this:
> 
>   GNUPLOT_DRIVER_DIR='' ./gnuplot
> 
> It should report an error message, but not crash.
> 
> Fixed in the attached patch

Got it. Thanks.

Re: pdfcairo terminal; seams between bars in the colorbox

[Ethan A M. <me...@uw...>]

On Sunday, 11 June 2023 20:44:31 PDT Dima Kogan wrote:

> Sorry for yet another email. I was dogfooding the patch, and found a
> minor bug. Sometimes term->image is not available, and the new colorbox
> implementation crashes in that case. I added a check to the top of the
> function, and it's good now:
> 
>   static void
>   draw_inside_colorbox_bitmap_smooth()
>   {
>       if(term->image == NULL)
>           return;
>   .....
> 
> I hit this with the "dumb" terminal. I think not rendering the colorbox
> at all in that case is fine. Are there other terminals where term->image
> is unavailable, but drawing discrete slices would still have produced
> good output?

It will take a while for me to work my way through your proposed changes.
But this much I can answer quickly -
Yes there are terminals that support smooth palette color but not term->image().
At a minimum they include
 block
 cgm
 emf
 fig
 sixeltek

	Ethan