gnuplot-beta

bug in 5.4rc2 X11 terminal

[Pieter-Tjerk de B. <p.t...@ut...>]

Hello,

Something goes wrong with fonts or character encodings on the x11 terminal.
See the attached which is the result of simply  
  plot 1e6
The vertical labels have a few unintended characters instead of multiplication
signs (crosses).

Debian's stock gnuplot 5.2pl6 version displays the multiplication sign
correctly on the x11 terminal, as does 5.4rc2 with the Qt terminal.

Any idea what this could be? Any test I could do to help debug it?

Regards,
  Pieter-Tjerk

Re: bug in 5.4rc2 X11 terminal

[Ethan A M. <me...@uw...>]

On Tuesday, 23 June 2020 13:17:19 PDT Pieter-Tjerk de Boer wrote:
> Hello,
> 
> Something goes wrong with fonts or character encodings on the x11 terminal.
> See the attached which is the result of simply  
>   plot 1e6
> The vertical labels have a few unintended characters instead of multiplication
> signs (crosses).
> 
> Debian's stock gnuplot 5.2pl6 version displays the multiplication sign
> correctly on the x11 terminal, as does 5.4rc2 with the Qt terminal.
> 
> Any idea what this could be? Any test I could do to help debug it?

Most likely your gnuplot session is operating with encoding utf8,
but your x11 terminal font handling is not set up to handle it.
It is possible to set it up, but it's a horrible pain and depends on exactly
what fonts you have available.   Either switch to
     set encoding iso_8859_15    (probably)
or switch to another terminal that has kept up with the modern world
and knows how to handle utf8.  I long ago switched to from x11 to qt
for daily use, but wxt or even sixel might fit your environment better.

If you stick with x11  and iso_8859_15, remember to switch the
encoding back to utf8 for output to pdf or TeX.

	Ethan

> 
> Regards,
>   Pieter-Tjerk
> 

Re: bug in 5.4rc2 X11 terminal

[Dima K. <gn...@di...>]

I use Debian and the x11 terminal routinely, and have never seen this
issue.

Ethan: if there's some sort of utf-8 issue as you suspect, would it show
up in "show encoding"? I see this on my machine:

  gnuplot> show encoding

          nominal character encoding is default
          however LC_CTYPE in current locale is C

I guess the locale being "C" means I'm not doing unicode, and that's why
things work for me?

Re: bug in 5.4rc2 X11 terminal

[Pieter-Tjerk de B. <p.t...@ut...>]

On Tue, Jun 23, 2020 at 10:43:32PM -0700, Dima Kogan wrote:
> I use Debian and the x11 terminal routinely, and have never seen this
> issue.
> 
> Ethan: if there's some sort of utf-8 issue as you suspect, would it show
> up in "show encoding"? I see this on my machine:
> 
>   gnuplot> show encoding
> 
>           nominal character encoding is default
>           however LC_CTYPE in current locale is C

In Debian's native gnuplot 5.2pl6 I get:

  gnuplot> show encoding
          nominal character encoding is default
          however LC_CTYPE in current locale is C.UTF-8

while in 5.4rc2 I get:

  gnuplot> show encoding
          nominal character encoding is utf8
          however LC_CTYPE in current locale is C.UTF-8

Setting encoding to "default" rather than "utf8" indeed fixes the problem
I had in the x11 terminal on 5.4rc2.

> I guess the locale being "C" means I'm not doing unicode, and that's why
> things work for me?

Ah, interesting: if I start gnuplot 5.4rc2 with LC_CTYPE set to C, I get
the same as you:

  gnuplot> show encoding
          nominal character encoding is default
          however LC_CTYPE in current locale is C

So, somehow, Debian's native gnuplot sets encoding to default even in
the C.UTF-8 locale, while 5.4rc2 only does that in the C locale.

Thanks for the help; this provides me with sufficient ways to circumvent
the problem.

Regards,
  Pieter-Tjerk

Re: bug in 5.4rc2 X11 terminal

[Dima K. <gn...@di...>]

Pieter-Tjerk de Boer <p.t...@ut...> writes:

> So, somehow, Debian's native gnuplot sets encoding to default even in
> the C.UTF-8 locale, while 5.4rc2 only does that in the C locale.
>
> Thanks for the help; this provides me with sufficient ways to
> circumvent the problem.

For what it's worth, my local environment is set to use C instead of
UTF-8, which is probably why things consistently work here.

I tried to set the locale environment in various ways to reproduce the
issue you saw, and I can't. IF gnuplot 5.4 has this issue with the x11
terminal (requiring the user to explicitly turn off unicode), then I
think we should fix it in some way.

But I can't reproduce the problem, so maybe there isn't anything to fix?
Can anybody else see this issue?

Re: bug in 5.4rc2 X11 terminal

[Ethan A M. <me...@uw...>]

On Wednesday, 24 June 2020 09:04:32 PDT Dima Kogan wrote:
> Pieter-Tjerk de Boer <p.t...@ut...> writes:
> 
> > So, somehow, Debian's native gnuplot sets encoding to default even in
> > the C.UTF-8 locale, while 5.4rc2 only does that in the C locale.
> >
> > Thanks for the help; this provides me with sufficient ways to
> > circumvent the problem.
> 
> For what it's worth, my local environment is set to use C instead of
> UTF-8, which is probably why things consistently work here.
> 
> I tried to set the locale environment in various ways to reproduce the
> issue you saw, and I can't. IF gnuplot 5.4 has this issue with the x11
> terminal (requiring the user to explicitly turn off unicode), then I
> think we should fix it in some way.
> 
> But I can't reproduce the problem, so maybe there isn't anything to fix?
> Can anybody else see this issue?

Back when people cared about x11, the tools and fonts necessary to
set up utf8 support were maintained, albeit difficult to use.
Now I cannot get it to work at all.  I imagine it is still possible but
I cannot point to a recipe for how to do it.

I think the best advice is "don't use the x11 terminal".

	Ethan
  

Re: bug in 5.4rc2 X11 terminal

[Dima K. <gn...@di...>]

Ethan A Merritt <me...@uw...> writes:

> Back when people cared about x11, the tools and fonts necessary to set
> up utf8 support were maintained, albeit difficult to use. Now I cannot
> get it to work at all. I imagine it is still possible but I cannot
> point to a recipe for how to do it.
>
> I think the best advice is "don't use the x11 terminal".

Yeah, that's too bad, but I think we can do better than that advice
suggests.

I use the x11 terminal almost exclusively for one reason: it's
dramatically faster and lighter than all the alternatives. The
difference is very noticeable when X-forwarding or when repeatedly
replotting (interactively rotating a 3D plot, using feedgnuplot
--stream, etc).

So I really would like to keep this working reasonably. If we currently
have a unicode problem that makes tic mark labels render improperly, an
easy "fix" would be to not feed unicode to the x11 terminal. If somebody
has the time and motivation to fix it properly, that'd be awesome of
course, but just sending ascii across would solve the 99% case in the
meantime.

But I can't reproduce the issue in this report, and until that happens,
there's nothing to "fix".

Re: bug in 5.4rc2 X11 terminal

[Ethan A M. <me...@uw...>]

On Wednesday, 24 June 2020 10:20:48 PDT Dima Kogan wrote:
> Ethan A Merritt <me...@uw...> writes:
> 
> > Back when people cared about x11, the tools and fonts necessary to set
> > up utf8 support were maintained, albeit difficult to use. Now I cannot
> > get it to work at all. I imagine it is still possible but I cannot
> > point to a recipe for how to do it.
> >
> > I think the best advice is "don't use the x11 terminal".
> 
> Yeah, that's too bad, but I think we can do better than that advice
> suggests.
> 
> I use the x11 terminal almost exclusively for one reason: it's
> dramatically faster and lighter than all the alternatives. The
> difference is very noticeable when X-forwarding or when repeatedly
> replotting (interactively rotating a 3D plot, using feedgnuplot
> --stream, etc).
> 
> So I really would like to keep this working reasonably. If we currently
> have a unicode problem that makes tic mark labels render improperly, an
> easy "fix" would be to not feed unicode to the x11 terminal.

That is exactly what "set encoding XXX" does, for any XXX that is not utf8.

Or did you mean that <x11 terminal + non-ascii encoding> would be
a special case when generating tic labels? 
We generally try to keep terminal-specific tests out of the core code,
but we do have tests for TeX-based terminals so it's not an absolute
prohibition.

> If somebody
> has the time and motivation to fix it properly, that'd be awesome of
> course, but just sending ascii across would solve the 99% case in the
> meantime.
> 
> But I can't reproduce the issue in this report, and until that happens,
> there's nothing to "fix".

For a long time now utf8 has been the default on linux.
So I would expect that most linux users would have to deal with this
when using the x11 terminal with their default locale.  

	Ethan

Re: bug in 5.4rc2 X11 terminal

[Allin C. <cot...@wf...>]

On Wed, 24 Jun 2020, Ethan A Merritt wrote:

> For a long time now utf8 has been the default on linux.
> So I would expect that most linux users would have to deal with this
> when using the x11 terminal with their default locale.

It "just works" here. Xterm using utf8, gnuplot using utf8 (as 
everything should!). So the multiplication signs come out right on 
the axes if I do, e.g.,

set term x11
plot sin(x)*1.0e-5

The only slightly odd thing is what "show encoding" produces:

         nominal character encoding is default
         however LC_CTYPE in current locale is en_US.utf8

I'm not sure what the first line means, or what the force of the 
"however" is.

Allin Cottrell

Re: bug in 5.4rc2 X11 terminal

[Dima K. <gn...@di...>]

Ethan A Merritt <me...@uw...> writes:

> On Wednesday, 24 June 2020 10:20:48 PDT Dima Kogan wrote:
>> So I really would like to keep this working reasonably. If we currently
>> have a unicode problem that makes tic mark labels render improperly, an
>> easy "fix" would be to not feed unicode to the x11 terminal.
>
> That is exactly what "set encoding XXX" does, for any XXX that is not utf8.
>
> Or did you mean that <x11 terminal + non-ascii encoding> would be
> a special case when generating tic labels?
> We generally try to keep terminal-specific tests out of the core code,
> but we do have tests for TeX-based terminals so it's not an absolute
> prohibition.

I meant that if we have a known issue where utf8 breaks tic labels with
the x11 terminal, then we can do the "set encoding xxx" internally,
without making the user work around the bug.


> For a long time now utf8 has been the default on linux. So I would
> expect that most linux users would have to deal with this when using
> the x11 terminal with their default locale.

Yes, but I can't reproduce it. I tried today to take away my local
ascii-only setting, and it still seems to work. I even tried to "set
encoding utf8", and my tic labels STILL look ok. And I use gnuplot a LOT
on all sorts of boxes, and I've never seen this. So I would guess that
this isn't an issue that gnuplot has with the default settings for most
people.

Pieter-Tjerk: do you have any theories about what it is about your setup
that triggered this problem?