#2747 windows: error in LaTeX when gnuplot is used with (semi)log axis in pgfplots

[Stefan Pinnow]

If I run

%% ----- minimal example -----
% used PGFPlots v1.18.1
\documentclass[border=5pt]{standalone}
\usepackage{pgfplots}
    \pgfplotsset{
        compat=1.18,
    }
\begin{document}
\begin{tikzpicture}
    \begin{semilogxaxis}
        % always works
        \addplot+ [raw gnuplot, id=raw] gnuplot {
            set xrange [0.01:100];
            plot log10(x)
        };
        % doesn't work with gnuplot 6.02
        \addplot+ [domain=0.01:100, id=non-raw] gnuplot {log10(x)};
    \end{semilogxaxis}
\end{tikzpicture}
\end{document}
%% ---------------------------

with an up-to-date LaTeX (using pdfLaTeX or LuaLaTeX) using gp5410-win64-mingw.exe it works flawlessly. But if I do the same with gp602-win64-mingw.exe I get the error

! Text line contains an invalid character.
<read 2> g^^@
     n^^@u^^@p^^@l^^@o^^@t^^@ ^^@6^^@.^^@0^^@ ^^@p^^@a^^@t^^@c^^@h^^@l^^@e^^@v
^^@e^^@l^^@ ^^@2^^@
l.20 ...in=0.01:100, id=non-raw] gnuplot {log10(x)};

I am not sure if this is a problem of gnuplot or TikZ/pgfplots, but since the problem only occurs with gnuplot v6 I first filed the issue here (and also because pgfplots currently is unmaintained...).

Any ideas?

[Ethan Merritt]

I'm not on windows, and don't have pgfplots, so I can only offer a wild guess that it's an encoding mismatch. Also it's not clear to me which program produced the snippet of output you show, or what you used to dump it. As printed it looks like the second line of the gnuplot banner:
Version 6.0 patchlevel 2 last modified 2024-12-04 as if it were emitted in a Windows encoding user wide characters (4 bytes per character) but printed byte-by-byte as if were UTF8 or ISO8859-1 (one byte per character).

The obvious next thing to test is whether your TeX installation can handle output generated directly from gnuplot.

gnuplot> set term tikz standalone
gnuplot> set output 'foo.tex'
gnuplot> plot [0:10]log10(x)
gnuplot> quit
$ lualatex foo

[Stefan Pinnow]

I can produce the foo.tex file, but I can't run it because of several errors. But these are unrelated to gnuplot itself, but for the packages or their options.

But my TeX installation can handle output from gnuplot. Therefore I added the first addplot line (with the raw gnuplot option, which works perfectly fine with both stated gnuplot versions.

The only difference between the both \addplot lines is in how I pass options to gnuplot. In the first case I give them in gnuplot syntax directly while in the second case I give them in pgfplots syntax which are then converted to gnuplot options. This is done by creating a new file which in this case contains

set table "minimal.non-raw.table"; set format "%.5f"
set format "%.7e";; set samples 25; set dummy x; set logscale x 2.71828182845905; plot [x=0.01:100] log10(x);

I guess you know or can image, what then happens ;)

Interestingly there is created another file as well which in this case has the name minimal.vrs. Unfortunately I neither could find something about a VRS file in PGFPlots nor in the Gnuplot manual, so I don't have a clue who produces it.

But with your hint that the strange stuff reads as

gnuplot 6.0 patchlevel 2

this is exactly the content of the VRS file (for that version). And I also could find a view that corresponds to the strange LaTeX error when opening the file with WinMerge. In the attached WinMerge.pngscreenshot you see in the "green boxes" that the line endings in the files for v5 and v6 are different as well as left of that green boxes the automatically chosen encodings.

Showing the files with Notepad++ (see Notepad++.png attachment) the encodings seem to be found correctly (again in the status bars on the bottom). So it seems that LaTeX can't handle the UTF-16 Little Endian encoding. This assumption seems to be correct, if https://tex.stackexchange.com/a/554224/95441 is to be believed.

Can you confirm that the output encoding was changed? (I couldn't find any information on that in the release notes.) Is there an option to choose/set the output encoding?

[Ethan Merritt]

I now see that this seems to be the same problem as reported in Bug 2532
https://sourceforge.net/p/gnuplot/bugs/2532/
But that report from two years ago was seeing the problem with gnuplot 5.4.
So I don't think this is anything to do with the gnuplot version, although possibly it has to do with the specific windows executable of gnuplot being used. Please have a look at that earlier report. You may understand more of the discussion than I do, since I'm not on Windows.

[Stefan Pinnow]

You are right, that looks exactly like the same problem.
Unfortunately no solution to the problem was given. I had a look at the RELEASE_NOTES and found

Changes in 5.4.8

• FIX version reporting in Windows binaries
  No other changes

I tested the outputs of v5.4.3 up to v6.0.2. All v5.x write in "UTF-8" and all v6.x in "UTF-16 Little Endian" encoding. So unfortunately at least I don't see a chance to find out how bug 2532 was fixed. In addition, does that mean that the fix to the bug was backported?

[Ethan Merritt]

I don't think there is any evidence of a source-code level bug. If I understand the comments attached to Bug 2532, the issue was what encoding was used in the default run-time environment as opposed to the encoding assumed in the build-time environment. What exactly that means on Windows, I don't know. If there is a way to force all executables to default to the same encoding, that should do it. That's why I added a link to a Microsoft page that appears to give a recipe for setting the default to UTF-8. Did you try that?

[Stefan Pinnow]

but when I tested all the gnuplot versions, my run-time environment is always the same. But gnuplot v5 output is different from v6 output ...
I had another look at the link to the Microsoft page. But that is about "app development" and therefore should be unrelated. But anyway. I don't see anything there to try. The only thing that I see I could have tried would be to change the Windows setting given in the first "Note" block. But unfortunately I can't find it. (Maybe this is because I have a non-English Windows version and I am too bad to find a proper translation for that setting or even the setting doesn't exist any more. As a Linux user you might wonder about that, but these kind of settings change regularly in Windows (at leas nowadays).

I am still pretty sure that gnuplot (or something related to the installation environment, i.e. the mingw) is the root-cause for the problem.

[Ethan Merritt]

Possibly relevant:
https://learn.microsoft.com/en-us/windows/apps/design/globalizing/use-utf8-code-page