#415 Improve utf-8 treatment in interactive session on gnuplot for windows

[Tatsuro MATSUOKA]

I have discussed the problem of Japanese characters in gnuplot system between Unix and windows.

https://sourceforge.net/p/gnuplot/bugs/1577/

Now unicode is default in many systems, it is better to use utf-8 encoded messages in pause command or print command etcs on gnuplot for windows.

Any comments?

[Shigeharu TAKENO]

Attached patch is experimental for that purpose. Wgnuplot.exe and gnulot.exe applied it may translate the string on pause/print command for stderr to locale encoding according with the "set encoding" setting.

Well, the translation is done for WIN32 only in the patch, because I think it is not need for Unix. In many Unix terminal emulator we can change the character encoding by LANG variable and by the terminal emulator switch.

[Tatsuro MATSUOKA]

Thanks!

I have tested on the cvs source (the latest changelog 2015-03-19)

With the patch I can use utf-8 encoding characters in pause and print command in the interactive session for wgnuplot.exe and wgnuplot_pipes.exe.

[Tatsuro MATSUOKA]

The second attachments

[Tatsuro MATSUOKA]

The third attachment:

[Tatsuro MATSUOKA]

However, I cannnot represent utf-8 ended code on gnuplot.exe.
The code page set to 65001 using
gnuplot> !chcp 65001

The using type command, characters utf-8 encoded file were represented as expected on console.

gnuplot> ! type utf8.gp

However,

gnuplot> load 'utf8.gp'

The print command and pause command does not represent utf-8 characters.

See attached the file:

[Tatsuro MATSUOKA]

I add the below two lines for debug purpose.

ifdef WGP_CONSOLE

printf("output codepage is %d\n", GetConsoleOutputCP()); <----- add 1
printf("input code page is %d\n", GetConsoleCP());       <------ add2 
if (mode == 0) loccp = GetConsoleOutputCP(); /* output codepage */
else loccp = GetConsoleCP(); /* input code page */

The results are (after !chcp 65001)
gnuplot> load 'utf8.gp'
output codepage is 65001
input code page is 65001
譌
output codepage is 65001
input code page is 65001
･ｬｪ

The code page is recognized properly.
Mmmmm!

[Tatsuro MATSUOKA]

I have mislead for gnuplot.exe

I have set codepage to 932

gnuplot> load 'utf8.gp'
output codepage is 932
input code page is 932
日本語
output codepage is 932
input code page is 932
{

The print command seems to work but pause command does not work as expected.

There might be some special treatments for pause command message for gnuplot.exe.

[Shigeharu TAKENO]

Please modify

if (mode == 0) loccp = GetConsoleOutputCP();

to

if (mode == 0) loccp = _getmbcp();

Changing of codepage by "!chcp 65001" on gnuplot may be reset to 932 by setlocale() used in gnuplot. cf.
http://d.hatena.ne.jp/yamidori98/touch/searchdiary?word=*%5BWin%B3%AB%C8%AF%5

GetConsoleOutputCP() returns the codepage set by chcp, but we should get the default codepage by _getmbcp().

[Tatsuro MATSUOKA]

I have changed:
if (mode == 0) loccp = GetConsoleOutputCP();
to
if (mode == 0) loccp = _getmbcp();

Active code page: 65001
gnuplot> load 'utf8.gp'
日
{ <= pause -1 '日本語'
gnuplot>

現在のコード ページ: 932
gnuplot> load 'utf8.gp'
日本語
{ <= pause -1 '日本語'
gnuplot>

For 65001, print command was in failure and pause command was in failure for both cp 932 and 65001.

[Tatsuro MATSUOKA]

For pause command, without patch and using sjis encoded file.

sjis.gp

set encoding sjis
set terminal qt
print '日本語'
set title '日本語'
pause -1 '日本語'
plot sin(x)

sjis.gp

work fine for wgnuplot.exe but for gnuplot.exe

gnuplot> load 'sjis.gp'
日本語
{ <= for pause command

The above happens for 4.6.6, 5.0.0 (Both Bastian and my build) and 5.1.0 but work correctly for Kakuto's build (5.1.0). Perhaps the problem comes from MinGW complier.

I will file this bug to bug tracker.

[Tatsuro MATSUOKA]

The above seems to be happened with combination "pause -1", "Japanese Character" and "MinGW Complier"

I have filed to
https://sourceforge.net/p/gnuplot/bugs/1580/

Shigeharu's patch seems to be good.
I hope that Shigeharu's patch is attached to the CVS (5.1) branch.

[Tatsuro MATSUOKA]

After applying patch for command.c in
https://sourceforge.net/p/gnuplot/bugs/1580/

Shigeharu's patch works perfect on gnnuplot.exe built with MinGW.

[Ethan Merritt]

I have applied Shigeharu's patch to 5.1 and 5.0 cvs (will be included in 5.0.1)

[Tatsuro MATSUOKA]

I have confirmed that patch is attached to the cvs trees.

I have updated the binary distribution.
http://www.tatsuromatsuoka.com/gnuplot/Eng/winbin/

Thanks for Shigeharu and Ethan