Re: Some observations building gnuplot for win by gcc-4.9.2 (MinGW64)

[Tatsuro M. <tma...@ya...>]

----- Original Message -----
> From: Tatsuro MATSUOKA 
> To: Merritt Ethan 
> Cc: gnuplot-beta
> Date: 2015/3/4, Wed 18:45
> Subject: Re: Some observations building gnuplot for win by gcc-4.9.2 (MinGW64)
> 
> 
> 
> 
> 
> ----- Original Message -----
>>  From: sfeam 
>>  To: Tatsuro MATSUOKA 
>>  Cc: gnuplot-beta
>>  Date: 2015/3/4, Wed 12:34
>>  Subject: Re: Some observations building gnuplot for win by gcc-4.9.2 
> (MinGW64)
>> 
>>  On Wednesday, 04 March 2015 09:54:08 AM Tatsuro MATSUOKA wrote:
>>> 
>>>   ----- Original Message -----
>>>   >From: Ethan A Merritt 
>>>   >To: gnuplot-beta@ Tatsuro MATSUOKA 
>>>   >Date: 2015/3/4, Wed 05:29
>>>   >Subject: Re: Some observations building gnuplot for win by 
> gcc-4.9.2 
>>  (MinGW64)
>>>   >[snip]
>>>   > 
>>>   >> 32 bit
>>>   >> ******
>>>   >> gnuplot crashes by segmentation fault when one use cairo 
> based 
>>  terminals.
>>>   >> 
>>>   >> Segmentation fault : g_utf8_validate gutf8.c:1634 on MinGW-64 
> 32 
>>  bit gcc-4.9.2
>>>   >> 
>>>   >> I have met a  Segmentation fault at using gnuplot-5.1 built 
> by 
>>  myself on windows.
>>>   >> The Segmentation fault seems to be related with the cairo 
> based 
>>  terminals.
>>>   >> (The cairo based terminals use libglib.)
>>>   >> 
>>>   >> Compiler gcc-4.9.2 (MinGW-w64 32bit. dwarf, win32 thread).
>>>   >> Glib version : 2.42.1. Build from source.
>>>   >> 
>>>   >> gdb message:
>>>   >> 
>>>   >> Program received signal SIGSEGV, Segmentation fault.
>>>   >> 0x6862318f in g_utf8_validate (str=0x2888a68 "-1", 
>>  max_len=-1, end=0xffffffff)
>>>   >>     at ../../glib-2.42.1/glib/gutf8.c:1634
>>>   >> 1634        *end = p;
>>>   >> 
>>>   >> Program received signal SIGSEGV, Segmentation fault.
>>>   >> 0x6862318f in g_utf8_validate (str=0xb28a68 "-1", 
>>  max_len=-1, end=0xffffffff)
>>>   >>     at ../../glib-2.42.1/glib/gutf8.c:1634
>>>   >> 1634        *end = p;
>>>   >> 
>>>   >> 
>>>   >> Perhaps this is a bug of glib (2.42.1 and 2.33.14).
>>>   >> 
>>>   >> I have filed the issue to  the bug tracker of Gnome.
>>>   >>  
>>>   >> https://bugzilla.gnome.org/show_bug.cgi?id=745485
>>>   >> 
>>>   >> At this moment, I abandon the complier update. However, is 
> the 
>>  trouble 2 for 64 bit case not a issue of gnuplot code?
>>>   > 
>>>   >That is possible.
>>>   >Can you provide a more complete trace of the segfault that
>>>   >shows where in the gnuplot code this failure happened?
>>>   > 
>>>   >Ethan
>>>   > 
>>> 
>>> 
>>>   Thank you for your response:
>>> 
>>>   I copy the back trace.
>> 
>>>   For wxt terminal,
>>>   (gdb) bt
>>>   #0  0x6862318f in g_utf8_validate (str=0x2a97fc8 "-1", 
>>  max_len=-1,
>>>       end=0xffffffff) at ../../glib-2.42.1/glib/gutf8.c:1634
>>>   #1  0x00529d21 in gp_cairo_convert (plot=0x2a0e0f0, string=0x2a97fc8 
>>  "-1")
>>>       at ../../src/wxterminal/gp_cairo.c:737
>> 
>>  [snip]
>> 
>>  I think this indicates a bug in glib.
>>  Gnuplot is prepared for an error return from the call to g_utf8_validate,
>>  but apparently glib faults instead of cleanly returning an error.
>>  The gnuplot code is:
>>          if (g_utf8_validate(string, -1, NULL)) {
>>              string_utf8 = g_strdup(string);
>>          } else {
>>              charset = gp_cairo_get_encoding(plot);
>>              string_utf8 = g_convert(string, -1, "UTF-8", charset, 
> 
>>  &bytes_read, NULL, &error);
>>          }
>> 
>>  The strange thing is that the string being converted seems totally
>>  harmless: "-1".   I could more easily understand if the failure
>>  occurred for a string in, for example, SHIFT_JIS encoding since 
>>  I imagine the glib people have not tested that extensively.
>> 
>>  The only thing I can think of to try in the gnuplot code is to invert
>>  the order of the tests.  Something like
>> 
>>          charset = gp_cairo_get_encoding(plot);
>>          if (<some test on charset != UTF8>) {
>>              string_utf8 = g_convert(string, -1, "UTF-8", charset, 
> 
>>  &bytes_read, NULL, &error);
>>          } else if (g_utf8_validate(string, -1, NULL)) {
>>              string_utf8 = g_strdup(string);
>>          } else {
>>              more serious error code or failure
>>          }
>> 
>>  I am not sure what are the proper test in the pseudo-code above
>>  when run on Windows.
>> 
>>      Ethan
> 
> 
>  Thank you for your reply. I will consider <some test on charset != UTF8>.
> 
> Tatsuro


I have executed a short test using nmh
https://github.com/shnya/nmh

Perhaps it is better to use ICU - International Components for Unicode but
I used nmh for rough test.

Applied changes:

--- gp_cairo.orig.c2014-12-14 08:42:38.000000000 +0900
+++ gp_cairo.c2015-03-06 10:04:45.142151900 +0900
@@ -76,6 +76,7 @@
 
 #include <pango/pangocairo.h>
 #include <glib.h>
+#include <nmh.h>
 
 #ifdef _MSC_VER
 #define rint(x) floor((x)+0.5L)
@@ -732,14 +733,21 @@
 gsize bytes_read;
 GError *error = NULL;
 const char *charset = NULL;
+const unsigned char *ucharset = NULL;
 gchar * string_utf8;
 
-if (g_utf8_validate(string, -1, NULL)) {
-    string_utf8 = g_strdup(string);
-} else {
-    charset = gp_cairo_get_encoding(plot);
-    string_utf8 = g_convert(string, -1, "UTF-8", charset, &bytes_read, NULL, &error);
-}
+charset = gp_cairo_get_encoding(plot);
+ucharset = charset;
+fprintf(stderr, "%f\n", nmh_is_utf8(ucharset, strlen(ucharset)));
+       if (nmh_is_utf8(ucharset, strlen(ucharset))) { 
+fprintf(stderr, "%f\n", nmh_is_utf8(ucharset, strlen(ucharset)));
+   string_utf8 = g_convert(string, -1, "UTF-8", charset, &bytes_read, NULL, &error); 
+fprintf(stderr, "%f\n", nmh_is_utf8(ucharset, strlen(ucharset)));
+   } else if (g_utf8_validate(string, -1, NULL)) { 
+            string_utf8 = g_strdup(string); 
+        } else { 
+            fprintf(stderr, "more serious error code or failure \n");
+        } 
 
 /* handle error case */
 if (error != NULL) {


float nmh_is_utf8(const unsigned char *str, int size);
   argument:  str  input strings
              size size of input strings without NULL character. (wequals to strlen(str))
   retern value: 
            value=> 0 possibility of utf-8 is high; 
            value=< 0 possibility of utf-8 is low;
            value=> 1 possibility of utf-8n is high;

*****************************************
The results are:

Terminal type set to 'wxt'
gnuplot> plot sin(x)
[New Thread 21120.0x52b4]
[New Thread 21120.0x5228]
[New Thread 21120.0x4c80]
[New Thread 21120.0x4bdc]
[New Thread 21120.0x5148]
[New Thread 21120.0x52f8]
[New Thread 21120.0x3088]
[New Thread 21120.0x4ef8]
0.500000
0.500000

Program received signal SIGSEGV, Segmentation fault.
0x75d608ca in strncpy () from C:\Windows\syswow64\msvcrt.dll
(gdb) bt
#0  0x75d608ca in strncpy () from C:\Windows\syswow64\msvcrt.dll
#1  0x009e67f0 in ?? ()
#2  0x00000038 in ?? ()
#3  0x00000000 in ?? ()
(gdb)
**********************************
The first and second fprintf(stderr) are excuted but third one are not executed.
Therefore the segmentation fault is related to :
string_utf8 = g_convert(string, -1, "UTF-8", charset, &bytes_read, NULL, &error);


Regards

Tatsuro