Re: [ruby-gnome2-devel-en] String encoding problem with ruby 1.9.2p0

[Guillaume C. <gco...@gm...>]

>>   - Ruby String passed to GTK+ then retrieved from GTK+ (my case in
>> GtkTreeStore, cases when reading GtkLabel, etc)
>>
>>   - default stuff from GTK+ (not much I guess, and much probably in
>> US-ASCII - some people have actual ideas about it?)

I forgot to handle cases of Strings got from GTK+ to represent file
paths (from GtkFileChooser for example). There, the system encoding is
used and we must handle that specifically.

>> So my patch converts Ruby String to UTF-8 if it's not already in UTF-8
>> before passing to GTK+; and properly creates an UTF-8 Ruby String when
>> retrieving from GTK+.
>
> I almost agree with your patch but I have some comments.

Thanks for the fast review :)

>  * It's true that GTK+ uses UTF-8 as it's internal/external
>    encoding as Mike said. It's OK to convert Ruby String to
>    UTF-8 for passing to GTK+.
>
>  * But there are some cases that GTK+ doesn't return
>    UTF-8. e.g. g_convert()(*1), g_utf8_to_utf16()(*2) and
>    so on. So, we need to provide an API for specify
>    encoding like CSTR2RVAL_ENCODING().
>
>    (*1) http://library.gnome.org/devel/glib/stable/glib-Character-Set-Conversion.html#g-convert
>    (*2) http://library.gnome.org/devel/glib/stable/glib-Unicode-Manipulation.html#g-utf8-to-utf16

True enough! I am not too fluent in glib stuff :/

>  * UTF-8 is the standard encoding in Japan, now. :-)

Great :) But it represents an overhead for you guys, no? E.g. even
katakanas seem to need 3 bytes in UTF-8 :/

http://www.fileformat.info/info/unicode/char/30a2/index.htm

>  * Please wait to change this after 0.90.2 is
>    released. I'll release 0.90.2 soon. :-)

Yes, of course. It has been too much time, and this is too
"sensitive", for me to commit without your approval!

>> Notice: RVAL2CSTR would be now an alias to RVAL2CSTR_ACCEPT_NIL, and I
>> think it is more safe because it's always better to properly handle
>> NIL case.
>
> We need to RVAL2CSTR with nil check because there are many
> API that doesn't accept NULL as gchar * argument. We will
> need a function that do StringValuePtr() with encoding handling.

Sorry, I didn't properly understand that. I see now.

>> Now, this patch properly works on the simple GtkLabel test, and my
>> GtkTreeStore based test posted in this thread earlier. If the
>> principle is fine with you, I think we must convert all rb_str_new2 of
>> sourcecode to CSTR2RVAL, and StringValuePtr to RVAL2CSTR (are there
>> other macros/ruby internals used to convert between Ruby String and
>> C?), to fix other similar problems.
>
> It seems OK for me. But we don't forget that there are some
> exceptions like g_convert(). We need to handle those
> exceptions by hands.

Ok, I'll try to improve my patch soon.

-- 
Guillaume Cottenceau - http://zarb.org/~gc/