lisp-cffi-qt4-list

[Lisp-cffi-qt4-list] Unicode

[Radoslav A. <rsd...@gm...>]

Hi, Paul :)

What about Unicode support in lisp-cffi-q4?
You see, I'm a Bulgarian student in Germany, and currently I'm learning 
greek :-D Even for simple notes I need some Unicode support...

AFAIK Qt4 has a mature Unicode support. There are CL implementations 
with good Unicode support. So what could break the flow is either CFFI 
or lisp-cffi-qt4. I spent a little time trying to understand what about 
CFFI, but I couldn't, so I decided just to try.

What I tried is the following:
As I work on Windows, I had to choose CLisp as an implementation, as ECL 
is still not quite Unicode compliant. I had to use 2.38, because the 
last one (2.41) had some problems loading "qt.lisp", probably with CFFI 
... (On 2.38 lisp-cffi-qt4 works just fine, at least the simple tests.)

I tried this (having UTF-8 encoded file utf.txt, containing Bulgarian, 
German and Greek words on its first line):
> (load "asdf")
> (load "qt")
>
> (defpackage qt-user (:use cl qt))
> (in-package qt-user)
>
> (defvar *unicode-string* nil)
> (let ((custom:*default-file-encoding* charset:utf-8))
>     (with-open-file (f "utf.txt")
>         (setf *unicode-string* (read-line f))))
>         
> (with-qt ()
>     (object lb `(label ,*unicode-string*)))
(All CLisp guru's, excuse my ignorance of a better way to write the code 
above...)

And got:
> ...
> ;;   Loaded file 
> D:\Opt\Lisp\qt\exe\clisp\examples\cffi\src\foreign-vars.fas
> 0 errors, 0 warnings
> ;;  Loaded file D:\Opt\Lisp\qt\exe\clisp\examples\qt.lisp
> *** - 1087 cannot be converted to the foreign type FFI:UCHAR
> The following restarts are available:
> ...
Of what I see, I can suppose that (at least CLisp's mplementation of) 
CFFI doesn't support Unicode. Am I right, or the way lisp-cffi-qt4 uses 
CFFI is not Unicode compliant?

Regards,
Rado

Re: [Lisp-cffi-qt4-list] Unicode (2)

[Radoslav A. <rsd...@gm...>]

Hi, Paul.
At first, thank you for the quick response.

Unfortunately I still cannot pass UTF-8 encoded string from CLISP. I 
think the problem is either in the way CFFI is used or in CFFI 
(implementation for CLISP) itself.
Later I found a statement <http://www.cliki.net/Unicode%20support> 
saying that "CLISP: FFI can pass strings only with single-byte encoding" 
and this explains the error message I got: "1087 cannot be converted to 
the foreign type FFI:UCHAR".

I think this statement is a outdated, but still looking at 
http://clisp.cons.org/impnotes.html#dffi-types it seems that CLISP's 
ffi:uchar is 8 bit. I also tried
 (setf custom:*foreign-encoding* charset:utf-8)
but the error persisted.

And finally one more thing: (the latest version of) CFFI don't work 
properly with the latest version of CLISP (2.41). It's quite normal of 
course, at it is still new and probably they changed their FFI in some 
way. But my idea is: is it possible at all to pass a UTF-8 string with 
CLISP FFI, and if, how to make it work with CFFI? I'll continue trying, 
but if you have some ideas please share ;)

myteam - Paul wrote:
> Hi,
>
> I just released a new version of lisp-cffi-qt4 (0.2j), where I added 
> (among other, not unicode related things) some small changes to the 
> C-string to QString conversion (I simply added a "QString::fromUtf8()" 
> in some places).
>
> Sorry for my little understanding of UTF8, I didn't know that it can 
> be represented in a simple C-string, so there should be no need for 
> real unicode support when displaying UTF8 strings.
I'm also not a Unicode expert :)) But while UTF8 is to some extent 8bit 
transparent, yet it is not a full righted 8bit string. For example 8bit 
functions for string length won't give correct result, as they count 
each byte as a character, which is not the case in UTF-8 (some 
characters are encoded with much more bytes...).
>
> I did some little testing, and it seemed possible to display UTF8 
> strings with ECL.
Would you paste some code snippet?
>
> I never needed unicode in my programs, so maybe I'm currently no big 
> help for your problems, but I will try to get into it in the near 
> future...
>
> Anyway, I hope that helps you somehow, otherwise I'm sorry for the 
> moment...
No problem ;))The good thing with me is I'm not in a hurry. I really 
have too much to do, so I hardly find time for Lisp. But still I dream 
for the day I'll have the tools and the time to write production, 
crossplatform Lisp/Qt (/Unicode) code :)) For now I would be content 
with the little I can contribute...
>
> Paul
>
>