wxlua-users

[wxlua-users] Problem in using Turkish in wxLua scripts

[Hakki D. <dog...@tr...>]

Hi,

(Mingw, wxLua cvs, wx2.7.2/wx2.8 ANSI and Unicode, WinXP Turkish)


I can't use Turkish chars (ie. ğüşiöçı ĞÜŞİÖÇİ) in label, title, etc.

If I change the following functions to old implementation, it works:
lua2wx, wx2lua.


What to do?


--
Regards,
Hakki Dogusan

Re: [wxlua-users] Problem in using Turkish in wxLua scripts

[John L. <jla...@gm...>]

T24gMTIvNS8wNiwgSGFra2kgRG9ndXNhbiA8ZG9ndXNhbmhAdHIubmV0PiB3cm90ZToKPiAoTWlu
Z3csIHd4THVhIGN2cywgd3gyLjcuMi93eDIuOCBBTlNJIGFuZCBVbmljb2RlLCBXaW5YUCBUdXJr
aXNoKQo+Cj4gSSBjYW4ndCB1c2UgVHVya2lzaCBjaGFycyAoaWUuIPD8/mn25/0g0Nze3dbH3Skg
aW4gbGFiZWwsIHRpdGxlLCBldGMuCj4KPiBJZiBJIGNoYW5nZSB0aGUgZm9sbG93aW5nIGZ1bmN0
aW9ucyB0byBvbGQgaW1wbGVtZW50YXRpb24sIGl0IHdvcmtzOgo+IGx1YTJ3eCwgd3gybHVhLgoK
VGhlIGNoYXJhY3RlcnMgeW91IHNlbnQgYWJvdmUgd29yayBpbiB0aGUgdGl0bGUgb2YgYSB3eEZy
YW1lIGluIHd4THVhCmNvbXBpbGVkIGluIExpbnV4LCB1bmljb2RlIHcvIGdjYy4gU28gaXQncyBh
IHByb2JsZW0gaW4gTVNXLiBJIGFsc28KY2hlY2tlZCB3aXRoIHNvbWUgb3RoZXIgY2hhcnMgb24g
dGhpcyBwYWdlLgpodHRwOi8vd3d3LmNvbHVtYmlhLmVkdS9rZXJtaXQvdXRmOC5odG1sCgpUaGUg
Y29kZSBJIHRoaW5rIHlvdSdyZSB0YWxraW5nIGFib3V0IGlzIGJlbG93LiBUaGluZ3Mgd2VyZSBj
aGFuZ2VkIHRvCmZpeCB1bmljb2RlIHN0cmluZ3MgaW4gTVNXaW5kb3dzLgoKUGxlYXNlIHNlZSB0
aGlzIG1lc3NhZ2UKUmU6IFtXeGx1YS11c2Vyc10gd3hTdHJpbmcsIFVuaWNvZGUgcHJvYmxlbSAu
Li4uIChGaXhlZCwpIFN0ZXZlIEtpZXUKaHR0cDovL3d3dy5tYWlsLWFyY2hpdmUuY29tL3d4bHVh
LXVzZXJzQGxpc3RzLnNvdXJjZWZvcmdlLm5ldC9pbmRleC5odG1sIzAwNjkyCgoKLy8gQ29udmVy
dCBhIDgtYml0IEx1YSBTdHJpbmcgaW50byB3eFN0cmluZwppbmxpbmUgV1hETExJTVBFWFBfV1hM
VUEgd3hTdHJpbmcgbHVhMnd4KGNvbnN0IGNoYXIqIGx1YXN0cikKewogICAgaWYgKGx1YXN0ciA9
PSBOVUxMKSByZXR1cm4gd3hFbXB0eVN0cmluZzsgLy8gY2hlY2sgZm9yIE5VTEwKCiNpZiB3eFVT
RV9VTklDT0RFCiAgICByZXR1cm4gd3hTdHJpbmcobHVhc3RyLCB3eENvbnZVVEY4KTsKI2Vsc2UK
ICAgIHJldHVybiB3eFN0cmluZyh3eENvbnZVVEY4LmNNQjJXQyhsdWFzdHIpLCAqd3hDb252Q3Vy
cmVudCk7CiNlbmRpZiAvLyB3eFVTRV9VTklDT0RFCgogIC8vcmV0dXJuIHd4Q29udmVydE1CMldY
KGx1YXN0cik7IC8vIG9sZCB3YXkgdGhhdCBtb3N0bHkgd29ya3MKfQoKLy8gQ29udmVydCBhIHd4
U3RyaW5nIHRvIDgtYml0IEx1YSBTdHJpbmcKaW5saW5lIGNvbnN0IFdYRExMSU1QRVhQX1dYTFVB
IHd4Q2hhckJ1ZmZlciB3eDJsdWEoY29uc3Qgd3hTdHJpbmcmIHd4c3RyKQp7CiAgICAvL3d4Q2hh
ckJ1ZmZlciBidWZmZXIod3hDb252ZXJ0V1gyTUIod3hzdHIuY19zdHIoKSkpOyAvLyBvbGQgd2F5
CnRoYXQgbW9zdGx5IHdvcmtzCiAgICB3eENoYXJCdWZmZXIKYnVmZmVyKHd4Q29udlVURjguY1dD
Mk1CKHd4c3RyLndjX3N0cigqd3hDb252Q3VycmVudCkpKTsgLy8gc2tpZXUKICAgIHJldHVybiBi
dWZmZXI7Cn0KCgo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0KCkkgaGF2ZSBh
c2tlZCBhYm91dCBjb252ZXJzaW9ucyBvbiB3eC11c2VycyBhIGxvbmcgdGltZSBhZ28gYW5kIGRp
ZG4ndApnZXQgYSBjb25jcmV0ZSBhbnN3ZXIgYXMgdG8gdGhlIHNpbmdsZSBiZXN0IHdheSB0byBk
byBpdC4gVGhlIG9sZCB3YXkKaXMgY29waWVkIGZyb20gdGhlIGZ1bmN0aW9ucyB3eDJzdGMgYW5k
IHN0YzJ3eCB0aGF0IHdlcmUgdXNlZCBpbiB0aGUKd3hTdHlsZWRUZXh0Q3RybCwgaG93ZXZlciBt
b3JlIHJlY2VudCB2ZXJzaW9ucyB1c2UgdGhlIHN0cmluZwpjb252ZXJzaW9uIGZ1bmNpb25zIHRo
YXQgYXJlIHBhcnQgb2YgU2NpbnRpbGxhLCBzZWUKd3hXaWRnZXRzL2NvbnRyaWIvc3JjL3N0Yy9z
Y2ludGlsbGEvc3JjL1VuaUNvbnZlcnNpb24uY3h4LgoKQ291bGQgeW91IHRyeSB0byByZXBsYWNl
IHRoZSBjb2RlIGluIHd4Mmx1YSBhbmQgbHVhMnd4IHdpdGggd3gyc3RjIGFuZApzdGMyd3guIFlv
dSdsbCBoYXZlIHRvICNpbmNsdWRlICJ3eC9zdGMvc3RjLmgiLiBQbGVhc2UgbGV0IHVzIGtub3cg
aWYKaXQgd29ya3MuCgoKSG9wZWZ1bGx5IFN0ZXZlIEtpZXUgd2lsbCByZWFkIHRoaXMgYW5kIGNo
aW1lIGluIHNpbmNlIGhlIHNlZW1zIHByZXR0eQpmYW1pbGlhciB3aXRoIHVuaWNvZGUgc3RyaW5n
cy4KClJlZ2FyZHMsCiAgICBKb2huIExhYmVuc2tpCg==

Re: [wxlua-users] Problem in using Turkish in wxLua scripts

[Hakki D. <dog...@tr...>]

Hi,

John Labenski wrote:
> On 12/5/06, Hakki Dogusan <dog...@tr...> wrote:
>> (Mingw, wxLua cvs, wx2.7.2/wx2.8 ANSI and Unicode, WinXP Turkish)
>>
>> I can't use Turkish chars (ie. ğüşiöçı ĞÜŞİÖÇİ) in label, title, etc.
>>
>> If I change the following functions to old implementation, it works:
>> lua2wx, wx2lua.
> 
> The characters you sent above work in the title of a wxFrame in wxLua
> compiled in Linux, unicode w/ gcc. So it's a problem in MSW. I also
> checked with some other chars on this page.
> http://www.columbia.edu/kermit/utf8.html
> 
> The code I think you're talking about is below. Things were changed to
> fix unicode strings in MSWindows.
> 
> Please see this message
> Re: [Wxlua-users] wxString, Unicode problem .... (Fixed,) Steve Kieu
> http://www.mail-archive.com/wxl...@li.../index.html#00692
> 
> 
> // Convert a 8-bit Lua String into wxString
> inline WXDLLIMPEXP_WXLUA wxString lua2wx(const char* luastr)
> {
>     if (luastr == NULL) return wxEmptyString; // check for NULL
> 
> #if wxUSE_UNICODE
>     return wxString(luastr, wxConvUTF8);
> #else
>     return wxString(wxConvUTF8.cMB2WC(luastr), *wxConvCurrent);
> #endif // wxUSE_UNICODE
> 
>   //return wxConvertMB2WX(luastr); // old way that mostly works
> }
> 
> // Convert a wxString to 8-bit Lua String
> inline const WXDLLIMPEXP_WXLUA wxCharBuffer wx2lua(const wxString& wxstr)
> {
>     //wxCharBuffer buffer(wxConvertWX2MB(wxstr.c_str())); // old way
> that mostly works
>     wxCharBuffer
> buffer(wxConvUTF8.cWC2MB(wxstr.wc_str(*wxConvCurrent))); // skieu
>     return buffer;
> }
> 
> 
> ====================================
> 
> I have asked about conversions on wx-users a long time ago and didn't
> get a concrete answer as to the single best way to do it. The old way
> is copied from the functions wx2stc and stc2wx that were used in the
> wxStyledTextCtrl, however more recent versions use the string
> conversion funcions that are part of Scintilla, see
> wxWidgets/contrib/src/stc/scintilla/src/UniConversion.cxx.
> 
> Could you try to replace the code in wx2lua and lua2wx with wx2stc and
> stc2wx. You'll have to #include "wx/stc/stc.h". Please let us know if
> it works.
> 
> 
> Hopefully Steve Kieu will read this and chime in since he seems pretty
> familiar with unicode strings.
> 
> Regards,
>     John Labenski

(Sorry for not joining to unicode discussions; I don't know it well.
I'm following wxLua since the beginning, but only recently I decide
to write a program with it.)


If I'm not mistaken stc uses UTC2 internally. So using stc's conversion
functions may not help here. Instead, I tested using FlameRobin's
(www.flamerobin.org) functions from StringUtils.h;
std::string wx2std(const wxString& input, wxMBConv* conv=wxConvCurrent);
wxString std2wx(const std::string& input, wxMBConv* conv=wxConvCurrent);


Changed lua2wx, wx2lua and wxLuaCharBuffer's ctor as follows:

inline WXDLLIMPEXP_WXLUA wxString lua2wx(const char* luastr)
{
     if (luastr == NULL) return wxEmptyString; // check for NULL
     return wxString(luastr, *wxConvCurrent);
}

inline const WXDLLIMPEXP_WXLUA wxCharBuffer wx2lua(const wxString& wxstr)
{
     wxCharBuffer buffer(wxstr.mb_str(*wxConvCurrent));
     return buffer;
}

wxLuaCharBuffer(const wxString &wxstr) : m_buffer((const char *)NULL)
{
     m_buffer = wxCharBuffer(wxstr.mb_str(*wxConvCurrent));
}


I tested with wxUSE_UNICODE=1 and =0 configurations.

I can use Turkish characters now.

(Mingw, wxLua cvs, wx2.8 ASCII and Unicode, WinXP Turkish)


ps. Thank you for working hard on wxLua!


--
Regards,
Hakki Dogusan

Re: [wxlua-users] Problem in using Turkish in wxLua scripts

[John L. <jla...@gm...>]

T24gMTIvNi8wNiwgSGFra2kgRG9ndXNhbiA8ZG9ndXNhbmhAdHIubmV0PiB3cm90ZToKPiBKb2hu
IExhYmVuc2tpIHdyb3RlOgo+ID4gT24gMTIvNS8wNiwgSGFra2kgRG9ndXNhbiA8ZG9ndXNhbmhA
dHIubmV0PiB3cm90ZToKPiA+PiAoTWluZ3csIHd4THVhIGN2cywgd3gyLjcuMi93eDIuOCBBTlNJ
IGFuZCBVbmljb2RlLCBXaW5YUCBUdXJraXNoKQo+ID4+Cj4gPj4gSSBjYW4ndCB1c2UgVHVya2lz
aCBjaGFycyAoaWUuIPD8/mn25/0g0Nze3dbH3SkgaW4gbGFiZWwsIHRpdGxlLCBldGMuCj4gPj4K
PiA+PiBJZiBJIGNoYW5nZSB0aGUgZm9sbG93aW5nIGZ1bmN0aW9ucyB0byBvbGQgaW1wbGVtZW50
YXRpb24sIGl0IHdvcmtzOgo+ID4+IGx1YTJ3eCwgd3gybHVhLgo+ID4KPiA+IFBsZWFzZSBzZWUg
dGhpcyBtZXNzYWdlCj4gPiBSZTogW1d4bHVhLXVzZXJzXSB3eFN0cmluZywgVW5pY29kZSBwcm9i
bGVtIC4uLi4gKEZpeGVkLCkgU3RldmUgS2lldQo+ID4gaHR0cDovL3d3dy5tYWlsLWFyY2hpdmUu
Y29tL3d4bHVhLXVzZXJzQGxpc3RzLnNvdXJjZWZvcmdlLm5ldC9pbmRleC5odG1sIzAwNjkyCj4g
Pgo+ID4gLy8gQ29udmVydCBhIDgtYml0IEx1YSBTdHJpbmcgaW50byB3eFN0cmluZwo+ID4gaW5s
aW5lIFdYRExMSU1QRVhQX1dYTFVBIHd4U3RyaW5nIGx1YTJ3eChjb25zdCBjaGFyKiBsdWFzdHIp
Cj4gPiB7Cj4gPiAgICAgaWYgKGx1YXN0ciA9PSBOVUxMKSByZXR1cm4gd3hFbXB0eVN0cmluZzsg
Ly8gY2hlY2sgZm9yIE5VTEwKPiA+Cj4gPiAjaWYgd3hVU0VfVU5JQ09ERQo+ID4gICAgIHJldHVy
biB3eFN0cmluZyhsdWFzdHIsIHd4Q29udlVURjgpOwo+ID4gI2Vsc2UKPiA+ICAgICByZXR1cm4g
d3hTdHJpbmcod3hDb252VVRGOC5jTUIyV0MobHVhc3RyKSwgKnd4Q29udkN1cnJlbnQpOwo+ID4g
I2VuZGlmIC8vIHd4VVNFX1VOSUNPREUKPiA+Cj4gPiAgIC8vcmV0dXJuIHd4Q29udmVydE1CMldY
KGx1YXN0cik7IC8vIG9sZCB3YXkgdGhhdCBtb3N0bHkgd29ya3MKPiA+IH0KPiA+Cj4gPiAvLyBD
b252ZXJ0IGEgd3hTdHJpbmcgdG8gOC1iaXQgTHVhIFN0cmluZwo+ID4gaW5saW5lIGNvbnN0IFdY
RExMSU1QRVhQX1dYTFVBIHd4Q2hhckJ1ZmZlciB3eDJsdWEoY29uc3Qgd3hTdHJpbmcmIHd4c3Ry
KQo+ID4gewo+ID4gICAgIC8vd3hDaGFyQnVmZmVyIGJ1ZmZlcih3eENvbnZlcnRXWDJNQih3eHN0
ci5jX3N0cigpKSk7IC8vIG9sZCB3YXkKPiA+IHRoYXQgbW9zdGx5IHdvcmtzCj4gPiAgICAgd3hD
aGFyQnVmZmVyCj4gPiBidWZmZXIod3hDb252VVRGOC5jV0MyTUIod3hzdHIud2Nfc3RyKCp3eENv
bnZDdXJyZW50KSkpOyAvLyBza2lldQo+ID4gICAgIHJldHVybiBidWZmZXI7Cj4gPiB9Cj4gPgo+
ID4gPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Cj4gPgo+Cj4gSWYgSSdtIG5v
dCBtaXN0YWtlbiBzdGMgdXNlcyBVVEMyIGludGVybmFsbHkuIFNvIHVzaW5nIHN0YydzIGNvbnZl
cnNpb24KPiBmdW5jdGlvbnMgbWF5IG5vdCBoZWxwIGhlcmUuIEluc3RlYWQsIEkgdGVzdGVkIHVz
aW5nIEZsYW1lUm9iaW4ncwo+ICh3d3cuZmxhbWVyb2Jpbi5vcmcpIGZ1bmN0aW9ucyBmcm9tIFN0
cmluZ1V0aWxzLmg7Cj4gc3RkOjpzdHJpbmcgd3gyc3RkKGNvbnN0IHd4U3RyaW5nJiBpbnB1dCwg
d3hNQkNvbnYqIGNvbnY9d3hDb252Q3VycmVudCk7Cj4gd3hTdHJpbmcgc3RkMnd4KGNvbnN0IHN0
ZDo6c3RyaW5nJiBpbnB1dCwgd3hNQkNvbnYqIGNvbnY9d3hDb252Q3VycmVudCk7CgpIdW1tLCBp
bnRlcmVzdGluZy4uLiBzZWUgYmVsb3cuCgo+IENoYW5nZWQgbHVhMnd4LCB3eDJsdWEgYW5kIHd4
THVhQ2hhckJ1ZmZlcidzIGN0b3IgYXMgZm9sbG93czoKPgo+IGlubGluZSBXWERMTElNUEVYUF9X
WExVQSB3eFN0cmluZyBsdWEyd3goY29uc3QgY2hhciogbHVhc3RyKQo+IHsKPiAgICAgIGlmIChs
dWFzdHIgPT0gTlVMTCkgcmV0dXJuIHd4RW1wdHlTdHJpbmc7IC8vIGNoZWNrIGZvciBOVUxMCj4g
ICAgICByZXR1cm4gd3hTdHJpbmcobHVhc3RyLCAqd3hDb252Q3VycmVudCk7Cj4gfQoKVGhlIG9s
ZCB3YXkgb2YgZG9pbmcgdGhlIGNvbnZlcnNpb24gZGlkIGl0IHRoaXMgd2F5LgoKLy9yZXR1cm4g
d3hDb252ZXJ0TUIyV1gobHVhc3RyKTsgLy8gb2xkIHdheSB0aGF0IG1vc3RseSB3b3JrcwojZGVm
aW5lIHd4Q29udmVydE1CMldYKHMpICAgd3hDb252Q3VycmVudC0+Y01CMldYKHMpCgpUaGlzIGlz
IHRoZSBjb2RlIGZvciB0aGUgd3hTdHJpbmcgZnVuY3Rpb24gd2hpY2ggZG9lcyB0aGUgc2FtZSBh
cyB3eENvbnZlcnRNQjJXWAp3eFN0cmluZzo6d3hTdHJpbmcoY29uc3QgY2hhciAqcHN6LCBjb25z
dCB3eE1CQ29udiYgY29udiwgc2l6ZV90IG5MZW5ndGgpCnsKLi4uCiAgICAgIHd4V0NoYXJCdWZm
ZXIgd2J1ZiA9IGNvbnYuY01CMldDKHBzeiwgbkxlbmd0aCwgJm5MZW5XaWRlKTsKCj09PT09PT09
PT09PT09PT09PQpJIHNlZSB0aGF0IHdlIGhhdmUgMyBkaWZmZXJlbnQgd2F5cyB0byBjb252ZXJ0
IHRoZSBzdHJpbmdzLiBXaGljaCB3YXkKaXMgInJpZ2h0IiBmb3IgYWxsIGNhc2VzLgoKMSkgZG9l
c24ndCB3b3JrIGZvciBzb21lIHVua25vd24gY2FzZXMKICAgIHd4Q29udmVydE1CMldYID09IHd4
Q29udkN1cnJlbnQtPmNNQjJXWChzKSA9PQp3eENvbnZDdXJyZW50LT5jTUIyV0MKCjIpIERvZXNu
J3Qgd29yayBmb3IgVHVya2lzaCAoYW5kIHVuZG91YnRlZGx5IG90aGVycykKI2lmIHd4VVNFX1VO
SUNPREUKICAgIHJldHVybiB3eFN0cmluZyhsdWFzdHIsIHd4Q29udlVURjgpOwojZWxzZQogICAg
cmV0dXJuIHd4U3RyaW5nKHd4Q29udlVURjguY01CMldDKGx1YXN0ciksICp3eENvbnZDdXJyZW50
KTsKI2VuZGlmIC8vIHd4VVNFX1VOSUNPREUKCjMpIG5ldz8gd29ya3M/CiAgICB3eFN0cmluZyhs
dWFzdHIsICp3eENvbnZDdXJyZW50KTsgPT0KICAgIHd4V0NoYXJCdWZmZXIgd2J1ZiA9IGNvbnYu
Y01CMldDKHBzeiwgbkxlbmd0aCwgJm5MZW5XaWRlKTsKCgpUaGUgZGlmZmVyZW5jZXMgSSBzZWUg
aXMgdGhhdAoKCgo+IGlubGluZSBjb25zdCBXWERMTElNUEVYUF9XWExVQSB3eENoYXJCdWZmZXIg
d3gybHVhKGNvbnN0IHd4U3RyaW5nJiB3eHN0cikKPiB7Cj4gICAgICB3eENoYXJCdWZmZXIgYnVm
ZmVyKHd4c3RyLm1iX3N0cigqd3hDb252Q3VycmVudCkpOwo+ICAgICAgcmV0dXJuIGJ1ZmZlcjsK
PiB9Cj4KPiB3eEx1YUNoYXJCdWZmZXIoY29uc3Qgd3hTdHJpbmcgJnd4c3RyKSA6IG1fYnVmZmVy
KChjb25zdCBjaGFyICopTlVMTCkKPiB7Cj4gICAgICBtX2J1ZmZlciA9IHd4Q2hhckJ1ZmZlcih3
eHN0ci5tYl9zdHIoKnd4Q29udkN1cnJlbnQpKTsKPiB9Cj4KPgo+IEkgdGVzdGVkIHdpdGggd3hV
U0VfVU5JQ09ERT0xIGFuZCA9MCBjb25maWd1cmF0aW9ucy4KPgo+IEkgY2FuIHVzZSBUdXJraXNo
IGNoYXJhY3RlcnMgbm93Lgo+Cj4gKE1pbmd3LCB3eEx1YSBjdnMsIHd4Mi44IEFTQ0lJIGFuZCBV
bmljb2RlLCBXaW5YUCBUdXJraXNoKQo+Cj4KPiBwcy4gVGhhbmsgeW91IGZvciB3b3JraW5nIGhh
cmQgb24gd3hMdWEhCj4KPgo+IC0tCj4gUmVnYXJkcywKPiBIYWtraSBEb2d1c2FuCj4KPiAtLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tCj4gVGFrZSBTdXJ2ZXlzLiBFYXJuIENhc2guIEluZmx1ZW5jZSB0aGUgRnV0
dXJlIG9mIElUCj4gSm9pbiBTb3VyY2VGb3JnZS5uZXQncyBUZWNoc2F5IHBhbmVsIGFuZCB5b3Un
bGwgZ2V0IHRoZSBjaGFuY2UgdG8gc2hhcmUgeW91cgo+IG9waW5pb25zIG9uIElUICYgYnVzaW5l
c3MgdG9waWNzIHRocm91Z2ggYnJpZWYgc3VydmV5cyAtIGFuZCBlYXJuIGNhc2gKPiBodHRwOi8v
d3d3LnRlY2hzYXkuY29tL2RlZmF1bHQucGhwP3BhZ2U9am9pbi5waHAmcD1zb3VyY2Vmb3JnZSZD
SUQ9REVWREVWCj4gX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f
X18KPiB3eGx1YS11c2VycyBtYWlsaW5nIGxpc3QKPiB3eGx1YS11c2Vyc0BsaXN0cy5zb3VyY2Vm
b3JnZS5uZXQKPiBodHRwczovL2xpc3RzLnNvdXJjZWZvcmdlLm5ldC9saXN0cy9saXN0aW5mby93
eGx1YS11c2Vycwo+Cg==