Menu
▾
▴
python-ldap-dev
|
From: <js...@ne...> - 2001-10-16 17:41:29
|
The iconv is part of glibc 2.2.2 and above, so iconv.h should be in any recent LINUX system if the C developement environment is configured. If older than glic 2.2.2, you might have to install the iconvlib seperately. I did try to use PyString_(De/En)code, but it is not very well suited to openldap type of applications, especially if dn has mixed ascii and utf8 encodings. But I will be happy if someone can prove I am wrong. Thanks. J. David Leonard <dav...@it...> wrote: > >On Sat, 13 Oct 2001, js...@ne... typed thusly: >> I just worked out the python-ldap so that it will convert the >> local charactors to utf8 or vice versa according to the user locale >> setup. This is done because OpenLdap currently only support utf8, >> so one has to make the translation. I tested on zh_CN.GB2312 >> (simplified Chinese), and it worked. By no means it is a complete >> test, but it is a good start, and it worked for me. The patch was >> generated against the CVS tree check out yesterday, so it is pretty >> recent. >> Here is the patch in the attachment. > >hi > >sorry i have been unable to respond to you - i had marked your >message important but have been busy of late. :( > >i'm glad you send it to the list, because it will stand a better chance >of being reviewed there.. a better chance than just sitting in my mailbox. > >here are my comments from a quick perusal of the diff > >... the diff is reversed. (took me a moment to figure out what was >going on there!.. the order is 'diff <old> <new>') > >... you're relying on an iconv.h header file, which isn't on every >system. (especially mine) > >.. I think instead to use PyString_Decode() and PyString_Encode().. >that way you get python unicode string objects. > >i'm not an expert on these things, but here is a cut and paste from >a python header file: > >/* Create a string object by decoding the encoded string s of the > given size. */ > >extern DL_IMPORT(PyObject*) PyString_Decode( > const char *s, /* encoded string */ > int size, /* size of buffer */ > const char *encoding, /* encoding */ > const char *errors /* error handling */ > ); > >/* Encodes a char buffer of the given size and returns a > Python string object. */ > >extern DL_IMPORT(PyObject*) PyString_Encode( > const char *s, /* string char buffer */ > int size, /* number of chars to encode */ > const char *encoding, /* encoding */ > const char *errors /* error handling */ > ); > > >d > >-- >David Leonard Dav...@it... >Dept of Inf. Tech. and Elec. Engg _ Ph:+61 404 844 850 >The University of Queensland |+| http://www.itee.uq.edu.au/~leonard/ >QLD 4072 AUSTRALIA ~` '~ B73CD65FBEF4C089B79A8EBADF1A932F13EA0FC8 > >Make sure that you have enough free memory in your temp directory. > - Sun StarOffice README > > > __________________________________________________________________ Your favorite stores, helpful shopping tools and great gift ideas. Experience the convenience of buying online with Shop@Netscape! http://shopnow.netscape.com/ Get your own FREE, personal Netscape Mail account today at http://webmail.netscape.com/ |
|
From: <js...@ne...> - 2001-10-17 05:19:14
|
UNIX(LINUX) is the only OS I know that has the internationalization (i18n) concept. Windows, VMS and many other less known OS, such as milkyway etc., only have the concept of localization (L10?). This patch may be used for localization purpose as well, but its main target is the i18n type of applications. Don't know about mac, but since it is mainly a personal system, my guess is that it is just like windows, L10 only. any way, I appreciate the good work in the free software community, and like to contribute something back. I have no interest getting into the OS squabbling, especially with mac fans. Thanks J. jml...@ma... wrote: >I wouldn't suggest that the audience here is 100% linux. Rather, a >majority likely is not > >On Tuesday, October 16, 2001, at 10:41 AM, js...@ne... wrote: > >> The iconv is part of glibc 2.2.2 and above, so iconv.h should be in any >> recent LINUX system if the C developement environment is configured. If >> older than glic 2.2.2, you might have to install the iconvlib >> seperately. >> >> I did try to use PyString_(De/En)code, but it is not very well suited >> to openldap type of applications, especially if dn has mixed ascii and >> utf8 encodings. But I will be happy if someone can prove I am wrong. >> >> Thanks. >> J. >> >> >> David Leonard <dav...@it...> wrote: >> >>> >>> On Sat, 13 Oct 2001, js...@ne... typed thusly: >>>> I just worked out the python-ldap so that it will convert the >>>> local charactors to utf8 or vice versa according to the user locale >>>> setup. This is done because OpenLdap currently only support utf8, >>>> so one has to make the translation. I tested on zh_CN.GB2312 >>>> (simplified Chinese), and it worked. By no means it is a complete >>>> test, but it is a good start, and it worked for me. The patch was >>>> generated against the CVS tree check out yesterday, so it is pretty >>>> recent. >>>> Here is the patch in the attachment. >>> >>> hi >>> >>> sorry i have been unable to respond to you - i had marked your >>> message important but have been busy of late. :( >>> >>> i'm glad you send it to the list, because it will stand a better chance >>> of being reviewed there.. a better chance than just sitting in my >>> mailbox. >>> >>> here are my comments from a quick perusal of the diff >>> >>> ... the diff is reversed. (took me a moment to figure out what was >>> going on there!.. the order is 'diff <old> <new>') >>> >>> ... you're relying on an iconv.h header file, which isn't on every >>> system. (especially mine) >>> >>> .. I think instead to use PyString_Decode() and PyString_Encode().. >>> that way you get python unicode string objects. >>> >>> i'm not an expert on these things, but here is a cut and paste from >>> a python header file: >>> >>> /* Create a string object by decoding the encoded string s of the >>> given size. */ >>> >>> extern DL_IMPORT(PyObject*) PyString_Decode( >>> const char *s, /* encoded string */ >>> int size, /* size of buffer */ >>> const char *encoding, /* encoding */ >>> const char *errors /* error handling */ >>> ); >>> >>> /* Encodes a char buffer of the given size and returns a >>> Python string object. */ >>> >>> extern DL_IMPORT(PyObject*) PyString_Encode( >>> const char *s, /* string char buffer */ >>> int size, /* number of chars to encode */ >>> const char *encoding, /* encoding */ >>> const char *errors /* error handling */ >>> ); >>> >>> >>> d >>> >>> -- >>> David Leonard Dav...@it... >>> Dept of Inf. Tech. and Elec. Engg _ Ph:+61 404 844 850 >>> The University of Queensland |+| >>> http://www.itee.uq.edu.au/~leonard/ >>> QLD 4072 AUSTRALIA ~` '~ >>> B73CD65FBEF4C089B79A8EBADF1A932F13EA0FC8 >>> >>> Make sure that you have enough free memory in your temp directory. >>> - Sun StarOffice README >>> >>> >>> >> >> >> __________________________________________________________________ >> Your favorite stores, helpful shopping tools and great gift ideas. >> Experience the convenience of buying online with Shop@Netscape! >> http://shopnow.netscape.com/ >> >> Get your own FREE, personal Netscape Mail account today at >> http://webmail.netscape.com/ >> >> >> _______________________________________________ >> Python-LDAP-dev mailing list >> Pyt...@li... >> https://lists.sourceforge.net/lists/listinfo/python-ldap-dev >> > > __________________________________________________________________ Your favorite stores, helpful shopping tools and great gift ideas. Experience the convenience of buying online with Shop@Netscape! http://shopnow.netscape.com/ Get your own FREE, personal Netscape Mail account today at http://webmail.netscape.com/ |
|
From: David L. <dav...@it...> - 2001-10-17 05:26:26
|
to get this thread back on track, does anyone have objections to Michael's solution? i.e. leave the data as 'raw' strings, and use codecs.utf_8_decode() when you know that it is UTF-8 encoded. d On Wed, 17 Oct 2001, js...@ne... typed thusly: > UNIX(LINUX) is the only OS I know that has the internationalization (i18n) concept. Windows, VMS and many other less known OS, such as milkyway etc., only have the concept of localization (L10?). This patch may be used for localization purpose as well, but its main target is the i18n type of applications. Don't know about mac, but since it is mainly a personal system, my guess is that it is just like windows, L10 only. -- David Leonard Dav...@it... Dept of Inf. Tech. and Elec. Engg _ Ph:+61 404 844 850 The University of Queensland |+| http://www.itee.uq.edu.au/~leonard/ QLD 4072 AUSTRALIA ~` '~ B73CD65FBEF4C089B79A8EBADF1A932F13EA0FC8 Make sure that you have enough free memory in your temp directory. - Sun StarOffice README |
|
From: Joe L. <jl...@op...> - 2001-10-17 05:44:51
|
well, mac is now UNIX, its also already I18N natively Don't start a flame war without fuel :).... I'm merely stating the the python-ldap module needs to maintain its ability to be os agnostic. On Tuesday, October 16, 2001, at 10:18 PM, js...@ne... wrote: > UNIX(LINUX) is the only OS I know that has the internationalization > (i18n) concept. Windows, VMS and many other less known OS, such as > milkyway etc., only have the concept of localization (L10?). This > patch may be used for localization purpose as well, but its main target > is the i18n type of applications. Don't know about mac, but since it > is mainly a personal system, my guess is that it is just like windows, > L10 only. > > any way, I appreciate the good work in the free software community, and > like to contribute something back. I have no interest getting into the > OS squabbling, especially with mac fans. > > Thanks > J. > > > jml...@ma... wrote: > >> I wouldn't suggest that the audience here is 100% linux. Rather, a >> majority likely is not >> >> On Tuesday, October 16, 2001, at 10:41 AM, js...@ne... wrote: >> >>> The iconv is part of glibc 2.2.2 and above, so iconv.h should be in >>> any >>> recent LINUX system if the C developement environment is configured. >>> If >>> older than glic 2.2.2, you might have to install the iconvlib >>> seperately. >>> >>> I did try to use PyString_(De/En)code, but it is not very well suited >>> to openldap type of applications, especially if dn has mixed ascii and >>> utf8 encodings. But I will be happy if someone can prove I am wrong. >>> >>> Thanks. >>> J. >>> >>> >>> David Leonard <dav...@it...> wrote: >>> >>>> >>>> On Sat, 13 Oct 2001, js...@ne... typed thusly: >>>>> I just worked out the python-ldap so that it will convert the >>>>> local charactors to utf8 or vice versa according to the user locale >>>>> setup. This is done because OpenLdap currently only support utf8, >>>>> so one has to make the translation. I tested on zh_CN.GB2312 >>>>> (simplified Chinese), and it worked. By no means it is a complete >>>>> test, but it is a good start, and it worked for me. The patch was >>>>> generated against the CVS tree check out yesterday, so it is pretty >>>>> recent. >>>>> Here is the patch in the attachment. >>>> >>>> hi >>>> >>>> sorry i have been unable to respond to you - i had marked your >>>> message important but have been busy of late. :( >>>> >>>> i'm glad you send it to the list, because it will stand a better >>>> chance >>>> of being reviewed there.. a better chance than just sitting in my >>>> mailbox. >>>> >>>> here are my comments from a quick perusal of the diff >>>> >>>> ... the diff is reversed. (took me a moment to figure out what was >>>> going on there!.. the order is 'diff <old> <new>') >>>> >>>> ... you're relying on an iconv.h header file, which isn't on every >>>> system. (especially mine) >>>> >>>> .. I think instead to use PyString_Decode() and PyString_Encode().. >>>> that way you get python unicode string objects. >>>> >>>> i'm not an expert on these things, but here is a cut and paste from >>>> a python header file: >>>> >>>> /* Create a string object by decoding the encoded string s of the >>>> given size. */ >>>> >>>> extern DL_IMPORT(PyObject*) PyString_Decode( >>>> const char *s, /* encoded string */ >>>> int size, /* size of buffer */ >>>> const char *encoding, /* encoding */ >>>> const char *errors /* error handling */ >>>> ); >>>> >>>> /* Encodes a char buffer of the given size and returns a >>>> Python string object. */ >>>> >>>> extern DL_IMPORT(PyObject*) PyString_Encode( >>>> const char *s, /* string char buffer */ >>>> int size, /* number of chars to encode */ >>>> const char *encoding, /* encoding */ >>>> const char *errors /* error handling */ >>>> ); >>>> >>>> >>>> d >>>> >>>> -- >>>> David Leonard Dav...@it... >>>> Dept of Inf. Tech. and Elec. Engg _ Ph:+61 404 844 850 >>>> The University of Queensland |+| >>>> http://www.itee.uq.edu.au/~leonard/ >>>> QLD 4072 AUSTRALIA ~` '~ >>>> B73CD65FBEF4C089B79A8EBADF1A932F13EA0FC8 >>>> >>>> Make sure that you have enough free memory in your temp directory. >>>> - Sun StarOffice README >>>> >>>> >>>> >>> >>> >>> __________________________________________________________________ >>> Your favorite stores, helpful shopping tools and great gift ideas. >>> Experience the convenience of buying online with Shop@Netscape! >>> http://shopnow.netscape.com/ >>> >>> Get your own FREE, personal Netscape Mail account today at >>> http://webmail.netscape.com/ >>> >>> >>> _______________________________________________ >>> Python-LDAP-dev mailing list >>> Pyt...@li... >>> https://lists.sourceforge.net/lists/listinfo/python-ldap-dev >>> >> >> > > > __________________________________________________________________ > Your favorite stores, helpful shopping tools and great gift ideas. > Experience the convenience of buying online with Shop@Netscape! > http://shopnow.netscape.com/ > > Get your own FREE, personal Netscape Mail account today at > http://webmail.netscape.com/ > > > _______________________________________________ > Python-LDAP-dev mailing list > Pyt...@li... > https://lists.sourceforge.net/lists/listinfo/python-ldap-dev > |
|
From: Michael <mi...@st...> - 2001-10-17 07:56:51
|
Joe Little wrote: > > I'm merely stating the the python-ldap module needs to maintain its > ability to be os agnostic. Yes. Although my personal preference is using Linux as OS it's an important requirement that python-ldap should run on any platform where Python has been ported to. If somebody needs special codecs not part of the Python distribution he/she should look into the iconv-based Python codec project. Ciao, Michael. |
|
From: <js...@ne...> - 2001-10-17 15:38:55
|
I have something like uid=someone,ou=ç«,dc=domain,dc=org. The ou value is in gb2312 encoding. If I use PyString_(En/De) the whole dn string, OpenLDAP doesn't like it. So I have to parse out the ou=, and only PyString_(En/De) that piece, which make things quite bit more complicated, and I did not spend time to figure out why. I choose a quick and easy(dirty ?) way out for myself, and get this thread started. I will certainly vote for someone who can device an effecient and clean way to do it cross platforms.
Michael Ströder <mi...@st...> wrote:
>js...@ne... wrote:
>>
>> The iconv is part of glibc 2.2.2 and above, so iconv.h should be in
>> any recent LINUX system
>
>web2ldap based on python-ldap aims to run on Win32 platform too...
>
>Anyway all character set translation should be based on Unicode
>support introduced in Python 1.6 because the codec concept looks
>very clean. There are Unicode codecs for Python based on iconv lib
>if the built-in Python codecs are not sufficient.
>
>> I did try to use PyString_(De/En)code, but it is not very well
>> suited to openldap type of applications, especially if dn has mixed
>> ascii and utf8 encodings.
>
>I'm not sure what you mean with "mixed ascii and utf8 encodings".
>
>UTF-8 is a variable length encoding of the ISO-10646 character set
>which maps the page containing the ASCII characters directly to
>ASCII.
>
>Example with my last name:
>
>>>> unicode('Str?der','iso-8859-1').encode('utf-8')
>'Str\xc3\xb6der'
>
>Ciao, Michael.
>
__________________________________________________________________
Your favorite stores, helpful shopping tools and great gift ideas. Experience the convenience of buying online with Shop@Netscape! http://shopnow.netscape.com/
Get your own FREE, personal Netscape Mail account today at http://webmail.netscape.com/
|
|
From: Michael <mi...@st...> - 2001-10-18 11:34:33
|
js...@ne... wrote: > > I have something like uid=someone,ou=ç?«,dc=domain,dc=org. The > ou value is in gb2312 encoding. It seems that your DNs are not compliant to RFC2253. All attribute values should be UTF-8 encoded ISO-10646 Unicode. Check the component which builds this DN. Ciao, Michael. |
|
From: <jml...@ma...> - 2001-10-16 18:31:28
|
I wouldn't suggest that the audience here is 100% linux. Rather, a majority likely is not On Tuesday, October 16, 2001, at 10:41 AM, js...@ne... wrote: > The iconv is part of glibc 2.2.2 and above, so iconv.h should be in any > recent LINUX system if the C developement environment is configured. If > older than glic 2.2.2, you might have to install the iconvlib > seperately. > > I did try to use PyString_(De/En)code, but it is not very well suited > to openldap type of applications, especially if dn has mixed ascii and > utf8 encodings. But I will be happy if someone can prove I am wrong. > > Thanks. > J. > > > David Leonard <dav...@it...> wrote: > >> >> On Sat, 13 Oct 2001, js...@ne... typed thusly: >>> I just worked out the python-ldap so that it will convert the >>> local charactors to utf8 or vice versa according to the user locale >>> setup. This is done because OpenLdap currently only support utf8, >>> so one has to make the translation. I tested on zh_CN.GB2312 >>> (simplified Chinese), and it worked. By no means it is a complete >>> test, but it is a good start, and it worked for me. The patch was >>> generated against the CVS tree check out yesterday, so it is pretty >>> recent. >>> Here is the patch in the attachment. >> >> hi >> >> sorry i have been unable to respond to you - i had marked your >> message important but have been busy of late. :( >> >> i'm glad you send it to the list, because it will stand a better chance >> of being reviewed there.. a better chance than just sitting in my >> mailbox. >> >> here are my comments from a quick perusal of the diff >> >> ... the diff is reversed. (took me a moment to figure out what was >> going on there!.. the order is 'diff <old> <new>') >> >> ... you're relying on an iconv.h header file, which isn't on every >> system. (especially mine) >> >> .. I think instead to use PyString_Decode() and PyString_Encode().. >> that way you get python unicode string objects. >> >> i'm not an expert on these things, but here is a cut and paste from >> a python header file: >> >> /* Create a string object by decoding the encoded string s of the >> given size. */ >> >> extern DL_IMPORT(PyObject*) PyString_Decode( >> const char *s, /* encoded string */ >> int size, /* size of buffer */ >> const char *encoding, /* encoding */ >> const char *errors /* error handling */ >> ); >> >> /* Encodes a char buffer of the given size and returns a >> Python string object. */ >> >> extern DL_IMPORT(PyObject*) PyString_Encode( >> const char *s, /* string char buffer */ >> int size, /* number of chars to encode */ >> const char *encoding, /* encoding */ >> const char *errors /* error handling */ >> ); >> >> >> d >> >> -- >> David Leonard Dav...@it... >> Dept of Inf. Tech. and Elec. Engg _ Ph:+61 404 844 850 >> The University of Queensland |+| >> http://www.itee.uq.edu.au/~leonard/ >> QLD 4072 AUSTRALIA ~` '~ >> B73CD65FBEF4C089B79A8EBADF1A932F13EA0FC8 >> >> Make sure that you have enough free memory in your temp directory. >> - Sun StarOffice README >> >> >> > > > __________________________________________________________________ > Your favorite stores, helpful shopping tools and great gift ideas. > Experience the convenience of buying online with Shop@Netscape! > http://shopnow.netscape.com/ > > Get your own FREE, personal Netscape Mail account today at > http://webmail.netscape.com/ > > > _______________________________________________ > Python-LDAP-dev mailing list > Pyt...@li... > https://lists.sourceforge.net/lists/listinfo/python-ldap-dev > |
|
From: Michael <mi...@st...> - 2001-10-17 07:57:01
|
js...@ne... wrote:
>
> The iconv is part of glibc 2.2.2 and above, so iconv.h should be in
> any recent LINUX system
web2ldap based on python-ldap aims to run on Win32 platform too...
Anyway all character set translation should be based on Unicode
support introduced in Python 1.6 because the codec concept looks
very clean. There are Unicode codecs for Python based on iconv lib
if the built-in Python codecs are not sufficient.
> I did try to use PyString_(De/En)code, but it is not very well
> suited to openldap type of applications, especially if dn has mixed
> ascii and utf8 encodings.
I'm not sure what you mean with "mixed ascii and utf8 encodings".
UTF-8 is a variable length encoding of the ISO-10646 character set
which maps the page containing the ASCII characters directly to
ASCII.
Example with my last name:
>>> unicode('Ströder','iso-8859-1').encode('utf-8')
'Str\xc3\xb6der'
Ciao, Michael.
|
