python-ldap-dev

encodings while modify AD with linux-client

[Ingo S. <st...@un...>]

Hello

I've got some encoding-trouble accessing Active Directory with python-ldap=
=20
(slightly modified 2.0.6) on linux (openldap-client 2.1.30).

I can modify a container-description using umlauts without an Error/Excepti=
on:

[..]
lo.simple_bind_s(login_dn, login_pw)

modlist=3D[(ldap.MOD_REPLACE,"description",[u"hall=F6le".encode('ISO-8859-1=
')])]
lo.modify_s("CN=3Dmoreusers,CN=3DUsers,%s"%ldap_base,modlist)
[..]

Search for this container or view it in windows will result the description=
=20
"hallle" (umlauts are "cutted"). Using an different encoding (different fro=
m=20
ISo-8859-1/Latin) will result in an exception, like this one with unicode:

  File "/usr/lib/python2.3/site-packages/ldap/ldapobject.py", line 298, in=
=20
modify_s
    msgid =3D self.modify(dn,modlist)
  File "/usr/lib/python2.3/site-packages/ldap/ldapobject.py", line 295, in=
=20
modify
    return self.modify_ext(dn,modlist,None,None)
  File "/usr/lib/python2.3/site-packages/ldap/ldapobject.py", line 268, in=
=20
modify_ext
    return=20
self._ldap_call(self._l.modify_ext,dn,modlist,serverctrls,clientctrls)
  File "/usr/lib/python2.3/site-packages/ldap/ldapobject.py", line 94, in=20
_ldap_call
    result =3D func(*args,**kwargs)
TypeError: ('expected a string in the list', u'hal\xf6le')

It is documented that Active Directory uses ISO-8859-1 and not utf8 (like=20
Openldap an others).=20

So, is this a python-ldap or openldap-problem (I know, great chanceto start=
 an=20
AD-Flamewar)? Any experiences/solutions?

Thanks!

Ingo Steuwer

=2D-=20
Ingo Steuwer       st...@un...         fon: +49 421 22 232- 0
Entwicklung        Linux for Your Business
Univention GmbH    http://www.univention.de/     fax: +49 421 22 232-99

Re: encodings while modify AD with linux-client

[<mi...@st...>]

Ingo Steuwer wrote:
> 
> modlist=[(ldap.MOD_REPLACE,"description",[u"hallöle".encode('ISO-8859-1')])]
> lo.modify_s("CN=moreusers,CN=Users,%s"%ldap_base,modlist)

LDAPv3 mandates use of Unicode with UTF-8 encoding. You are accessing
Active Directory via its LDAPv3 interface. Hence it expects UTF-8.

> It is documented that Active Directory uses ISO-8859-1 and not utf8 (like 
> Openldap an others). 

I doubt that. Well, depends on what "Active Directory uses ISO-8859-1"
really means...

But the charset of the internal storage of AD is not relevant when
accessing it through LDAPv3.

> So, is this a python-ldap or openldap-problem

Nope. The applications using python-ldap are responsible to provide the
proper charset and encoding at the API level.

> (I know, great chanceto start an AD-Flamewar)?

(Not at all.)

> Any experiences/solutions?

I've tried it. It works for me with UTF-8.

Ciao, Michael.

Re: encodings while modify AD with linux-client

[Ingo S. <st...@un...>]

Am Montag, 2. Mai 2005 20:22 schrieb Michael Str=F6der:
> Ingo Steuwer wrote:
> > modlist=3D[(ldap.MOD_REPLACE,"description",[u"hall=F6le".encode('ISO-88=
59-1')
> >])] lo.modify_s("CN=3Dmoreusers,CN=3DUsers,%s"%ldap_base,modlist)
>
> LDAPv3 mandates use of Unicode with UTF-8 encoding. You are accessing
> Active Directory via its LDAPv3 interface. Hence it expects UTF-8.
>
> > It is documented that Active Directory uses ISO-8859-1 and not utf8 (li=
ke
> > Openldap an others).
>
> I doubt that. Well, depends on what "Active Directory uses ISO-8859-1"
> really means...

This was mentioned in a documentation of a python-ldap-based tool a can't f=
ind=20
anymore... google stuff.

> But the charset of the internal storage of AD is not relevant when
> accessing it through LDAPv3.
>
> > So, is this a python-ldap or openldap-problem
>
> Nope. The applications using python-ldap are responsible to provide the
> proper charset and encoding at the API level.
>
[..]
>
> I've tried it. It works for me with UTF-8.

Just to be sure: You tried it which way? My experiences are that python-lda=
p=20
doesn't allow the use of unicode in a modlist (see Exception in my first=20
mail, "expected a string in the list"). Which python-version do you use?

=46urthermore it seems to me that python-ldap does not use unicode internal=
y. If=20
I read from AD I get unicode strings which are handled by python like Latin=
=20
=2D-  which means I have to convert them like "unicode(value,'utf8')". I=20
expected that modlist will need also unicode as normal strings but that wil=
l=20
give me the other conversion-exception ("ordinal not in range(128)").

Regards
Ingo Steuwer

> Ciao, Michael.

=2D-=20
Ingo Steuwer       st...@un...         fon: +49 421 22 232- 0
Entwicklung        Linux for Your Business
Univention GmbH    http://www.univention.de/     fax: +49 421 22 232-99

Re: encodings while modify AD with linux-client

[Ingo S. <st...@un...>]

OK, got it -- but it was't obvious...:

modlist=3D[(ldap.MOD_REPLACE,"description","hall=F6le".decode('latin').enco=
de('utf8'))]

Greetings
Ingo

Am Dienstag, 3. Mai 2005 09:41 schrieb Ingo Steuwer:
> Am Montag, 2. Mai 2005 20:22 schrieb Michael Str=F6der:
> > Ingo Steuwer wrote:
> > > modlist=3D[(ldap.MOD_REPLACE,"description",[u"hall=F6le".encode('ISO-=
8859-1
> > >') ])] lo.modify_s("CN=3Dmoreusers,CN=3DUsers,%s"%ldap_base,modlist)
> >
> > LDAPv3 mandates use of Unicode with UTF-8 encoding. You are accessing
> > Active Directory via its LDAPv3 interface. Hence it expects UTF-8.
> >
> > > It is documented that Active Directory uses ISO-8859-1 and not utf8
> > > (like Openldap an others).
> >
> > I doubt that. Well, depends on what "Active Directory uses ISO-8859-1"
> > really means...
>
> This was mentioned in a documentation of a python-ldap-based tool a can't
> find anymore... google stuff.
>
> > But the charset of the internal storage of AD is not relevant when
> > accessing it through LDAPv3.
> >
> > > So, is this a python-ldap or openldap-problem
> >
> > Nope. The applications using python-ldap are responsible to provide the
> > proper charset and encoding at the API level.
>
> [..]
>
> > I've tried it. It works for me with UTF-8.
>
> Just to be sure: You tried it which way? My experiences are that
> python-ldap doesn't allow the use of unicode in a modlist (see Exception =
in
> my first mail, "expected a string in the list"). Which python-version do
> you use?
>
> Furthermore it seems to me that python-ldap does not use unicode internal=
y.
> If I read from AD I get unicode strings which are handled by python like
> Latin --  which means I have to convert them like "unicode(value,'utf8')".
> I expected that modlist will need also unicode as normal strings but that
> will give me the other conversion-exception ("ordinal not in range(128)").
>
> Regards
> Ingo Steuwer
>
> > Ciao, Michael.

=2D-=20
Ingo Steuwer       st...@un...         fon: +49 421 22 232- 0
Entwicklung        Linux for Your Business
Univention GmbH    http://www.univention.de/     fax: +49 421 22 232-99