python-ldap-dev

Re: Implementation of LDAPControls

[Deepak G. <de...@ar...>]

Hi, guys! 

Has there been any progress on implementing LDAPControl objects in
python-ldap? There were some messages posted on the list in late
December/early January about this, but I haven't heard much since then.

The project I'm working on is at a point where this feature is a
necessity, so if implementation of this feature is at a standstill I'm
willing to get the ball rolling again.

Is there any work that's already begun? I didn't see anything in the CVS
repository. If anyone has any existing code, I'd much appreciate it if I
could take a look at it.

Cheers!
Deepak

ps - earlier, my colleague posted on the OpenLDAP mailing list about
what we're trying to accomplish:
http://www.openldap.org/lists/openldap-software/200502/msg00392.html

--
Deepak Giridharagopal
Applied Research Laboratories
University of Texas at Austin

Re: Implementation of LDAPControls

[Deepak G. <de...@ar...>]

Hello, me again. :)

On Thu, 2005-02-17 at 16:30 -0600, Deepak Giridharagopal wrote:
> The project I'm working on is at a point where this feature is a
> necessity, so if implementation of this feature is at a standstill I'm
> willing to get the ball rolling again.

As a proof of concept, I've gone ahead and begun coding up support for
LDAP Controls. My goal for the test was to successfully do an
ldap_modify_ext operation on our Active Directory server, using AD's
Security Descriptor Modification control (http://tinyurl.com/5tlok).

It works great!

I've only so far implemented control support in:
set_option
ldap_search_ext
ldap_modify_ext

...but adding support for the other LDAP operations (I hope) should be
easy.

My approach has been to model the OpenLDAP LDAPControl struct as a
tuple:

(OID <string>, Criticality Flag <boolean>, Value <string/list of bytes>)

The "Value" field needs to be an ASN.1 encoded list of bytes. I've taken
the position that however the user actually encodes his Python data
structure into ASN.1 is (for the moment) not my concern. This way, the C
code remains simple. 

I figure that once we settle on a pure-python ASN.1 encoding module, we
can handle marshalling/unmarshalling an LDAPControl object's payload at
a higher level than the C code, perhaps in a utility module like
"modlist.py" ("control_builder.py"?).

That said, for the moment I've been using Pices' ASN.1 encoding module.
It's a single Python file and trivial to use:

http://www.cnri.reston.va.us/software/pisces/manual/module-pisces.asn1.html

Here is the code I use to set a control:

#######################
import asn1 # This is the Pices module

# This is the OID for the MS security descriptor control I described
# above.
oid = "1.2.840.113556.1.4.801"
criticality = 1

# Here's where I actually construct the payload. The payload for this
# AD control needs to be an ASN.1 sequence with a single int inside.
payload = asn1.Sequence()
payload.append(0xf)

# Ta da, here is the control
control = ( oid, criticality, payload.encode() )

# Now do the operation
dn = "cn=foobar,cn=users,dc=activedirectory,dc=com"
modifications = [(ldap.MOD_REPLACE, "ntSecurityDescriptor", "blah")]
conn.modify_ext_s(dn, modifications, serverctrls=[control])
######################

Like I said, it works great!

The C code I've written looks very similar to the existing code in
LDAPObject.c that handles LDAPMod objects (I used the LDAPMod code as a
template for my stuff). If you guys want, I can start throwing patches
your way. 

Any thoughts on all of this?

Cheers!
deepak

--
Deepak Giridharagopal
Applied Research Laboratories
University of Texas at Austin

Re: Implementation of LDAPControls

[Ingo S. <st...@un...>]

Am Freitag, 18. Februar 2005 19:53 schrieb Deepak Giridharagopal:
> Hello, me again. :)
>
> On Thu, 2005-02-17 at 16:30 -0600, Deepak Giridharagopal wrote:
> > The project I'm working on is at a point where this feature is a
> > necessity, so if implementation of this feature is at a standstill I'm
> > willing to get the ball rolling again.
>
> As a proof of concept, I've gone ahead and begun coding up support for
> LDAP Controls. My goal for the test was to successfully do an
> ldap_modify_ext operation on our Active Directory server, using AD's
> Security Descriptor Modification control (http://tinyurl.com/5tlok).
>
> It works great!
>

Hello

I answered your mail yesterday but it was rejected because my attachement w=
as=20
too big. Enclosed you find my mail including the necessary code-extraction=
=20
for basic ldap-controls in python-ldap.

Regards
Ingo Steuwer

> Hello
>=20
> we've implemented basic support for LDAP-Controls in Python LDAP. We defi=
ned=20
> an LDAPControl Python-class and some helper function and pre-defined OID'=
s=20
> for using it. It is prepared for but lacks support of bervals (we don't n=
eed=20
> it at the moment). We're using this version in our test-cases and think i=
t=20
is=20
> stable.
>=20
> You can i.e. see deleted DN's in AD by using:
>=20
> ------------------------------------------------------------------------
> import ldap
> lo =3D ldap.open("ad-host")
> login_dn =3D "cn=3DAdministrator,cn=3DUsers,dc=3Dwindomain"
> login_pw =3D 'secret'
>=20
> lo.simple_bind_s(login_dn, login_pw)
> lc1 =3D ldap.create_control("LDAP_SERVER_SHOW_DELETED_OID")
> res =3D lo.search_ext_s(
> =A0"dc=3Dw2k3st,dc=3Dunivention,dc=3Dde",ldap.SCOPE_SUBTREE,sys.argv[1],s=
erverctrls=3D[lc1])
> # print res
> ------------------------------------------------------------------------
>=20
> I wanted to send you this files earlier, but didn't had the time for it.=
=20
> Attached you will find a tgz of "our" python-ldap -- I'm still to short o=
n=20
> time to make a patch...
>=20
> Included is an other extension of python-ldap which makes it possible to=
=20
> request ldap-schema-definitions from an open-ldap-server. We're using thi=
s=20
> for a long time but AFAIK it is not our implementation (at least not mine=
).
>=20
> To get a part of the "official" python-ldap package our=20
> LDAP-Control-Implementation is provided with acknowledgment of the=20
> python-license and there provided without any guaranty -- which means we =
are=20
> not responsible for your problems with our code. Enough=20
> "Legal-Department"-comments.
>=20
> We're interested in your meanings and experiences looking at or using our=
=20
code=20
> and will try to help if there are any problems.
>=20
> Regards
> Ingo Steuwer
>=20


=2D-=20
Ingo Steuwer       st...@un...         fon: +49 421 22 232- 0
Entwicklung        Linux for Your Business
Univention GmbH    http://www.univention.de/     fax: +49 421 22 232-99

Re: Implementation of LDAPControls

[<mi...@st...>]

Ingo Steuwer wrote:
> 
> Enclosed you find my mail including the necessary code-extraction 
> for basic ldap-controls in python-ldap.

Glad to see some progress in this field. Thanks.

Could you please send a unified diff against the CVS tree? I will take a 
closer look at it.

There's also another issue with licensing:
Which plans Univention has with this code? Would Univention be willing 
to give away copyright so that some time in the future we can 
stream-line the licensing?
I'm not a lawyer. But I guess strictly speaking we would need kind of a 
legal person to give the copyright to. Maybe contributing the code under 
Python style license would be easier...sigh!

Please consult your management and your legal department before we can 
incorporate code into the python-ldap CVS.

Ciao, Michael.

Re: Implementation of LDAPControls

[Deepak G. <de...@ar...>]

On Mon, 2005-02-21 at 19:02 +0100, Michael Str=F6der wrote:
> Ingo Steuwer wrote:
> >=20
> > Enclosed you find my mail including the necessary code-extraction=20
> > for basic ldap-controls in python-ldap.
>=20
> Glad to see some progress in this field. Thanks.
>=20
> Could you please send a unified diff against the CVS tree? I will take =
a=20
> closer look at it.

Well, I'm not Ingo, but for my university here I've finished support for
LDAP controls. This includes server and client control support for every
ldap_*_ext method in LDAPObject.c, and I've also implemented set_option
and get_option (from options.c) for both types of controls as well.

My modifications of the ldap_*_ext functions are always the same 3
steps:
1) Parse the serverctrls and clientctrls PyObjects into an array of
LDAPControl structs

2) Modify the actual libldap function call to use those parsed arrays
instead of NULL like we're doing now

3) De-allocate those newly created LDAPControl arrays.

You can see an example of usage in my previous email to the list.

I've attached a unified diff of my changes, which as I said before are
modeled closely after the existing implementation of LDAPMod structures.

> I'm not a lawyer. But I guess strictly speaking we would need kind of a=
=20
> legal person to give the copyright to. Maybe contributing the code unde=
r=20
> Python style license would be easier...sigh!

Copyright for the patch is all yours! I wouldn't turn down a mention at
the bottom of the README file, though. :)

Cheers!
deepak

--
Deepak Giridharagopal
Applied Research Laboratories
University of Texas at Austin

Re: Implementation of LDAPControls

[<mi...@st...>]

Deepak Giridharagopal wrote:
> 
> I've attached a unified diff of my changes, which as I said before are
> modeled closely after the existing implementation of LDAPMod structures.

I've committed your patches to CVS since they seem to be the most 
complete solution. But I'd really prefer to have code without 
goto-statements. Could you please work out a solution for this? And 
maybe it would be nice to extract the helper functions for LDAPControls 
to a separate C source text called ldapcontrols.c?

One thing I wonder about is whether it's possible to send a control 
where the control value is absent (e.g. by passing None as control 
value)? See RFC3296 chapter 3 for an example (ManageDsaIT Control).

Now for the Python parts:

I fixed some wrong argument lists in ldap.ldapobject.LDAPObject to make 
it work correctly at that level.
(Note that importing _ldap is considered bad!)

I also started to work on a basic Python class API in sub-module 
ldap.controls. I could not test it yet. But you get the idea.

Regarding ManageDsaIT Control: Once support for LDAPControls is complete 
I think we SHOULD drop l_ldap_manage_dsa_it() in LDAPObject.c and 
re-implement it completely in method 
ldap.ldapobject.LDAPObject.manage_dsa_it().

> I wouldn't turn down a mention at
> the bottom of the README file, though. :)

Already in CVS. Thanks again for working on it!

Ciao, Michael.

Re: Implementation of LDAPControls

[Deepak G. <de...@ar...>]

On Fri, 2005-02-25 at 18:29 +0100, Michael Str=F6der wrote:
> I've committed your patches to CVS since they seem to be the most=20
> complete solution.=20

Great! :)=20

Regarding your suggestions, I've attached a new patch (I diff'ed against
the cvs HEAD).=20

> But I'd really prefer to have code without=20
> goto-statements. Could you please work out a solution for this?=20

Done!

(FWIW, I'd prefer to have code without goto-statements, too, but I'm of
the opinion that error handling in C is one of the few situations in
which goto's actually *help* readability. But since you're the boss,
I'll heed your conventions. :) )

> And=20
> maybe it would be nice to extract the helper functions for LDAPControls=
=20
> to a separate C source text called ldapcontrols.c?

Done!

> One thing I wonder about is whether it's possible to send a control=20
> where the control value is absent (e.g. by passing None as control=20
> value)? See RFC3296 chapter 3 for an example (ManageDsaIT Control).

Done!

(Before, an error would have been thrown. Now, 'None' is permitted.)

> Thanks again for working on it!

No problem. :)

Cheers!
deepak

--
Deepak Giridharagopal
Applied Research Laboratories
University of Texas at Austin

Re: Implementation of LDAPControls

[<mi...@st...>]

Deepak Giridharagopal wrote:
 >
 > Regarding your suggestions, I've attached a new patch (I diff'ed
 > against the cvs HEAD).

Thanks again for contributing! Committed to CVS.

 >>One thing I wonder about is whether it's possible to send a control
 >>where the control value is absent (e.g. by passing None as control
 >>value)? See RFC3296 chapter 3 for an example (ManageDsaIT Control).
 >
 > Done!
 > (Before, an error would have been thrown. Now, 'None' is permitted.)

Now I can start re-implementing LDAPObject.manage_dsa_it()...

Ciao, Michael.

Seg fault with ldap.get_option(ldap.OPT_SERVER_CONTROLS)

[<mi...@st...>]

Michael Ströder wrote:
> Deepak Giridharagopal wrote:
>  >
>  > Regarding your suggestions, I've attached a new patch (I diff'ed
>  > against the cvs HEAD).
> 
> Thanks again for contributing! Committed to CVS.

I have problems using ldap.get_option(ldap.OPT_SERVER_CONTROLS):

$ python2.4 -c "import ldap;ldap.get_option(ldap.OPT_SERVER_CONTROLS)"
Segmentation fault

Any clue? I'm not a C programmer so I can't tell where the bug lies.

Ciao, Michael.

Re: Seg fault with ldap.get_option(ldap.OPT_SERVER_CONTROLS)

[Deepak G. <de...@ar...>]

On Tue, 2005-03-01 at 20:41 +0100, Michael Str=F6der wrote:
> I have problems using ldap.get_option(ldap.OPT_SERVER_CONTROLS):
>=20
> $ python2.4 -c "import ldap;ldap.get_option(ldap.OPT_SERVER_CONTROLS)"
> Segmentation fault
>=20
> Any clue? I'm not a C programmer so I can't tell where the bug lies.

I found it...there was one spot where I forgot to check to make sure a
pointer wasn't NULL before I messed with it. Embarrassing! :) It's a
2-line fix, and I've attached the patch.

Maybe we should start throwing together some simple tests and throw them
into Tests/Lib/ldap/test_ldapcontrols.py or the like?

Also, as an aside, I noticed that in the CVS head the setup.py file
doesn't include the new ldapcontrol.c file in its list of files to be
compiled. This (a 1-line change) is also in the patch.

Cheers!
deepak

--
Deepak Giridharagopal
Applied Research Laboratories
University of Texas at Austin

Re: Seg fault with ldap.get_option(ldap.OPT_SERVER_CONTROLS)

[<mi...@st...>]

Deepak Giridharagopal wrote:
> On Tue, 2005-03-01 at 20:41 +0100, Michael Ströder wrote:
> 
>>I have problems using ldap.get_option(ldap.OPT_SERVER_CONTROLS):
>>
>>$ python2.4 -c "import ldap;ldap.get_option(ldap.OPT_SERVER_CONTROLS)"
>>Segmentation fault
> 
> I found it...there was one spot where I forgot to check to make sure a
> pointer wasn't NULL before I messed with it.

Thanks for the quick fix. Seems to work.

> Also, as an aside, I noticed that in the CVS head the setup.py file
> doesn't include the new ldapcontrol.c file in its list of files to be
> compiled.

Already noticed that. Please sync your CVS and test.

Ciao, Michael.

Re: Implementation of LDAPControls

[<mi...@st...>]

Deepak Giridharagopal wrote:
> 
> That said, for the moment I've been using Pices' ASN.1 encoding module.
> It's a single Python file and trivial to use:
> 
> http://www.cnri.reston.va.us/software/pisces/manual/module-pisces.asn1.html

I use a slightly improved version of it in web2ldap for the built-in 
X.509 certificate/CRL parser.

There are some issues with the CNRI license which prevents this module 
to be shipped with Debian Linux. See http://bugs.debian.org/80195 for a 
discussion about it.

Jeremy, the author of picses, already planned to negotiate a more 
liberal license with CNRI. A similar license issue was solved for Python 
itself before. But AFAIK no progress so far.

I have to look into module pysnmp.ber. Found some old conversation with 
Ilya in my archive about implementing a ASN.1 module...
(In the year 2000! time is running...!)

Ciao, Michael.

Re: Implementation of LDAPControls

[<mi...@st...>]

Deepak Giridharagopal wrote:
> 
> My approach has been to model the OpenLDAP LDAPControl struct as a
> tuple:
> 
> (OID <string>, Criticality Flag <boolean>, Value <string/list of bytes>)

I'd prefer to have a dedicated class hierachy for this in a separate 
sub-module ldap.controls. I will provide something for it which is 
compatible with your modifications to LDAPObject.c.

> The "Value" field needs to be an ASN.1 encoded list of bytes.
> [..]
> # Here's where I actually construct the payload. The payload for this
> # AD control needs to be an ASN.1 sequence with a single int inside.
> payload = asn1.Sequence()
> payload.append(0xf)
> 
> # Ta da, here is the control
> control = ( oid, criticality, payload.encode() )

I think we could move the invocation of .encode() for all controls in a 
list into the wrapper classldap.ldapobject.LDAPObject. Together with a 
well-defined control class possible type conflicts would clash early in 
the Python wrapper code. This makes debugging easier.

Ciao, Michael.

Re: Implementation of LDAPControls

[Deepak G. <de...@ar...>]

On Tue, 2005-02-22 at 09:57 +0100, Michael Str=F6der wrote:
> > My approach has been to model the OpenLDAP LDAPControl struct as a
> > tuple:
> >=20
> > (OID <string>, Criticality Flag <boolean>, Value <string/list of byte=
s>)
>=20
> I'd prefer to have a dedicated class hierachy for this in a separate=20
> sub-module ldap.controls. I will provide something for it which is=20
> compatible with your modifications to LDAPObject.c.

Great! I held off on implementing a higher-level abstraction since I
wanted to get opinions first on the lower level patch. But it's awesome
that we're moving forward on this. :)

On Tue, 2005-02-22 at 09:43 +0100, Michael Str=F6der wrote:=20
> Jeremy, the author of picses, already planned to negotiate a more=20
> liberal license with CNRI. A similar license issue was solved for Pytho=
n=20
> itself before. But AFAIK no progress so far.

That's too bad. :( I'll take a look at pysnmp.ber and see how it
compares.

Cheers!
deepak

--
Deepak Giridharagopal
Applied Research Laboratories
University of Texas at Austin

Re: Implementation of LDAPControls

[Ingo S. <st...@un...>]

Am Dienstag, 22. Februar 2005 09:57 schrieb Michael Str=F6der:
> Deepak Giridharagopal wrote:
> > My approach has been to model the OpenLDAP LDAPControl struct as a
> > tuple:
> >
> > (OID <string>, Criticality Flag <boolean>, Value <string/list of bytes>)
>
> I'd prefer to have a dedicated class hierachy for this in a separate
> sub-module ldap.controls. I will provide something for it which is
> compatible with your modifications to LDAPObject.c.
>
> > The "Value" field needs to be an ASN.1 encoded list of bytes.
> > [..]
> > # Here's where I actually construct the payload. The payload for this
> > # AD control needs to be an ASN.1 sequence with a single int inside.
> > payload =3D asn1.Sequence()
> > payload.append(0xf)
> >
> > # Ta da, here is the control
> > control =3D ( oid, criticality, payload.encode() )
>
> I think we could move the invocation of .encode() for all controls in a
> list into the wrapper classldap.ldapobject.LDAPObject. Together with a
> well-defined control class possible type conflicts would clash early in
> the Python wrapper code. This makes debugging easier.

This is prepared in our implementation, we would see the BerVal (or "payloa=
d")=20
as python-class which internaly saves types and values which should be used=
=20
for encoding. But I don't expect that we will implement this in the near=20
future...

Regards
Ingo


> Ciao, Michael.
>
>
> -------------------------------------------------------
> SF email is sponsored by - The IT Product Guide
> Read honest & candid reviews on hundreds of IT Products from real users.
> Discover which products truly live up to the hype. Start reading now.
> http://ads.osdn.com/?ad_id=3D6595&alloc_id=3D14396&op=3Dclick
> _______________________________________________
> Python-LDAP-dev mailing list
> Pyt...@li...
> https://lists.sourceforge.net/lists/listinfo/python-ldap-dev

=2D-=20
Ingo Steuwer       st...@un...         fon: +49 421 22 232- 0
Entwicklung        Linux for Your Business
Univention GmbH    http://www.univention.de/     fax: +49 421 22 232-99