python-ldap-dev — discussion among developers

Re: Tagging the CVS archive

[Konstantin C. <Kon...@da...>]

Hi All,

I can't understand why I haven't received a few recent messages sent to
the mailing list. I've checked that I'm subscribed. Anyway,

>     FROM: Steffen Ries
>     DATE: 07/15/2001 06:17:50
>     SUBJECT: RE:  Tagging the CVS archive
>
>     Michael Ströder <> writes:
>
>     > Could we find some meaningful set of tags to mark the python-ldap
>     > CVS archive? Maybe we can borrow some ideas from
>     > http://www.openldap.org/software/repo.html ?
>     >
>     I think you should create a branch tag for OpenLDAP 1 related
>     stuff. That way both fixes for the "old" API and new stuff can go in
>     in parallel.
>
>     I would suggest a scheme where upper case tags are applied to
>     releases, and lower case tags are branches.
>
>     e.g.:
>     REL-1-10ALPHA3
>     REL-1-11
>     ...
>     openldap-1-api
>     ...
>
Are you sure the tags are case sensitive?
The other thing: I'd personally prefer (I'm not insisting though :-) "_"
instead of "-".
In any case, if we're going to have branches dependant on the OpenLDAP
API support, they could be called just OPENLDAP_1 or similar, while
releases could be named REL_1_11 and REL_1_10.

Unfortunately, I don't have enough experience in CVS to make branches.
Could somebody do that?

Regards,
    Konstantin.

--
          * *        Konstantin Chuguev - Application Engineer
       *      *              Francis House, 112 Hills Road
     *                       Cambridge CB2 1PQ, United Kingdom
 D  A  N  T  E       WWW:    http://www.dante.net

Re: Tagging the CVS archive

[Steffen R. <ste...@sy...>]

Michael Str=F6der <mi...@st...> writes:

> HI!
>=20
> Could we find some meaningful set of tags to mark the python-ldap
> CVS archive? Maybe we can borrow some ideas from
> http://www.openldap.org/software/repo.html ?
>=20
> I'd like to check in the new OpenLDAP 2-related patches and memory
> leak fixes since I spent some time sending patch mails around. But
> before that I'd like to mark the current CVS version with
> PYTHONLDAP_REL_ENG_1_2 or something alike. Not being a CVS expert
> though... :-/

I think you should create a branch tag for OpenLDAP 1 related
stuff. That way both fixes for the "old" API and new stuff can go in
in parallel.

I would suggest a scheme where upper case tags are applied to
releases, and lower case tags are branches.

e.g.:
REL-1-10ALPHA3
REL-1-11
...
openldap-1-api
...

The head revision ("main trunk") holds the latest and greatest
version, leading towards the next major release. The side branch is
for fixing bugs in the current release (memory leaks...). The
openldap-1-api branch should lead to a 1.10final release.

/steffen
--=20
ste...@sy...	<> Gravity is a myth -- the Earth sucks!

Drop CIDict (was: start tls)

[Michael <mi...@st...>]

Steffen Ries wrote:
> 
> David's CIDict patches
> are not included, so I disabled CIDict to get it running with
> python2.1.

BTW: Shouldn't we drop the CIDict thingy completely?

It can be easily implemented in Python by deriving from
UserDict.UserDict. IMHO we should leave that up to the Python part
for avoiding code bloat at the C wrapper level. Also CIDict is not
capable of handling more of the upcoming aliases problems with
attribute type names. Therefore it's almost useless anyway.

Ciao, Michael.

Re: OpenLDAP 2.0.x's libldap_r

[Stig V. <Sti...@un...>]

On Sun, Jul 15, 2001 at 01:19:35AM +0200, Michael Str=F6der wrote:
> HI!
>=20
> While we're at it: One thing which might bite other python-ldap
> developers as well is that the OpenLDAP libs are not reentrant.
> Despite the ugly hack with my wrapper module ldapthreadlock the
> problem is still not really solved.

I don't know much about this, it is something that I'll have to
look into for PHP as well when it's used with Apache 2.0 etc.
I generally thought that the API was thread safe if one made
sure all threads used different links, and that the _r lib was
for the case with several on the same link. But this is more or
less guessing. The new API is more threading friendly since there
are less use of static buffers. I can't think of any place with
static buffers right now.

One way of writing the Python LDAP API might be to map the low
level Python functions and write some higher level functions in
Python if they are needed. PHP has both high and low level LDAP
functions, but they are all written in C. And I suppose we should
try to make that thread safe.

I have still not looked at the Python API, so I don't really know
what I'm talking about (:

Stig

OpenLDAP 2.0.x's libldap_r

[Michael <mi...@st...>]

HI!

While we're at it: One thing which might bite other python-ldap
developers as well is that the OpenLDAP libs are not reentrant.
Despite the ugly hack with my wrapper module ldapthreadlock the
problem is still not really solved.

The message from Kurt Zeilenga (attached below) made me curious.
Unfortunately Kurt refused to be more verbose on that. Could anybody
look into this? One idea might be to wrap the lower-level function
calls of libldap_r and write a Python wrapper module above it which
takes care about how libldap_r has to be used.

Ciao, Michael.

-------- Original Message --------
Subject: Re: OpenLDAP libs thread-safe?
Date: Fri, 13 Apr 2001 09:57:05 -0700
From: "Kurt D. Zeilenga" <Ku...@Op...>
To: mi...@st...
CC: openldap-software <ope...@Op...>
References: <5.0.2.1.0.20010413085505.00a97600@127.0.0.1>

At 06:43 PM 4/13/01 +0200, Michael Str=F6der wrote:
>"Kurt D. Zeilenga" wrote:
>> =

>> At 02:42 PM 4/13/01 +0200, Michael Str=F6der wrote:
>> >The OpenLDAP 1.2.11 libs does not seem to be thread-safe. How about
>> >OpenLDAP 2.0.x libs?
>> =

>> While 2.0 provides an experimental -lldap_r.  It is not generally
>> thread-safe, but thread-safe if used in a specific manner.
>
>Can you please elaborate on that?

No easily.  Use of -lldap_r really requires one understands
how it is implemented and how its being used.  I won't attempt
to generalize the cases where it might be used.

Kurt

Module package ldaputil

[Michael <mi...@st...>]

HI!

I'd like to contribute some stuff to python-ldap I've implemented
for web2ldap after having it cleaned up.

Find attached a small module package containing two sub-modules.
These modules need Python 2.0+ (e.g. for Unicode support).

passwd
   Class implementing client-side password setting
   for various syntaxes. unicodePwd not yet implemented.
ldapurl
   Class for LDAP URL handling.

Off course both modules needs testing! More to come in the future.

I'd appreciate if some feedback would come from the list members.
- How to stuff that into python-ldap
- the APIs are ok?
- Licensing issues (also concering python-ldap in general)

Ciao, Michael.

Commits for python-ldap 1.11 (was: Referral handling in OpenLDAP 2.0.11)

[Michael <mi...@st...>]

Konstantin Chuguev wrote:
> =

> I've just committed the memory leak fix for OpenLDAPv1 only,

Great!

> to avoid dealing with OpenLDAPv2 before we get a new release.
> After that, I'll start committing OpenLDAPv2 patches with memory leak =

> fixes included.

Yepp.

> > See also my inquiry about CVS tagging on the list.
> =

> Haven't received it yet.

I already received it through the list. Attached it again below.

> The new OpenLDAPv1-only version: should it be 1.10beta1 or 1.11?

People were used to 1.10alpha3 as a release and refer to it as
version 1.10. To avoid a mess the next release should be 1.11 as
already set in CVS.

Ciao, Michael.

-------- Original Message --------
Subject: Tagging the CVS archive
Date: Fri, 13 Jul 2001 02:11:01 +0200
From: Michael Str=F6der <mi...@st...>
Reply-To: mi...@st...
Organization: stroeder.com
To: python-ldap-dev <pyt...@li...>

HI!

Could we find some meaningful set of tags to mark the python-ldap
CVS archive? Maybe we can borrow some ideas from
http://www.openldap.org/software/repo.html ?

I'd like to check in the new OpenLDAP 2-related patches and memory
leak fixes since I spent some time sending patch mails around. But
before that I'd like to mark the current CVS version with
PYTHONLDAP_REL_ENG_1_2 or something alike. Not being a CVS expert
though... :-/

Ciao, Michael.

_______________________________________________
Python-LDAP-dev mailing list
Pyt...@li...
http://lists.sourceforge.net/lists/listinfo/python-ldap-dev

Re: Referral handling in OpenLDAP 2.0.11

[Konstantin C. <Kon...@da...>]

Hi Michael,

Michael Ströder wrote:

> Konstantin,
>
> I've attached the latest patched message.c which hopefully contains
> all fixes for the memory leaks in python-ldap. Hm, it's time to
> check-in something to avoid all these patch mails... :-/
>

Thanks for the file. I've just committed the memory leak fix for OpenLDAPv1 only,
to avoid dealing with OpenLDAPv2 before we get a new release.
After that, I'll start committing OpenLDAPv2 patches with memory leak fixes
included.

>
> See also my inquiry about CVS tagging on the list.
>

Haven't received it yet.
The new OpenLDAPv1-only version: should it be 1.10beta1 or 1.11?

Regards,
    Konstantin.

--
          * *        Konstantin Chuguev - Application Engineer
       *      *              Francis House, 112 Hills Road
     *                       Cambridge CB2 1PQ, United Kingdom
 D  A  N  T  E       WWW:    http://www.dante.net

Tagging the CVS archive

[Michael <mi...@st...>]

HI!

Could we find some meaningful set of tags to mark the python-ldap
CVS archive? Maybe we can borrow some ideas from
http://www.openldap.org/software/repo.html ?

I'd like to check in the new OpenLDAP 2-related patches and memory
leak fixes since I spent some time sending patch mails around. But
before that I'd like to mark the current CVS version with
PYTHONLDAP_REL_ENG_1_2 or something alike. Not being a CVS expert
though... :-/

Ciao, Michael.

Re: More memory freeing patches to message.c

[Steffen R. <ste...@sy...>]

Michael Str=F6der <mi...@st...> writes:

> HI!
>=20
> Following some more advice by Stig Venaas (see message below) I've
> ifdef'ed some memory freeing calls provided by Steffen in message.c
> to keep the source compatible to OpenLDAP 1.2.x libs. I've attached
> my recent message.c.
>=20
> Backwards compability is not assumed to be crucial in the long run
> but seems nice if we can achieve it with fairly low efforts.

We need two changes to this version of 'message.c':
"ber_free(ber, 0)" needs to be called for API version 1.2.x too,
*except* when the ldap_next_attribute() has returned NULL. See the
complete patch against CVS below.

Looks like we are closing in now... :-)

/steffen

Index: message.c
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
RCS file: /cvsroot/python-ldap/python-ldap/Modules/message.c,v
retrieving revision 1.6
diff -u -w -r1.6 message.c
--- message.c	2000/08/18 00:21:46	1.6
+++ message.c	2001/07/12 11:31:00
@@ -51,7 +51,11 @@
 	 if (attrdict =3D=3D NULL) {
 		Py_DECREF(result);
 		ldap_msgfree( m );
+#if LDAP_API_VERSION >=3D 2000
+		ldap_memfree(dn);
+#else
 		free(dn);
+#endif
 		return NULL;
 	 }
=20
@@ -79,8 +83,15 @@
 	     if (valuelist =3D=3D NULL) {
 		Py_DECREF(attrdict);
 		Py_DECREF(result);
+		if (ber !=3D NULL)
+			ber_free(ber, 0);
 		ldap_msgfree( m );
+#if LDAP_API_VERSION >=3D 2000
+		ldap_memfree(attr);
+		ldap_memfree(dn);
+#else
 		free(dn);
+#endif
 		return NULL;
 	     }
=20
@@ -97,8 +108,15 @@
 			Py_DECREF(result);
 			Py_DECREF(valuestr);
 			Py_DECREF(valuelist);
+			if (ber !=3D NULL)
+				ber_free(ber, 0);
 			ldap_msgfree( m );
+#if LDAP_API_VERSION >=3D 2000
+			ldap_memfree(attr);
+			ldap_memfree(dn);
+#else
 			free(dn);
+#endif
 			return NULL;
 		    }
 		    Py_DECREF(valuestr);
@@ -106,14 +124,59 @@
 		ldap_value_free_len(bvals);
 	     }
 	     Py_DECREF( valuelist );
+#if LDAP_API_VERSION >=3D 2000
+	     ldap_memfree(attr);
+#endif
 	 }
=20
 	 entrytuple =3D Py_BuildValue("(sO)", dn, attrdict);
+#if LDAP_API_VERSION >=3D 2000
+	 ldap_memfree(dn);
+#else
          free(dn);
+#endif
 	 Py_DECREF(attrdict);
 	 PyList_Append(result, entrytuple);
 	 Py_DECREF(entrytuple);
+#if ( LDAP_API_VERSION > 2000 )
+	 if (ber !=3D NULL)
+		 ber_free(ber, 0);
+#endif
+     }
+#if LDAP_API_VERSION >=3D 2000
+     for(entry =3D ldap_first_reference(ld,m);
+	 entry !=3D NULL;
+	 entry =3D ldap_next_reference(ld,entry))
+     {
+	 char **refs =3D NULL;
+	 PyObject* entrytuple;
+	 PyObject* reflist =3D PyList_New(0);
+
+	 if (reflist =3D=3D NULL)  {
+	     Py_DECREF(result);
+	     ldap_msgfree( m );
+	     return NULL;
+	 }
+	 if (ldap_parse_reference(ld, entry, &refs, NULL, 0) !=3D LDAP_SUCCESS) {
+	     Py_DECREF(result);
+	     ldap_msgfree( m );
+	     return LDAPerror( ld, "ldap_parse_reference" );
+	 }
+	 if (refs) {
+	     int i;
+	     for (i=3D0; refs[i] !=3D NULL; i++) {
+		 PyObject *refstr =3D PyString_FromString(refs[i]);
+		 PyList_Append(reflist, refstr);
+		 Py_DECREF(refstr);
+	     }
+	     ber_memvfree( (void **) refs );
+	 }
+	 entrytuple =3D Py_BuildValue("(sO)", NULL, reflist);
+	 Py_DECREF(reflist);
+	 PyList_Append(result, entrytuple);
+	 Py_DECREF(entrytuple);
      }
+#endif
      ldap_msgfree( m );
      return result;
 }


--=20
ste...@sy...	<> Gravity is a myth -- the Earth sucks!

Re: Referral LDAP URL and slash after hostport

[Henny B. <Hen...@su...>]

Hi Michael,

At 12:23 12-7-2001 +0200, Michael Str=F6der wrote:
>Konstantin Chuguev wrote:
> > Hi Michael and All,
> > Michael Str=F6der wrote:
> > > When using (patched) python-ldap with OpenLDAP 2.0.x libs and I'm
> > > trying to access
> > > ldap://ldap.surfnet.nl/c=3DBE
> > > I get back the referral LDAP URL
> > > ldap://tor.dante.org.uk:1389??base
> > >
> > > That's almost ok. But the slash after hostport is missing. Is that
> > > intentional? IMHO it should be
> > > ldap://tor.dante.org.uk:1389/??base
> >
> > I had a talk with the ldap.surfnet.nl manager, Henny Bekker. There seems
> > to be a bug in their directory server. Henny told me they put the
> > correct data for referrals, but they get changed in LDAP responces. I
> > wasn't entirely convinced in the fact, until you got the same results...
> >
> > Anyway, they are going to migrate from their old server. They are
> > considering OpenLDAPv2.
>
>I've uploaded the root.ldif of the DIRECT project to my local
>OpenLDAP 2.0.11 as well and it seems to work right. Sorry, I did not
>expect the problem to be limited to ldap.surfnet.nl. (Since OpenLDAP
>2 is not able to hold a root naming context I have suffix directives
>for all national referral entries in slapd.conf).
>
> > P.S. Michael, could you try ldap.nameflow.net (root NC) for your
> > referral testing.
>
>That also seems to work quite ok with python-ldap built against
>OpenLDAP 2.
True..  But how about a 'one-level search' over all the defined
countries (in the root.ldif of the DIRECT project)..  If the
server isn't doing any caching of that LDAPv3 referrals (with
should take precedence over the info defined in the entries) a
one-level search will go to the referred site to fetch the info.
Thus a one-level search for e.g. the country-names will result
into querying all referred LDAP-servers which will take to long
(certainly when a country-level LDAP-server is unavailable) and
in not scalable..
See also URL: http://www.terena.nl/libr/tech/2000/direct-fr.pdf


Cheers, Henny

---------------------------------------------------------------------
E-Mail: H.B...@SU... Voice: +31 30 2305305 Fax: +31 30 2305329
Web:    http://www.surfnet.nl/surfnet/persons/henny/              o
Paper:  H.J.Bekker, SURFnet                                   _  /- _
         Po Box 19035, 3501 DA Utrecht, Nederland             (_) > (_)
----------------------------------------------------------------------

Re: Referral LDAP URL and slash after hostport

[Henny B. <Hen...@su...>]

Hi Michael,

At 12:23 12-7-2001 +0200, Michael Str=F6der wrote:
>Konstantin Chuguev wrote:
> > Hi Michael and All,
> > Michael Str=F6der wrote:
> > > When using (patched) python-ldap with OpenLDAP 2.0.x libs and I'm
> > > trying to access
> > > ldap://ldap.surfnet.nl/c=3DBE
> > > I get back the referral LDAP URL
> > > ldap://tor.dante.org.uk:1389??base
> > >
> > > That's almost ok. But the slash after hostport is missing. Is that
> > > intentional? IMHO it should be
> > > ldap://tor.dante.org.uk:1389/??base
> >
> > I had a talk with the ldap.surfnet.nl manager, Henny Bekker. There seems
> > to be a bug in their directory server. Henny told me they put the
> > correct data for referrals, but they get changed in LDAP responces. I
> > wasn't entirely convinced in the fact, until you got the same results...
> >
> > Anyway, they are going to migrate from their old server. They are
> > considering OpenLDAPv2.
>
>I've uploaded the root.ldif of the DIRECT project to my local
>OpenLDAP 2.0.11 as well and it seems to work right. Sorry, I did not
>expect the problem to be limited to ldap.surfnet.nl. (Since OpenLDAP
>2 is not able to hold a root naming context I have suffix directives
>for all national referral entries in slapd.conf).
>
> > P.S. Michael, could you try ldap.nameflow.net (root NC) for your
> > referral testing.
>
>That also seems to work quite ok with python-ldap built against
>OpenLDAP 2.
True..  But how about a 'one-level search' over all the defined
countries (in the root.ldif of the DIRECT project)..  If the
server isn't doing any caching of that LDAPv3 referrals (with
should take precedence over the info defined in the entries) a
one-level search will go to the referred site to fetch the info.
Thus a one-level search for e.g. the country-names will result
into querying all referred LDAP-servers which will take to long
(certainly when a country-level LDAP-server is unavailable) and
in not scalable..
See also URL: http://www.terena.nl/libr/tech/2000/direct-fr.pdf


Cheers, Henny

---------------------------------------------------------------------
E-Mail: H.B...@SU... Voice: +31 30 2305305 Fax: +31 30 2305329
Web:    http://www.surfnet.nl/surfnet/persons/henny/              o
Paper:  H.J.Bekker, SURFnet                                   _  /- _
         Po Box 19035, 3501 DA Utrecht, Nederland             (_) > (_)
----------------------------------------------------------------------

Re: Referral LDAP URL and slash after hostport

[Konstantin C. <Kon...@da...>]

Michael Ströder wrote:

> expect the problem to be limited to ldap.surfnet.nl. (Since OpenLDAP
> 2 is not able to hold a root naming context I have suffix directives
> for all national referral entries in slapd.conf).
>

The patch (attached) for 2.0.11 fixes the problem with root naming contexts.
It is incorporated to the current version of OpenLDAP-2 in CVS. I like open
source software! :-)


--
          * *        Konstantin Chuguev - Application Engineer
       *      *              Francis House, 112 Hills Road
     *                       Cambridge CB2 1PQ, United Kingdom
 D  A  N  T  E       WWW:    http://www.dante.net

Re: Referral LDAP URL and slash after hostport

[Michael <mi...@st...>]

Konstantin Chuguev wrote:
> =

> Hi Michael and All,
> =

> Michael Str=F6der wrote:
> =

> > When using (patched) python-ldap with OpenLDAP 2.0.x libs and I'm
> > trying to access
> > ldap://ldap.surfnet.nl/c=3DBE
> > I get back the referral LDAP URL
> > ldap://tor.dante.org.uk:1389??base
> >
> > That's almost ok. But the slash after hostport is missing. Is that
> > intentional? IMHO it should be
> > ldap://tor.dante.org.uk:1389/??base
> >
> =

> I had a talk with the ldap.surfnet.nl manager, Henny Bekker. There seem=
s
> to be a bug in their directory server. Henny told me they put the
> correct data for referrals, but they get changed in LDAP responces. I
> wasn't entirely convinced in the fact, until you got the same results..=
=2E
> =

> Anyway, they are going to migrate from their old server. They are
> considering OpenLDAPv2.

I've uploaded the root.ldif of the DIRECT project to my local
OpenLDAP 2.0.11 as well and it seems to work right. Sorry, I did not
expect the problem to be limited to ldap.surfnet.nl. (Since OpenLDAP
2 is not able to hold a root naming context I have suffix directives
for all national referral entries in slapd.conf).

> P.S. Michael, could you try ldap.nameflow.net (root NC) for your
> referral testing.

That also seems to work quite ok with python-ldap built against
OpenLDAP 2.

Ciao, Michael.

Re: Referral LDAP URL and slash after hostport

[Konstantin C. <Kon...@da...>]

Hi Michael and All,

Michael Ströder wrote:

> When using (patched) python-ldap with OpenLDAP 2.0.x libs and I'm
> trying to access
> ldap://ldap.surfnet.nl/c=BE
> I get back the referral LDAP URL
> ldap://tor.dante.org.uk:1389??base
>
> That's almost ok. But the slash after hostport is missing. Is that
> intentional? IMHO it should be
> ldap://tor.dante.org.uk:1389/??base
>

I had a talk with the ldap.surfnet.nl manager, Henny Bekker. There seems
to be a bug in their directory server. Henny told me they put the
correct data for referrals, but they get changed in LDAP responces. I
wasn't entirely convinced in the fact, until you got the same results...

Anyway, they are going to migrate from their old server. They are
considering OpenLDAPv2. And here goes a question to the core OpenLDAP
developers:

At a national level, there is a need to build an LDAP server containing
lots (hundreds) of referrals to organisation LDAP servers. We consider
such a server for browsing purposes (one-level search) only. Now, if a
client sends a one level search request, it will get lots (hundreds) of
referrals. It can choke on them easily. The idea is to keep cached
entries along with ref entries. What is needed here is the ability to
switch the request to DSA IT control mode automatically for every
one-level request, even if the client hasn't asked about it.
The patch for it is quite easy. The question is: would it be possible to
add this as a standard server's behaviour (switched by a configuration
directive)?

Regards,
    Konstantin.

--
          * *        Konstantin Chuguev - Application Engineer
       *      *              Francis House, 112 Hills Road
     *                       Cambridge CB2 1PQ, United Kingdom
 D  A  N  T  E       WWW:    http://www.dante.net

P.S. Michael, could you try ldap.nameflow.net (root NC) for your
referral testing.

More memory freeing patches to message.c

[Michael <mi...@st...>]

HI!

Following some more advice by Stig Venaas (see message below) I've
ifdef'ed some memory freeing calls provided by Steffen in message.c
to keep the source compatible to OpenLDAP 1.2.x libs. I've attached
my recent message.c.

Backwards compability is not assumed to be crucial in the long run
but seems nice if we can achieve it with fairly low efforts.

Ciao, Michael.

-------- Original Message --------
Subject: Re: python-ldap and OpenLDAP 2
Date: Thu, 12 Jul 2001 10:47:35 +0200
From: Stig Venaas <Sti...@un...>
To: Michael Str=F6der <mi...@st...>
References: <3B4...@st...>
<200...@sv...>
<3B4...@st...>
<200...@sv...>
<3B4...@st...>

On Thu, Jul 12, 2001 at 09:31:39AM +0200, Michael Str=F6der wrote:
> Stig Venaas wrote:
> > =

> > These memory leaks was also reported and discussed
> > in the OpenLDAP ITS system a few days ago. Someone reported memory
> > leaks and Kurt and I responded. See
> > http://www.OpenLDAP.org/lists/openldap-bugs/200107/msg00018.html
> > for my respons.
> =

> According to this and Steffen's message the ldap_memfree(attr) and
> ber_free(ber, 0) should be ifdef'ed. What about the free(dn) call?
> =

> Well, we have no problems dropping support for linking against
> OpenLDAP 1.x completely if we will have a really stable version and
> it's clear that we can achieve all goals on this track.

PHP supports OpenLDAP 1.x and others that use the RFC1823 API, as
well as Netscape SDK and OpenLDAP 2.x and others that use the new
API. The same should be possible for Python.

I think I've found a bug in PHP when using old API and
ldap_get_dn().
It's not freed! I've never tested much with old API. The PHP code
looks
like this:

#if ( LDAP_API_VERSION > 2000 ) || HAVE_NSLDAP || WINDOWS
                ldap_memfree(dn);
#endif

For Python you could use:
#if ( LDAP_API_VERSION > 2000 )
                ldap_memfree(dn);
#else
		free(dn);
#endif

ldap_memfree() is currently the same as free() but it should be done
this way anyway. Regarding the other leaks, you can use code like:

#if ( LDAP_API_VERSION > 2000 )
                        ldap_memfree(attribute);
#endif

and

#if ( LDAP_API_VERSION > 2000 )
                if (ber !=3D NULL)
                        ber_free(ber, 0);
#endif

This way it should work with both old and new APIs.

> I have changed the calls to free() to ldap_memfree(), which means
> the
> code will compile only against OpenLDAP 2.0.x, since 1.2 does not
> define ldap_memfree().

Hence the trick above.

Stig

Re: compiling against OpenLDAP 2.0.11

[Konstantin C. <Kon...@da...>]

Hi Steffen,

Steffen Ries wrote:

> Michael Ströder <mi...@st...> writes:
>
> > Anyone did notice the forwarded warning message by Stig about
> > possible memory leaks with OpenLDAP 2.0.x API?
> > Please glance over it:
> > http://www.geocrawler.com/archives/3/1568/2001/6/0/6036658/
>
> I looked through it.
>
> The first issue (ldap_first_attribute) has changed from OpenLDAP 1.2
> to 2.0.x so that the memory is *never* freed by the library. In 1.1 it
> was freed, when the last attribute was read. However, the current
> python-ldap code did never free the memory. IOW, there is a possible
> memory leak in error cases.
>

> The second issue (ldap_get_dn) has *not* changed between OpenLDAP 1.2
> and 2.0.x, i.e. the same kind of memory leak exists for both API
> versions.
>

If we are going to release the latest snapshot without OpenLDAPv2 support
(which I think is a good idea), we would still like to eliminate any memory
leaks :-) In that case, could you please update your patch to do The Right
Thing (TM) for OpenLDAPv1 only? If you don't have enough time, I could do
it myself. I'd just like to get a hint in this case: is it only message.c
that needs to be patched?

>
> The attached patch should fix both issues. The last "ber_free()"
> should be surrounded by some #ifdef OPENLDAP_2_0_X conditional...
>
> I scanned through the documentation and did not notice any further
> changes in memory handling, but then again, I did not really look too
> hard.
>

After that, I would add memory leak fixes to my OpenLDAPv2 patches.

Regards,
    Konstantin.

--
          * *        Konstantin Chuguev - Application Engineer
       *      *              Francis House, 112 Hills Road
     *                       Cambridge CB2 1PQ, United Kingdom
 D  A  N  T  E       WWW:    http://www.dante.net

Re: compiling against OpenLDAP 2.0.11

[Konstantin C. <Kon...@da...>]

Michael Ströder wrote:

> David, take care of your PhD-ing and please let me test Konstantin's
> patches first. ;-)
>

Agree, I'm also going to test them, although just in a few
configurations...

> > I'll prepare another patch for python-ldap/Doc/libldap.tex.
>
> Urrgs. I think I should at least publish a "python setup.py sdist"
> of recent snapshot to make a 1.11 distribution before the commits.
> Comments?
>

As I can see, the current version from CVS hasn't got new features for a
long time, just bug fixes. So, it's a good candidate for a release.
After that, we could add OpenLDAPv2 functionality to the currnet branch
of the repository.

    Konstantin.

--
          * *        Konstantin Chuguev - Application Engineer
       *      *              Francis House, 112 Hills Road
     *                       Cambridge CB2 1PQ, United Kingdom
 D  A  N  T  E       WWW:    http://www.dante.net

Re: Subtle changes in error handling with OpenLDAP 2 (was: compiling against OpenLDAP 2.0.11)

[Michael <mi...@st...>]

Michael Str=F6der wrote:
> =

> The ldap.LDAPError base class was raised with
> OpenLDAP 2.0.x lib instead of e.g. ldap.PARTIAL_RESULTS raised in
> case of OpenLDAP 1.2.x lib.

Discovered exception class ldap.REFERRAL introduced with LDAPv3 by
looking at ldap.h of OpenLDAP 2 and errors.c...

Ciao, Michael.

Re: compiling against OpenLDAP 2.0.11

[Michael <mi...@st...>]

Michael Str=F6der wrote:
> =

> Konstantin Chuguev wrote:
> >
> > Yes, OpenLDAP-2.0.11 and especially recent changes in python-ldap CVS=

> > repository require a new version of patches. I've attached them.
> =

> I tried these patches.

BTW: I see a lot of hex dump output (ber_dump) to stdin when using
these patches to python-ldap. Can I influence the debugging level
anywhere?

Ciao, Michael.

Subtle changes in error handling with OpenLDAP 2 (was: compiling against OpenLDAP 2.0.11)

[Michael <mi...@st...>]

Michael Str=F6der wrote:
> =

> Konstantin Chuguev wrote:
> >
> > Yes, OpenLDAP-2.0.11 and especially recent changes in python-ldap CVS=

> > repository require a new version of patches. I've attached them.
> =

> I tried these patches. They seem to work

I noticed another subtle change in the OpenLDAP 2 behaviour: When
adding an attribute to an entry which is unknown to the server
(schema violation) there's just the base expection class
ldap.LDAPError returned containing message ("Undefined attribute
type..."). This makes it impossible to specifically catch e.g.
ldap.OBJECT_CLASS_VIOLATION.
(web2ldap makes extensive use of catching
ldap.OBJECT_CLASS_VIOLATION, ldap.NAMING_VIOLATION and friends to
enable the user to re-edit the input etc.)

I experienced similar behaviour with exceptions raised in case of
referrals received. The ldap.LDAPError base class was raised with
OpenLDAP 2.0.x lib instead of e.g. ldap.PARTIAL_RESULTS raised in
case of OpenLDAP 1.2.x lib. I thought that in this special case it
has to do with the migration from LDAPv2+ pseudo referrals to real
LDAPv3 knowledge references. Not sure about this issue though.
(web2ldap 0.9.x already contains code to handle both cases but I'd
like to get rid of that bloat in the future.)

It seems that the whole exception-based error catching needs to be
tested thoroughly because there are probably other subtle changes in
the OpenLDAP 2 libs as well.

Ciao, Michael.

Re: compiling against OpenLDAP 2.0.11

[Michael <mi...@st...>]

Steffen,

thanks a lot for your patches.

Steffen Ries wrote:
> =

> Michael Str=F6der <mi...@st...> writes:
> =

> > Anyone did notice the forwarded warning message by Stig about
> > possible memory leaks with OpenLDAP 2.0.x API?
> > Please glance over it:
> > http://www.geocrawler.com/archives/3/1568/2001/6/0/6036658/
> =

> I looked through it.
> [..]
> The attached patch should fix both issues.

I tried to merge your patches with the patches provided by
Konstantin but it seg faults with OpenLDAP 2.0.11 libs.

Since your diff is not against recent CVS version there might be a
problem with some small differences. E.g. there's a call "free(dn);"
before "return NULL;" in recent CVS version of message.c which
affects your first diff.

Since I have no idea about the C sources I'd appreciate if you could
take a look at the recent CVS version in which some memory leaks
were fixed since python-ldap-1.10alpha3.

Ciao, Michael.

Re: compiling against OpenLDAP 2.0.11

[Michael <mi...@st...>]

Konstantin Chuguev wrote:
> 
> Yes, OpenLDAP-2.0.11 and especially recent changes in python-ldap CVS
> repository require a new version of patches. I've attached them.

I tried these patches. They seem to work (LDAPv3 connects with
recent web2ldap 0.9.x :-). But still there's the problem that LDAP
URLs in referrals are not properly formatted. From some discussions
on openldap-software I suspect the OpenLDAP 2 libs.

Example:

When searching
ldap://ldap.surfnet.nl/c=BE?ou,mail,uid,telephoneNumber,labeledurl,cn,objectClass,displayName?one

the OpenLDAP 2 lib returns the referral URL
ldap://tor.dante.org.uk:1389?ou,mail,uid,telephoneNumber,labeledurl,cn,objectClass,displayName?one

This LDAP URL does not contain a slash behind the hostport part. My
LDAP URL parser (usually in nitpicking mode) expects a trailing
slash after hostport (which might be empty) if there are any
parameters after hostport.

Glancing at RFC2255 seems to confirm this assumption:

       ldapurl    = scheme "://" [hostport] ["/"
                    [dn ["?" [attributes] ["?" [scope]
                    ["?" [filter] ["?" extensions]]]]]]

I tried to raise this at the OpenLDAP 2 libs but it was rather
ignored since I could not provide a detailed example. I could only
provide a python-ldap example there but surely Kurt would have
pointed me back to bugs in python-ldap.

Anybody willing to write a short C source example confirming this
possible bug? Search something which returns a referral LDAP URL and
look at the LDAP URL returned (switch off automatic referral
chasing).

> The patches don't change the behaviour of python-ldap when compiled
> against OpenLDAPv1, but create an alternative code when used with
> OpenLDAPv2.

That's great!

> Here are the differences between python-ldap compiled
> [..]
>    * new type of data added to the Python dictionary returned as a
>      result of ldap_result:
>      Dictionary keys are DNs, values are entry objects. If the key is
>      empty, the value is the list of referrals (URL text strings).

I completely forgot what we've defined in November but web2ldap
seems to display the search continuations nicely. ;-)
E.g.
ldap://ldap.nameflow.net:1389/c%3DFI??base?%28objectclass%3D%2A%29
shows
Referral => 
    ldap://193.166.0.77:389/dmdName=FunetDir,%20c=FI??base

in search result table.

Sorry Konstantin, the on-line demo at
http://sites.inka.de:8002/web2ldap is still running with python-ldap
built against OpenLDAP 1.2.x. You have to install web2ldap locally
to do the testing. Make sure to look at the output of the ConnInfo
button to confirm that it says "LDAPv3 connection to:".

Ciao, Michael.

Re: compiling against OpenLDAP 2.0.11

[Steffen R. <ste...@sy...>]

Michael Str=F6der <mi...@st...> writes:

> Anyone did notice the forwarded warning message by Stig about
> possible memory leaks with OpenLDAP 2.0.x API?
> Please glance over it:
> http://www.geocrawler.com/archives/3/1568/2001/6/0/6036658/

I looked through it.

The first issue (ldap_first_attribute) has changed from OpenLDAP 1.2
to 2.0.x so that the memory is *never* freed by the library. In 1.1 it
was freed, when the last attribute was read. However, the current
python-ldap code did never free the memory. IOW, there is a possible
memory leak in error cases.

The second issue (ldap_get_dn) has *not* changed between OpenLDAP 1.2
and 2.0.x, i.e. the same kind of memory leak exists for both API
versions.

The attached patch should fix both issues. The last "ber_free()"
should be surrounded by some #ifdef OPENLDAP_2_0_X conditional...

I scanned through the documentation and did not notice any further
changes in memory handling, but then again, I did not really look too
hard.

*** python-ldap-1.10alpha3/Modules/message.c	Mon Aug 14 18:37:37 2000
--- python-ldap-openldap-2/Modules/message.c	Mon Jul  2 10:55:30 2001
***************
*** 78,84 ****
--- 78,86 ----
  	     if (valuelist =3D=3D NULL) {
  		Py_DECREF(attrdict);
  		Py_DECREF(result);
+                 ber_free(ber, 0);
  		ldap_msgfree( m );
+                 ldap_memfree(attr);
  		return NULL;
  	     }
=20=20
***************
*** 95,101 ****
--- 97,105 ----
  			Py_DECREF(result);
  			Py_DECREF(valuestr);
  			Py_DECREF(valuelist);
+                         ber_free(ber, 0);
  			ldap_msgfree( m );
+                         ldap_memfree(attr);
  			return NULL;
  		    }
  		    Py_DECREF(valuestr);
***************
*** 103,114 ****
--- 107,121 ----
  		ldap_value_free_len(bvals);
  	     }
  	     Py_DECREF( valuelist );
+              ldap_memfree(attr);
  	 }
=20=20
  	 entrytuple =3D Py_BuildValue("(sO)", dn, attrdict);
+          ldap_memfree(dn);
  	 Py_DECREF(attrdict);
  	 PyList_Append(result, entrytuple);
  	 Py_DECREF(entrytuple);
+          ber_free(ber, 0);
       }
       ldap_msgfree( m );
       return result;



/steffen
--=20
ste...@sy...	<> Gravity is a myth -- the Earth sucks!

Re: compiling against OpenLDAP 2.0.11

[Michael <mi...@st...>]

Konstantin Chuguev wrote:
> 
> David Leonard wrote:
> 
> > this is good - commit it

David, take care of your PhD-ing and please let me test Konstantin's
patches first. ;-)

> I doubt I can do that: I'm not a python-ldap developer... :-)

Being a python-ldap developer can happen to come true faster than
you think these days... ;-)

Anyone did notice the forwarded warning message by Stig about
possible memory leaks with OpenLDAP 2.0.x API?
Please glance over it:
http://www.geocrawler.com/archives/3/1568/2001/6/0/6036658/

> > will the documentation need updating?
> 
> I'll prepare another patch for python-ldap/Doc/libldap.tex.

Urrgs. I think I should at least publish a "python setup.py sdist"
of recent snapshot to make a 1.11 distribution before the commits.
Comments?

Ciao, Michael.