python-ldap-dev — discussion among developers

Re: Bug in LDAPObject.c, l_ldap_result()

[Derrick 'd. H. <dm...@dm...>]

On Sat, Jun 29, 2002 at 07:54:12PM +0200, Michael Str=F6der wrote:
| Derrick 'dman' Hudson wrote:
| >
| >I see now :
| >
| >  def search_st(...)
| >-    return self.result(msgid,all=3D1,timeout=3Dtimeout)
| >+    return self.result(msgid,all=3D1,timeout=3Dtimeout)[1]
| >     =20
| >
| >That's precisely what I kept running into
|=20
| Please test it!

Now I have (some more, at least).  I copied async.py from CVS and
backported the patches to LDAPObject.c and ldapobject.py to version
2.0.0pre04 and now web2ldap works great (and faster!).  Re-timing the
maillist.py script I originally posted yields this :

    real    0m0.206s
    user    0m0.070s
    sys     0m0.040s

As crude as time(1) may be, that is clearly better and isn't long
enough to measure with my watch or to show up in top :-).

| This whole stuff with result() is somewhat=20
| incomplete anyway since the result type

Looks like your sentence was cut off.


The CVS version of python-ldap doesn't work for me, though.  I had to
change some of the sasl stuff in LDAPObject.c for it to compile, and I
get a SEGV with web2ldap.  (I also changed a couple other minor
details that gcc complained about)  I'll work on that problem later.

=20
| >| I guess I will add a derived class for the non-blocking version of=20
| >| LDAPObject.result(). So everyone will be free to choose.
| >
| >Isn't that what 'timeout=3D0' is for?
|=20
| Yes. But then the application has to handle that.

Isn't that the purpose of not blocking in the first place?

| The non-blocking version of LDAPObject.result() is the one which
| needed the time.sleep() hack.

The version that was in release 2.0.0pre04 was blocking if all was
true.  It just did the blocking in python/python-ldap instead of in
C/libldap2.

| I will add it to ldapobject.py in a separate class.

I have no complaints with that.
=20
| >I also think the C function needs
| >to be modified to treat Py_None as NULL to allow python code to
| >specify a timeout that is indefinitely long.
|=20
| IMHO that's done with timeout=3D-1. Looking at l_ldap_result() in=20
| LDAPObject.c it seems to be implemented correctly

Ahh, yeah, that is one suitable convention.  Doesn't
    timeout=3DNone
sound amusingly accurate though?  (in english at least)

-D

--=20

> SELECT * FROM users WHERE clue > 0
0 rows returned
                    (http://www.thinkgeek.com/images/products/zoom/no-clue.=
jpg)
=20
http://dman.ddts.net/~dman/

Re: Bug in LDAPObject.c, l_ldap_result()

[<mi...@st...>]

Derrick 'dman' Hudson wrote:
 >
> I see now :
> 
>   def search_st(...)
> -    return self.result(msgid,all=1,timeout=timeout)
> +    return self.result(msgid,all=1,timeout=timeout)[1]
>       
> 
> That's precisely what I kept running into

Please test it! This whole stuff with result() is somewhat 
incomplete anyway since the result type

> | I guess I will add a derived class for the non-blocking version of 
> | LDAPObject.result(). So everyone will be free to choose.
> 
> Isn't that what 'timeout=0' is for?

Yes. But then the application has to handle that. The non-blocking 
version of LDAPObject.result() is the one which needed the 
time.sleep() hack. I will add it to ldapobject.py in a separate class.

> I also think the C function needs
> to be modified to treat Py_None as NULL to allow python code to
> specify a timeout that is indefinitely long.

IMHO that's done with timeout=-1. Looking at l_ldap_result() in 
LDAPObject.c it seems to be implemented correctly (although 
reading C makes my eyes hurt):

     if (timeout >= 0) {
         tvp = &tv;
         set_timeval_from_double( tvp, timeout );
     } else {
         tvp = NULL;
     }

Ciao, Michael.

Re: Bug in LDAPObject.c, l_ldap_result()

[Derrick 'd. H. <dm...@dm...>]

On Sat, Jun 29, 2002 at 04:32:52PM +0200, Michael Str=F6der wrote:
| Derrick 'dman' Hudson wrote:

| >-       pmsg =3D Py_None;
| >+       pmsg =3D LDAPmessage_to_python( self->ldap, msg );

| >It really does help to not ignore the data that's returned :-).
|=20
| Oh, yes.
|=20
| >I haven't thoroughly tested it
|=20
| You already noticed the issues with web2ldap.

Yep.  I was beating myself over the head last night trying to figure
out the right type to return.

| One has to fix two other parts of python-ldap for this patch to work
| with web2ldap.

Ahh.
=20
| I've checked in Modules/LDAPObject.c Lib/ldap/ldapobject.py=20
| Lib/ldap/async.py. Now the C module's result() is directly wrapped=20
| and LDAPObject.c contains the fix above.

I see now :

  def search_st(...)
-    return self.result(msgid,all=3D1,timeout=3Dtimeout)
+    return self.result(msgid,all=3D1,timeout=3Dtimeout)[1]
     =20

That's precisely what I kept running into -- the line
    ldap_result[0][1]
_sometimes_ wasn't getting the right element.  I thought the result
function wasn't returning the right value (it needed another layer of
list nesting), but when I would add the extra list some other parts
wouldn't work!  That certainly explains it :-).

| I guess I will add a derived class for the non-blocking version of=20
| LDAPObject.result(). So everyone will be free to choose.

Isn't that what 'timeout=3D0' is for?  I also think the C function needs
to be modified to treat Py_None as NULL to allow python code to
specify a timeout that is indefinitely long.

-D

--=20

"GUIs normally make it simple to accomplish simple actions and
impossible to accomplish complex actions."
    --Doug Gwyn  (22/Jun/91 in comp.unix.wizards)
=20
http://dman.ddts.net/~dman/

Re: Bug in LDAPObject.c, l_ldap_result()

[<mi...@st...>]

Derrick 'dman' Hudson wrote:
 >
> --- LDAPObject.c.orig   Fri Jun 28 15:59:11 2002
> +++ LDAPObject.c        Fri Jun 28 15:59:37 2002
> @@ -818,7 +818,7 @@
>                 e = "ldap_parse_result";
>             return LDAPerror( self->ldap, e );
>         }
> -       pmsg = Py_None;
> +       pmsg = LDAPmessage_to_python( self->ldap, msg );
>      }
>  
>      result_str = LDAPconstant( res_type );
> 
> 
> It really does help to not ignore the data that's returned :-).

Oh, yes.

> I haven't thoroughly tested it

You already noticed the issues with web2ldap. One has to fix two 
other parts of python-ldap for this patch to work with web2ldap.

I've checked in Modules/LDAPObject.c Lib/ldap/ldapobject.py 
Lib/ldap/async.py. Now the C module's result() is directly wrapped 
and LDAPObject.c contains the fix above.

I guess I will add a derived class for the non-blocking version of 
LDAPObject.result(). So everyone will be free to choose.

Ciao, Michael.

Re: ldap_url_search ?

[<mi...@st...>]

Michael Str=F6der wrote:
>  > Hmm, it seems that ldap_url_search is no longer available in
>  > OpenLDAP
>  > 2.1.
>  > [..]
> Anyone having any objections against removing methods url_search*()?

I removed method url_search from LDAPObject.c.

We can reimplement that in the Python wrapper class=20
ldap.ldapobject.LDAPObject if there's request.

Ciao, Michael.

Connection pooling (was: defect)

[<mi...@st...>]

Derrick 'dman' Hudson wrote:
> On Fri, Jun 28, 2002 at 09:49:14AM -0700, Joe Little wrote:
> 
> | do not permit concurrent access to a single LDAP session (ie. use
> | one per thread or a mutex around it).
> 
> (FWIW)
> This sounds perfectly acceptable to me.
> Making a separate connection for each thread is likely to reduce
> contention anyways (improves performance).

This remembers me of an old idea I had to implement persistent 
LDAP connection pooling with Queue.Queue found in Python's 
standard lib.

E.g. if a HTTP server serves HTTP requests with 100 active threads 
you're not really keen to open a LDAP connection per thread because
1. your system ressources might get exhausted (e.g. file handles)
2. opening a connection is very slow (e.g. when initializing 
attributes read from RootDSE, SSL negotiation, SASL binds, etc.).

Ciao, Michael.

Building python-ldap with libldap_r (was: defect)

[<mi...@st...>]

Joe Little wrote:
>=20
> On Friday, June 28, 2002, at 12:00 PM, Michael Str=F6der wrote:
>=20
>> Note that most people still build against OpenLDAP 2.0.x. I don't want=
=20
>> to mandate OpenLDAP 2.1.x at the moment because there will surely be=20
>> more complaints about this than about bad performance.
>=20
> Agreed. I talked with Luke about 2.1's improvements in this area, and h=
e=20
> replied that 2.0 already was thread safe. I'll be diving into 2.1, but =

> 2.0 should be sufficient for coding thread safe python-ldap once it use=
s=20
> libldap_r.
>=20
>> Caveats:
>> 1. libldap_r has not been tested with python-ldap yet.
>> 2. libldap_r is not fully implemented in OpenLDAP 2.0.x. AFAIK in 2.0 =

>> it's just a hack to make slurpd work.
>=20
> What I understood from Luke is that it has been completed in 2.x, with =

> bugs worked out along the way up to 2.0.25.

I tried to build python-ldap against libldap_r from OpenLDAP's=20
REL_ENG_2 branch by simply modifying setup.cfg:

libs =3D ldap_r lber sasl

The build went fine. ldd shows output like expected:

$ ldd build/lib.linux-i686-2.2/_ldap.so
         libldap_r.so.2 =3D> /usr/lib/libldap_r.so.2 (0x4001f000)
         liblber.so.2 =3D> /usr/lib/liblber.so.2 (0x4004b000)
         libsasl.so.7 =3D> /usr/lib/libsasl.so.7 (0x40055000)
[..]

But import does not work.

$ python -c "import ldap"
Traceback (most recent call last):
   File "<string>", line 1, in ?
   File "/usr/lib/python2.2/site-packages/ldap/__init__.py", line=20
5, in ?
     from _ldap import *
ImportError: /usr/lib/python2.2/site-packages/_ldap.so: undefined=20
symbol: ldap_first_reference

Anyone willing to dig into this?

Ciao, Michael.

Module ldapurl (was: ldap_url_search ?)

[<mi...@st...>]

Derrick 'dman' Hudson wrote:
> On Fri, Jun 28, 2002 at 09:36:26PM +0200, Michael Str=F6der wrote:
> | Derrick 'dman' Hudson wrote:
> | >On Fri, Jun 28, 2002 at 04:44:54PM +0200, Michael Str=F6der wrote:
> | >| Michael Str=F6der wrote:
> | >|=20
> | >| Anyone having any objections against removing methods url_search*(=
)?
> | >|=20
> | >| Anyone really using it?
> | >
> | >The only objection I have is that it really helped me to learn how t=
o
> | >work with an LDAP URL.
> |=20
> | How about practicing with the module ldapurl?
>=20
> How?  The documentation is pretty sparse.
> [..]
> If the ldapurl module does everything that the current url_search
> methods do, then there is no compelling reason to keep the latter
> around.

It parses and unparses LDAP URLs and is designed to be completely=20
independent from module ldap.

Some sample code for parsing a LDAP URL:

 >>> import ldapurl
 >>> ldap_url =3D=20
ldapurl.LDAPUrl('ldap://localhost:1389/dc=3Dstroeder,dc=3Dcom?cn???bindna=
me=3Dcn=3DMichael%2cdc=3Dstroeder%2cdc=3Dcom,X-BINDPW=3Dsecret')
 >>> # Using the parsed LDAP URL by reading the class attributes
 >>> ldap_url.dn
u'dc=3Dstroeder,dc=3Dcom'
 >>> ldap_url.hostport
'localhost:1389'
 >>> ldap_url.attrs
['cn']
 >>> ldap_url.filterstr
u'(objectclass=3D*)'
 >>> ldap_url.who
'cn=3DMichael,dc=3Dstroeder,dc=3Dcom'
 >>> ldap_url.cred
'secret'
 >>> ldap_url.scope
0

Some sample code for the other way round:

 >>> ldap_url =3D=20
ldapurl.LDAPUrl(hostport=3D'localhost:1389',dn=3Du'dc=3Dstroeder,dc=3Dcom=
',attrs=3D['cn','mail'],who=3D'cn=3DMichael,dc=3Dstroeder,dc=3Dcom',cred=3D=
'secret')
 >>> ldap_url.unparse()
u'ldap://localhost:1389/dc=3Dstroeder,dc=3Dcom?cn,mail?base?(objectclass=3D=
*)?bindname=3Dcn=3DMichael%2Cdc=3Dstroeder%2Cdc=3Dcom,X-BINDPW=3Dsecret'

That's pretty much all it does. Note that you don't have to=20
specify all the class attributes. There are reasonable defaults=20
assumed. And who and cred maps to None if not specified. See the=20
tests at the bottom of ldapurl.py for more examples.

Advice: For security reasons you shouldn't specify passwords in=20
LDAP URLs unless you really know what you're doing.

Ciao, Michael.

Re: defect

[Derrick 'd. H. <dm...@dm...>]

On Fri, Jun 28, 2002 at 09:49:14AM -0700, Joe Little wrote:

| I haven't seen those complaints. The OpenLDAP client library is guaranteed
| to be thread safe as long as you

| do not permit concurrent access to a single LDAP session (ie. use
| one per thread or a mutex around it).

(FWIW)
This sounds perfectly acceptable to me.
Making a separate connection for each thread is likely to reduce
contention anyways (improves performance).


I don't see my patch (other thread) on the list, but don't use it --
it breaks web2ldap :-(.  I need to work on it some more.

-D

--=20

The Consultant's Curse:
    When the customer has beaten upon you long enough, give him
what he asks for, instead of what he needs.  This is very strong
medicine, and is normally only required once.
=20
http://dman.ddts.net/~dman/

Re: ldap_url_search ?

[Derrick 'd. H. <dm...@dm...>]

On Fri, Jun 28, 2002 at 09:36:26PM +0200, Michael Str=F6der wrote:
| Derrick 'dman' Hudson wrote:
| >On Fri, Jun 28, 2002 at 04:44:54PM +0200, Michael Str=F6der wrote:
| >| Michael Str=F6der wrote:
| >|=20
| >| Anyone having any objections against removing methods url_search*()?
| >|=20
| >| Anyone really using it?
| >
| >The only objection I have is that it really helped me to learn how to
| >work with an LDAP URL.
|=20
| How about practicing with the module ldapurl?

How?  The documentation is pretty sparse.  (I guess I could read the
source, though, and then I would say "Ok")

http://python-ldap.sourceforge.net/doc/python-ldap/module-ldapurl.html
http://python-ldap.sourceforge.net/pydoc/ldapurl.html


If the ldapurl module does everything that the current url_search
methods do, then there is no compelling reason to keep the latter
around.

-D

--=20

"...In the UNIX world, people tend to interpret `non-technical user' as
meaning someone who's only ever written one device driver."
    --Daniel Pead
=20
http://dman.ddts.net/~dman/

Re: Bug in LDAPObject.c, l_ldap_result()

[Derrick 'd. H. <dm...@dm...>]

On Thu, Jun 27, 2002 at 07:41:16PM +0200, Michael Str=F6der wrote:

| While digging into the performance issues with
| ldap.ldapobject.LDAPObject.result() I noticed that the original C
| module's result() method does not return results if called with
| all=3D1. See the test script attached.

| Hope to find a contributor with C knowledge who has some spare
| cycles to track that down...


--- LDAPObject.c.orig   Fri Jun 28 15:59:11 2002
+++ LDAPObject.c        Fri Jun 28 15:59:37 2002
@@ -818,7 +818,7 @@
                e =3D "ldap_parse_result";
            return LDAPerror( self->ldap, e );
        }
-       pmsg =3D Py_None;
+       pmsg =3D LDAPmessage_to_python( self->ldap, msg );
     }
=20
     result_str =3D LDAPconstant( res_type );


It really does help to not ignore the data that's returned :-).

I haven't thoroughly tested it (actually, I haven't tested any more
than the script you supplied tests it, except that I explicitly added
the 'all' and 'timeout' arguments), but it looks correct and seems to
work.

(the patch is against 2.0.0pre04, but is easy enough to do manually if
CVS has different line numbers)

-D

--=20

"Wipe Info uses hexadecimal values to wipe files. This provides more=20
security than wiping with decimal values." -- Norton SystemWorks 2002 Manual
=20
http://dman.ddts.net/~dman/

Re: ldap_url_search ?

[<mi...@st...>]

Derrick 'dman' Hudson wrote:
> On Fri, Jun 28, 2002 at 04:44:54PM +0200, Michael Str=F6der wrote:
> | Michael Str=F6der wrote:
> |=20
> | Anyone having any objections against removing methods url_search*()?
> |=20
> | Anyone really using it?
>=20
> The only objection I have is that it really helped me to learn how to
> work with an LDAP URL.

How about practicing with the module ldapurl? It has more features=20
anyway. And it can construct LDAP URLs from search parameters with=20
method unparse().

> Though if OpenLDAP 2.1 doesn't have the function, then that's a real
> strong argument for removing those methods (at the very least when
> compiled against OpenLDAP 2.1).

Well, another strong argument is that there's no one actively=20
maintaining the C module part under Modules/. Therefore I'd be=20
glad to keep it as short as possible.

Ciao, Michael.

Re: ldap_url_search ?

[Derrick 'd. H. <dm...@dm...>]

On Fri, Jun 28, 2002 at 04:44:54PM +0200, Michael Str=F6der wrote:
| Michael Str=F6der wrote:
| > Franco Spinelli wrote:
| >
| >> I have installed openldap 2.1.2 fron openldap site
| >  > [..]
| >>> symbol ldap_url_search: referenced symbol not found
| >
| > Hmm, it seems that ldap_url_search is no longer available in
| > OpenLDAP
| > 2.1. So far I've never tested python-ldap with OpenLDAP 2.1.
| >
| > [..]
| > * Remove any reference to ldap_url_search in
| >   Modules/LDAPObject.c. web2ldap does not use this function.
|=20
| Anyone having any objections against removing methods url_search*()?
|=20
| Anyone really using it?

The only objection I have is that it really helped me to learn how to
work with an LDAP URL.  I needed to use URLs to configure exim to pull
data from LDAP, and I was having trouble using 'ldapsearch' -- it
didn't like the URL no matter what I gave it.  For me it was a
valuable tool.

Though if OpenLDAP 2.1 doesn't have the function, then that's a real
strong argument for removing those methods (at the very least when
compiled against OpenLDAP 2.1).

| If yes, we could easily reimplement it later with help of module
| ldapurl.  The only issue would be that ldapurl needs Python 2.x (not
| compatible with Python 1.5.x).

Personally I don't consider compatibility with python 1.5.2 a
significant feature.  2.0 was released a long time ago, by internet
standards.

-D

--=20

Microsoft DNS service terminates abnormally when it receives a response
to a dns query that was never made.
Fix information: run your DNS service on a different platform.
                                                            -- bugtraq
=20
http://dman.ddts.net/~dman/

Re: defect

[Joe L. <jl...@op...>]

On Friday, June 28, 2002, at 12:00 PM, Michael Str=F6der wrote:

> Joe Little wrote:
>> two snippets from Luke when speaking about 2.1
>
> Note that most people still build against OpenLDAP 2.0.x. I don't want=20=

> to mandate OpenLDAP 2.1.x at the moment because there will surely be=20=

> more complaints about this than about bad performance.

Agreed. I talked with Luke about 2.1's improvements in this area, and=20
he replied that 2.0 already was thread safe. I'll be diving into 2.1,=20
but 2.0 should be sufficient for coding thread safe python-ldap once it=20=

uses libldap_r.

>
> I don't like #ifdef's either. Lesson learned so far: I'm not a C=20
> programmer and the guys adding the #ifdef's won't be available for=20
> maintaining the C code later.
>
>> When I mentioned complaints here about thread safety, his response:
>> I haven't seen those complaints. The OpenLDAP client library is=20
>> guaranteed
>> to be thread safe as long as you (a) link against libldap_r and (b) =
do
>> not permit concurrent access to a single LDAP session (ie. use one =
per
>> thread or a mutex around it).
>
> Ok, not really new information. That was also my understanding from=20
> Kurt's postings quite a while ago.
>
> Caveats:
> 1. libldap_r has not been tested with python-ldap yet.
> 2. libldap_r is not fully implemented in OpenLDAP 2.0.x. AFAIK in 2.0=20=

> it's just a hack to make slurpd work.

What I understood from Luke is that it has been completed in 2.x, with=20=

bugs worked out along the way up to 2.0.25.

> 3. We can't get completely rid of locking. We can do finer grained=20
> locking per LDAPObject instance. AFAIK global functions have still  to=20=

> be protected by module-wide locking.
>
>> So, libldap_r is one requirement...
>
> I'd be glad if someone dives into the gory details.
>
>> but the other seems to be already set by our use of internal thread=20=

>> locking to that single session.
>
> At the moment there's a single module-wide lock serializing all calls=20=

> into libldap since this is required with OpenLDAP 2.0.x. This can be=20=

> easily changed off course.
>

Well, I hope this small discussion shows a way forward.


> Ciao, Michael.
>
>
>
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Caffeinated soap. No kidding.
> http://thinkgeek.com/sf
> _______________________________________________
> Python-LDAP-dev mailing list
> Pyt...@li...
> https://lists.sourceforge.net/lists/listinfo/python-ldap-dev

Re: defect

[<mi...@st...>]

Joe Little wrote:
> two snippets from Luke when speaking about 2.1

Note that most people still build against OpenLDAP 2.0.x. I don't 
want to mandate OpenLDAP 2.1.x at the moment because there will 
surely be more complaints about this than about bad performance.

I don't like #ifdef's either. Lesson learned so far: I'm not a C 
programmer and the guys adding the #ifdef's won't be available for 
maintaining the C code later.

> When I mentioned complaints here about thread safety, his response:
> 
> I haven't seen those complaints. The OpenLDAP client library is guaranteed
> to be thread safe as long as you (a) link against libldap_r and (b) do
> not permit concurrent access to a single LDAP session (ie. use one per
> thread or a mutex around it).

Ok, not really new information. That was also my understanding 
from Kurt's postings quite a while ago.

Caveats:
1. libldap_r has not been tested with python-ldap yet.
2. libldap_r is not fully implemented in OpenLDAP 2.0.x. AFAIK in 
2.0 it's just a hack to make slurpd work.
3. We can't get completely rid of locking. We can do finer grained 
locking per LDAPObject instance. AFAIK global functions have still 
  to be protected by module-wide locking.

> So, libldap_r is one requirement...

I'd be glad if someone dives into the gory details.

> but the other seems to be already 
> set by our use of internal thread locking to that single session.

At the moment there's a single module-wide lock serializing all 
calls into libldap since this is required with OpenLDAP 2.0.x. 
This can be easily changed off course.

Ciao, Michael.

RE: defect

[Leif H. <lei...@pr...>]

> Note that there is a first call to self._l.result() outside both 
> while-loops and the time.sleep() is in the inner loop (while 
> ldap_result[0] is None:). IMHO it should be equivalent.

Yeah, my bad, you are absolutely correct. I still find it kinda kludgy to
have to put a sleep() statement in the loop like this, but it works for
us... ;-)

ciao,

-- leif

Re: defect

[Joe L. <jl...@op...>]

two snippets from Luke when speaking about 2.1 (and whether it was =
finally thread safe:

lukeh:
On what is thread safe? The client library? The server? I thought it was
always pretty much thread safe. I think they've shaken down some issues
with the transactional (bdb) backend.

When I mentioned complaints here about thread safety, his response:

I haven't seen those complaints. The OpenLDAP client library is =
guaranteed
to be thread safe as long as you (a) link against libldap_r and (b) do
not permit concurrent access to a single LDAP session (ie. use one per
thread or a mutex around it).

So, libldap_r is one requirement... but the other seems to be already =
set by our use of internal thread locking to that single session.

On Friday, June 28, 2002, at 09:43 AM, Michael Str=F6der wrote:

> Joe Little wrote:
>> We beat this one up a bit before... But I asked Luke Howard (who is =
generally in the know on this stuff) again about OpenLDAP thread-safety, =
and he said that it is thread safe.
>
> Since when? Which version? Which lib? libldap_r is supposed to be =
reentrant but that's not what python-ldap is built against. Furthermore =
it was not possible to get a positive statement from Kurt Zeilenga last =
time I asked.
>
> Ciao, Michael.

Re: defect

[<mi...@st...>]

Joe Little wrote:
> 
> We beat this one up a bit before... But I asked Luke Howard (who is 
> generally in the know on this stuff) again about OpenLDAP thread-safety, 
> and he said that it is thread safe.

Since when? Which version? Which lib? libldap_r is supposed to be 
reentrant but that's not what python-ldap is built against. 
Furthermore it was not possible to get a positive statement from 
Kurt Zeilenga last time I asked.

Ciao, Michael.

Re: defect

[Joe L. <jl...@op...>]

Michael.

We beat this one up a bit before... But I asked Luke Howard (who is =
generally in the know on this stuff) again about OpenLDAP thread-safety, =
and he said that it is thread safe. Perhaps Leif can chime in here on =
where it stands.



On Thursday, June 27, 2002, at 10:47 AM, Michael Str=F6der wrote:

> Jens Vagelpohl wrote:
>> On Wednesday, June 26, 2002, at 04:58 , Michael Str=F6der wrote:
>>> It's possible to make it somewhat simpler since we have a first =
result() call before the while loops.
>>>
>>>     while all:
>> this simplified version seems to slow down my setup.
>
> Yes. I wrote that when posting the code snippet.
>
>> here's a result set with leif's version::
>
> Leif's version sets non-zero timeout. Therefore it's faster since =
OpenLDAP's ldap_result() can use select() to determine just-in-time when =
data is ready to be read. But it blocks which is a bad thing because of =
ldap._ldap_lock serializing *all* calls...
>
> Just adding the time.sleep() hands over the CPU to the OS. Off course =
the while-loop is not just-in-time there if received data is ready.
>
> Again, the problem is that the OpenLDAP libs are not thread-safe...
>
> Ciao, Michael.
>
>
>
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Caffeinated soap. No kidding.
> http://thinkgeek.com/sf
> _______________________________________________
> Python-LDAP-dev mailing list
> Pyt...@li...
> https://lists.sourceforge.net/lists/listinfo/python-ldap-dev

Re: ldap_url_search ?

[Jens V. <je...@zo...>]

> Anyone having any objections against removing methods url_search*()?
>
> Anyone really using it? If yes, we could easily reimplement it
> later with help of module ldapurl. The only issue would be that
> ldapurl needs Python 2.x (not compatible with Python 1.5.x).
>

no objections. never used it.

jens

ldap_url_search ?

[<mi...@st...>]

Michael Str=F6der wrote:
 > Franco Spinelli wrote:
 >
 >> I have installed openldap 2.1.2 fron openldap site
 >  > [..]
 >>> symbol ldap_url_search: referenced symbol not found
 >>
 >
 > Hmm, it seems that ldap_url_search is no longer available in
 > OpenLDAP
 > 2.1. So far I've never tested python-ldap with OpenLDAP 2.1.
 >
 > [..]
 > * Remove any reference to ldap_url_search in
 >   Modules/LDAPObject.c. web2ldap does not use this function.

Anyone having any objections against removing methods url_search*()?

Anyone really using it? If yes, we could easily reimplement it
later with help of module ldapurl. The only issue would be that
ldapurl needs Python 2.x (not compatible with Python 1.5.x).

Ciao, Michael.

Re: defect

[<mi...@st...>]

Leif Hedstrom wrote:
> I think you want to do the sleep only if ldap_result[0] is None, otherwise
> you'll cause a sleep every loop, regardless if data is available or not.

???

       while ldap_result[0] is None:
         if (timeout>=0) and (time.time()-start_time>timeout):
           self._ldap_call(self._l.abandon,msgid)
           raise _ldap.TIMELIMIT_EXCEEDED(
             "LDAP time limit (%d secs) exceeded." % (timeout)
           )
         time.sleep(0.0001)
         ldap_result = self._ldap_call(self._l.result,msgid,0,0)

> That's why I originally put it after the call to result() inside the polling
> loop.

Note that there is a first call to self._l.result() outside both 
while-loops and the time.sleep() is in the inner loop (while 
ldap_result[0] is None:). IMHO it should be equivalent.

Ciao, Michael.

Bug in LDAPObject.c, l_ldap_result()

[<mi...@st...>]

HI!

While digging into the performance issues with
ldap.ldapobject.LDAPObject.result() I noticed that the original C
module's result() method does not return results if called with
all=1. See the test script attached.

Here's the output:

*** Using LDAPObject.result() ***
[('', {'objectClass': ['top', 'OpenLDAProotDSE']})]
*** Using C wrapper method directly ***
('RES_SEARCH_RESULT', None)

The RES_SEARCH_RESULT is ok but not None as second tuple item.

I guess nobody noticed that since everybody is solely using the
synchronous methods *_s(). I noticed that quite a while ago but
forgot this unsolved issue.

Since I'm not a C programmer and David is not available anymore
for python-ldap I'd like to ask if someone is willing to dig into
LDAPObject.c's function l_ldap_result(). I guess the
if-else-statement in question is (keep in mind the search 
references!):

      if (res_type == LDAP_RES_SEARCH_ENTRY
	    || res_type == LDAP_RES_SEARCH_REFERENCE
	)
	pmsg = LDAPmessage_to_python( self->ldap, msg );
      else {
	int result;
	char **refs = NULL;
	LDAP_BEGIN_ALLOW_THREADS( self );
	ldap_parse_result( self->ldap, msg, &result, NULL, NULL, &refs,
NULL, 0 );
	LDAP_END_ALLOW_THREADS( self );

	if (result != LDAP_SUCCESS) {		/* result error */
	    char *e, err[1024];
	    if (result == LDAP_REFERRAL && refs && refs[0]) {
		snprintf(err, sizeof(err), "Referral:\n%s", refs[0]);
		e = err;
	    } else
		e = "ldap_parse_result";
	    return LDAPerror( self->ldap, e );
	}
	pmsg = Py_None;
      }

Hope to find a contributor with C knowledge who has some spare
cycles to track that down...

Ciao, Michael.

Re: defect

[<mi...@st...>]

Jens Vagelpohl wrote:
>=20
> On Wednesday, June 26, 2002, at 04:58 , Michael Str=F6der wrote:
>=20
>> It's possible to make it somewhat simpler since we have a first=20
>> result() call before the while loops.
>>
>>     while all:
>=20
> this simplified version seems to slow down my setup.

Yes. I wrote that when posting the code snippet.

> here's a result set with leif's version::

Leif's version sets non-zero timeout. Therefore it's faster since=20
OpenLDAP's ldap_result() can use select() to determine=20
just-in-time when data is ready to be read. But it blocks which is=20
a bad thing because of ldap._ldap_lock serializing *all* calls...

Just adding the time.sleep() hands over the CPU to the OS. Off=20
course the while-loop is not just-in-time there if received data=20
is ready.

Again, the problem is that the OpenLDAP libs are not thread-safe...

Ciao, Michael.

RE: defect

[Leif H. <lei...@pr...>]

I think you want to do the sleep only if ldap_result[0] is None, otherwise
you'll cause a sleep every loop, regardless if data is available or not.
That's why I originally put it after the call to result() inside the polling
loop.

-- Leif