python-ldap-dev

Leaving a connection open

[Ron T. <Ro...@US...>]

Hi,

I have a few questions about leaving a bound connection open for sharing
(python 2.4.4, python-ldap 2.2.1 - openldap 2.3-2.3.30 on Ubuntu 7.04).

I'm using Async messages is there any benefit to using ReconnectLdapObject?
I noticed that the comments
http://vmspython.dyndns.org/pyhtmldoc/ldap.ldapobject.html said that the
class was intended for synchronous calls.

Is it ok to leave the connection open for long periods like a month?  Is it
realistic to believe that the connection would remain stable and be useable
for production if left open?

Is there a way to tell if the connection died so that I can reconnect a
shared connection if the connection dropped off?  I tried unbind and
whoami_s but got a very nasty memory error after a very long delay.

Thanks for your help!
Ron Teitelbaum

Re: Leaving a connection open

[Michael S. <mi...@st...>]

Ron Teitelbaum wrote:
> I have a few questions about leaving a bound connection open for sharing
> (python 2.4.4, python-ldap 2.2.1 - openldap 2.3-2.3.30 on Ubuntu 7.04).
> 
> I'm using Async messages is there any benefit to using ReconnectLdapObject?

No. If you're solely using the async methods you have to implement your own
try-except block catching ldap.SERVER_DOWN and re-initiate whatever LDAP
operation(s) seems appropriate in your application.

Please elaborate on why you're using the async methods. There are only rare
cases where this really makes sense (e.g. bulk data processing with
ldap.async or resiter, high-performance proxying with many outstanding
search requests). If you have a threaded application you might want to think
about using several pooled connections.

> I noticed that the comments
> http://vmspython.dyndns.org/pyhtmldoc/ldap.ldapobject.html said that the
> class was intended for synchronous calls.

Yupp. How else should it catch the ldap.SERVER_DOWN exception and do the
re-connect without the application noticing it?

> Is it ok to leave the connection open for long periods like a month?  Is it
> realistic to believe that the connection would remain stable and be useable
> for production if left open?

This also depends on your server's configuration. There are server
configuration directives to shorten the life-time of LDAP connections. I'd
recommend to always implement an appropriate re-connect functionality within
your application.

> Is there a way to tell if the connection died so that I can reconnect a
> shared connection if the connection dropped off?

I'd recommend to send the operation and re-connect and re-send the operation
if needed. Testing the connecting with a LDAP request will also result in a
ldap.SERVER_DOWN exception to be raised and it's an extra LDAP request sent
=> extra roundtrip time.

> I tried unbind and whoami_s but got a very nasty memory error after a
> very long delay.

Note that this shouldn't happen in python-ldap 2.3.1+ built from source
against OpenLDAP 2.3 libs straigt built with OpenSSL (not gnu-tls). But
please report memory errors providing more details. We're tracking down some
issues in recent CVS but not sure if you're hitting these bugs.

Please also note that always unbind_s() should be called. The unbind call is
synchronous by nature and closes the connection. Calling whoami_s() only
make sense if the LDAP server supports this particular extended operation.
Not many LDAP server do this though.

Ciao, Michael.

RE: Leaving a connection open

[Ron T. <Ro...@US...>]

> From: Michael Ströder
> 
> Ron Teitelbaum wrote:
> > I have a few questions about leaving a bound connection open for sharing
> > (python 2.4.4, python-ldap 2.2.1 - openldap 2.3-2.3.30 on Ubuntu 7.04).
> >
> > I'm using Async messages is there any benefit to using
> ReconnectLdapObject?
> 
> No. If you're solely using the async methods you have to implement your
> own
> try-except block catching ldap.SERVER_DOWN and re-initiate whatever LDAP
> operation(s) seems appropriate in your application.

Ok I'll switch it back out.  Thanks.
> 
> Please elaborate on why you're using the async methods. There are only
> rare
> cases where this really makes sense (e.g. bulk data processing with
> ldap.async or resiter, high-performance proxying with many outstanding
> search requests). If you have a threaded application you might want to
> think
> about using several pooled connections.

I'm calling out from smalltalk to python and our vm is hung whenever we are
waiting on python.  The async methods were just what we needed to allow
processing.  We are handling a large number of connections and multiple
threads and we are doing some pretty intensive processing on the app so
having the vm die for a call out is not an option. 

> 
> > I noticed that the comments
> > http://vmspython.dyndns.org/pyhtmldoc/ldap.ldapobject.html said that the
> > class was intended for synchronous calls.
> 
> Yupp. How else should it catch the ldap.SERVER_DOWN exception and do the
> re-connect without the application noticing it?

Yeah, I'm starting to understand that.  I'm getting
can't-contact-ldap-server errors that I thought this would help with.  Note,
I believe this is different from the server down error you are mentioning.  

How can I create the server_down error for testing?  Would shutting off
slapd cause this error (I assume), or would that cause other problems in
python-ldap, is there an easier way?

> 
> > Is it ok to leave the connection open for long periods like a month?  Is
> it
> > realistic to believe that the connection would remain stable and be
> useable
> > for production if left open?
> 
> This also depends on your server's configuration. There are server
> configuration directives to shorten the life-time of LDAP connections. I'd
> recommend to always implement an appropriate re-connect functionality
> within
> your application.
> 
> > Is there a way to tell if the connection died so that I can reconnect a
> > shared connection if the connection dropped off?
> 
> I'd recommend to send the operation and re-connect and re-send the
> operation
> if needed. Testing the connecting with a LDAP request will also result in
> a
> ldap.SERVER_DOWN exception to be raised and it's an extra LDAP request
> sent
> => extra roundtrip time.
> 
> > I tried unbind and whoami_s but got a very nasty memory error after a
> > very long delay.
> 
> Note that this shouldn't happen in python-ldap 2.3.1+ built from source
> against OpenLDAP 2.3 libs straigt built with OpenSSL (not gnu-tls). But
> please report memory errors providing more details. We're tracking down
> some
> issues in recent CVS but not sure if you're hitting these bugs.

We are assessing our production environment.  For now we are staying with
Python2.4.4, is python-ldap 2.3.1 stable with Python-2.4.4?

> 
> Please also note that always unbind_s() should be called. 

I thought unbind and unbind_s called the same method internally.  Do I need
to change my calls to unbind_s?  Is that for clarity or is there an
implementation difference?

> The unbind call
> is
> synchronous by nature and closes the connection. Calling whoami_s() only
> make sense if the LDAP server supports this particular extended operation.
> Not many LDAP server do this though.

Thank you very much for your help,

Ron

Re: Leaving a connection open

[Michael S. <mi...@st...>]

Ron Teitelbaum wrote:
> 
> I'm getting
> can't-contact-ldap-server errors that I thought this would help with.  Note,
> I believe this is different from the server down error you are mentioning.  

Believe me it's not different. ldap.SERVER_DOWN is the exact exception class 
which you have to catch with except ldap.SERVER_DOWN. "Can't contact LDAP 
server" is the descriptive text (diagnostic message) for that. Note that 
this very same exception is raised if anything goes wrong with SSL/TLS and 
cert checking but with another descriptive text coming from the underlying 
SSL lib.

 > How can I create the server_down error for testing?

Example for a connect to a non-existing server:

 >>> l=ldap.initialize('ldap://localhost:1234')
 >>> l.simple_bind_s('cn=root','blurb')
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
   File "/usr/lib/python2.6/site-packages/ldap/ldapobject.py", line 201, in 
simple_bind_s
     msgid = self.simple_bind(who,cred,serverctrls,clientctrls)
   File "/usr/lib/python2.6/site-packages/ldap/ldapobject.py", line 195, in 
simple_bind
     return 
self._ldap_call(self._l.simple_bind,who,cred,EncodeControlTuples(serverctrls),EncodeControlTuples(clientctrls))
   File "/usr/lib/python2.6/site-packages/ldap/ldapobject.py", line 96, in 
_ldap_call
     result = func(*args,**kwargs)
ldap.SERVER_DOWN: {'desc': "Can't contact LDAP server"}
 >>>

> Would shutting off slapd cause this error (I assume),

Yes. That's how ReconnectLDAPObject was tested.

> We are assessing our production environment.  For now we are staying with
> Python2.4.4, is python-ldap 2.3.1 stable with Python-2.4.4?

Provided python-ldap 2.3.1 was built from source it's stable. If you're 
using a binary package for which the package maintainer applied a patch set 
you have to ask the package maintainer.

Also note that stable means it has to be linked to stable OpenLDAP libs 
(mainly without bugs in libldap) which in turn has to be linked to stable 
versions of OpenSSL (not gnu-tls like in Debian), cyrus-sasl and Kerberos 
libs. Well, that's the caveat of "standing on the shoulders of giants".

>> Please also note that always unbind_s() should be called. 
> 
> I thought unbind and unbind_s called the same method internally.  Do I need
> to change my calls to unbind_s?  Is that for clarity or is there an
> implementation difference?

You have to grab the result() for unbind(). AFAIK unbind_s() should not 
block. So you should try using it.

Ciao, Michael.