Menu
▾
▴
|
From: Anders K. <and...@ce...> - 2002-06-10 08:04:36
|
I recently started to use Python-LDAP in a project. Since I first found the 1.10alpha3-version I used that. Everything worked fine and fast. Since it was built for Python 1.x I upgraded to Python-LDAP 2.0.0pre04 which worked fine, but considerable slower that the old version. A python program that changed some posts in the LDAP database and did two searches took about five seconds with 2.0.0pre04, compared to less than a second with 1.10alpha3. Why is there such big speed difference? Have I done something wrong or is= =20 it a known matter? Which version do you recomend me to use? Regards, -------------------------------------------------------------------------= - Anders Karlsson Email: and...@ce... Cendio Systems AB WWW: www.cendio.se Teknikringen 3, Voice: +46 - (0)13 - 21 46 00 SE-583 30 LINK=D6PING, SWEDEN Fax: +46 - (0)13 - 21 47 00 |
|
From: <mi...@st...> - 2002-06-14 16:51:58
|
Anders Karlsson wrote:
>
> Since it was built for Python 1.x I upgraded to Python-LDAP 2.0.0pre04
> which worked fine, but considerable slower that the old version. A python
> program that changed some posts in the LDAP database and did two searches
> took about five seconds with 2.0.0pre04, compared to less than a second
> with 1.10alpha3.
>
> Why is there such big speed difference?
Hmm, there might be different issues here:
1. I changed the implementation of the synchronous methods *_s()
to avoid getting blocked in OpenLDAP 2 _s() functions. Especially
there's a reimplementation of result() in
ldap.ldapobject.LDAPObject which does the timeout handling itself.
The main reason is that the OpenLDAP libs are not thread-safe and
therefore a module-wide lock is used to serialize all calls into
the OpenLDAP libs.
(If someone manages to use OpenLDAP 2.1's libldap_r instead of
libldap this might help to reduce some of the locking.)
2. You might also have upgraded the LDAP server to OpenLDAP 2
(which does stricter checking) or changed the indexing
configuration (which is a performance penalty when writing)?
3. The OpenLDAP 2.x libs are slower. (python-ldap 1.x was linked
against OpenLDAP 1.x libs.)
Now for 1. issue:
You could try a very simple result() implementation (tweak
Lib/ldap/ldapobject.py):
def result(self,msgid=_ldap.RES_ANY,all=1,timeout=-1):
return self._ldap_call(self._l.result,msgid,all,timeout)
I did some testing with this method implementation for measuring
the overhead of my non-blocking version. I read the RootDSE of
Netscape DS 4.16SP1 running on the same box. (Solely searching the
RootDSE is an appropriate method to eliminate the influence of
database backends and such.)
On average it seems to be approx. 65% faster to use this simple
method above. This is something to consider.
Hmm, but you are experiencing a performance difference which is
much higher (5 times slower as I understand your posting). I'd
really appreciate if you could send more information about your
environment and *all* the changes you did. Performance measurement
numbers done under really the same conditions are also appreciated
off course.
Ciao, Michael.
|
|
From: Mauro C. <mci...@si...> - 2002-06-14 16:59:30
Attachments:
smime.p7s
|
I have experienced significant delays, too, using the new version. Since I have recently managed to compile the whole stuff under Win32, I am doing some experimenting before releasing the binaries, and this is actually blocking me a bit. However, I have seen that the searches don't show significant differences with the old 1.x version. The single call that takes a lot is the bind step: when I ask a Python script to bind it just seems to sit there and wait, more or less for 5 seconds (strange coincidence). Could all this be related to the SASL libraries? Mauro |
|
From: <mi...@st...> - 2002-06-14 18:05:54
|
Mauro Cicognini wrote: > > However, I have seen that the searches don't show significant > differences with the old 1.x version. Can you provide numbers? > The single call that takes a lot is the bind step: when I ask a Python > script to bind it just seems to sit there and wait, more or less for 5 > seconds (strange coincidence). > > Could all this be related to the SASL libraries? Hmm, no idea. I guess you are you talking about a simple bind. Ciao, Michael. |
|
From: <mi...@st...> - 2002-06-14 18:06:03
|
Michael Str=F6der wrote: > Anders Karlsson wrote: >=20 > You could try a very simple result() implementation (tweak=20 > Lib/ldap/ldapobject.py): >=20 > def result(self,msgid=3D_ldap.RES_ANY,all=3D1,timeout=3D-1): > return self._ldap_call(self._l.result,msgid,all,timeout) >=20 Just a warning: This is not meant as serious replacement. Just for=20 measuring the performance. Rest of LDAPObject class would have to=20 be tweaked too. Ciao, Michael. |
|
From: Anders K. <and...@ce...> - 2002-06-17 08:39:28
|
On Fri, 14 Jun 2002, Michael Str=F6der wrote: > 2. You might also have upgraded the LDAP server to OpenLDAP 2=20 > (which does stricter checking) or changed the indexing=20 > configuration (which is a performance penalty when writing)? Yes, it's linked against OpenLDAP 2. But this can't be the problem alone=20 since the bundled ldap-tools (ldapadd, ldapsearch..) works fast. Thanks a lot for your answers and ideas! I will try to look deeper into m= y=20 environment and see if I can find the bottlenecks. Regards, -------------------------------------------------------------------------= - Anders Karlsson Email: and...@ce... Cendio Systems AB WWW: www.cendio.se Teknikringen 3, Voice: +46 - (0)13 - 21 46 00 SE-583 30 LINK=D6PING, SWEDEN Fax: +46 - (0)13 - 21 47 00 |
