From: Ed . <ep...@ho...> - 2003-06-23 12:13:57
|
Attached is the script I used to compare python-ldap with ldapsearch. Script ran on same machine as the ldap server. The two envirnments teste used fairly recent openldaps and in one case had python-ldap 2.0.0(pre6), the other was more recent. Ratio of python-ldap:ldapsearch seemed constant regardless of version. Ed #!/home/zope/bin/python from os import system, remove from posix import getpid, close from random import random import time import ldap, string search='(cn=*)' host='localhost' port=389 uname="" pw="" # set your directory base here base = "dc=da,dc=mod,dc=uk" iter = 1000 raw = [] def timeit(fn): start = time.time() fn() return time.time() - start # Search whole directory, returning all attributes, python and command line versions def dump_all_py(): global base, search, raw raw = l.search_s(base, ldap.SCOPE_SUBTREE, search) def dump_all_sh(): cmd = 'ldapsearch -h %s -p %i -b "%s" -s sub -x -D "%s" -w "%s" \'%s\' >/dev/null' % \ (host, port, base, uname, pw, search) system(cmd) #print cmd # Search whole directory, returning cn, python and command line versions def dump_cn_py(): global base, search, raw raw = l.search_s(base, ldap.SCOPE_SUBTREE, search, ['cn']) def dump_cn_sh(): cmd = 'ldapsearch -h %s -p %i -b "%s" -s sub -x -D "%s" -w "%s" \'%s\' %s >/dev/null' % \ (host, port, base, uname, pw, search, 'cn') system(cmd) #print cmd # Python cn lookup, exact and initial substring versions def lk_exact_py(): global ents, base for cn in ents: search = '(cn=%s)' % (cn) entry = l.search_s(base, ldap.SCOPE_SUBTREE, search) def lk_sub_py(): global ents, base for cn in ents: search = '(cn=%s*)' % (cn) entry = l.search_s(base, ldap.SCOPE_SUBTREE, search) # Command line cn lookup, exact and initial substring versions # Prepare datafiles for use with command line lookups def prep_sh_data(): global ents, base, tmpf1, tmpf2 tmpf1 = '/tmp/ldap%d-1' % (getpid()) tmpf2 = '/tmp/ldap%d-2' % (getpid()) t1 = open(tmpf1, 'w') t2 = open(tmpf2, 'w') for cn in ents: print >>t1, cn print >>t2, cn+'*' t1.close() t2.close() cmdstring = 'ldapsearch -f %s -h %s -p %i -b "%s" -s sub -x -D "%s" -w "%s" %s>/dev/null' def lk_exact_sh(): global base, tmpf1 cmd = cmdstring % (tmpf1, host, port, base, uname, pw, "'(cn=%s)'") system(cmd) #print cmd def lk_sub_sh(): global base, tmpf2 cmd = cmdstring % (tmpf2, host, port, base, uname, pw, "'(cn=%s*)'") system(cmd) #print cmd l = ldap.initialize('ldap://%s:%s' % (host, port)) l.simple_bind_s(uname, pw) print 'Reading complete directory all attributes' elapsed = timeit(dump_all_py) ct = len(raw) print 'PY: %i entries in %5.2f secs, %5.2f entries/sec' % (ct, elapsed, ct/elapsed) elapsed = timeit(dump_all_sh) l = ldap.initialize('ldap://%s:%s' % (host, port)) l.simple_bind_s(uname, pw) print 'Reading complete directory all attributes' elapsed = timeit(dump_all_py) ct = len(raw) print 'PY: %i entries in %5.2f secs, %5.2f entries/sec' % (ct, elapsed, ct/elapsed) elapsed = timeit(dump_all_sh) print ' C: %i entries in %5.2f secs, %5.2f entries/sec' % (ct, elapsed, ct/elapsed) print '\nReading complete directory, single attribute' raw=[] elapsed = timeit(dump_cn_py) ct = len(raw) print 'PY: %i entries in %5.2f secs, %5.2f entries/sec' % (ct, elapsed, ct/elapsed) elapsed = timeit(dump_cn_sh) print ' C: %i entries in %5.2f secs, %5.2f entries/sec' % (ct, elapsed, ct/elapsed) # First build up a list of entries to lookup ents = [] i = 0 while i < iter: e = int(random()*ct) # N.B. sort this out! ents.append(raw[e][1]['cn'][0]) i += 1 print '\n%i lookups (exact match)' % (iter) elapsed = timeit(lk_exact_py) print 'PY: %5.2f secs, %5.2f entries/sec' % (elapsed, iter/elapsed) prep_sh_data() elapsed = timeit(lk_exact_sh) print ' C: %5.2f secs, %5.2f entries/sec' % (elapsed, iter/elapsed) print '\n%i lookups (substring)' % (iter) elapsed = timeit(lk_sub_py) print 'PY: %5.2f secs, %5.2f entries/sec' % (elapsed, iter/elapsed) elapsed = timeit(lk_sub_sh) print ' C: %5.2f secs, %5.2f entries/sec' % (elapsed, iter/elapsed) remove(tmpf1) remove(tmpf2) l.unbind_s() _________________________________________________________________ Stay in touch with absent friends - get MSN Messenger http://www.msn.co.uk/messenger |
From: <mi...@st...> - 2003-06-23 12:52:36
|
Ed . wrote: > Attached is the script I used to compare python-ldap with ldapsearch. Not sure whether everything's ok with your script. On my LDAP server the following appears in the log: Jun 23 14:47:03 nb2 slapd[10223]: conn=67 op=9999 SEARCH RESULT tag=101 err=0 nentries=0 text= Jun 23 14:47:03 nb2 slapd[10218]: conn=67 op=10000 SRCH base="dc=stroeder,dc=de" scope=2 filter="(?=undefined)" The temporary file only contains the name, no attribute type for the search filter. Anyway, if you run it with python -O you will see a big performance gain since all the trace logging stuff in LDAPObject._ldap_call() is completely deactivated (enclose in if __debug__). Ciao, Michael. |