Re: [webmin-l] LDAP Users and Groups module performance

[Jamie C. <jca...@we...>]

On 21/Jan/2007 16:11 Murray Trainer wrote ..
> On Fri, 2007-01-19 at 00:23 -0800, Jamie Cameron wrote:
> > On 18/Jan/2007 22:46 Murray Trainer wrote ..
> > > Hi Jamie
> > > 
> > > We now have over 1000 users we manage with the LDAP Users module. 
> I
> > > have noticed lately that it is becoming very slow bringing up the
> > > initial screen.  I can imagine things only getting worse as the
> > > directory gets larger.  I haven't checked the queries My guess is that
> > > the module tries to search the LDAP directory for the whole list of
> > > users, then after realising there are too many, it then says there
> are
> > > too many to display and fails.  Even if I set the maximum number of
> > > users or groups to display to 0 or 1 it still takes a fair few seconds
> > > to display the opening screen.  
> > > 
> > > The recommed method is to process each LDAP user entry one at a time
> as
> > > they are returned. The module would stop when it reached the limit
> of
> > > users for that page which could be a configuration value.  The module
> > > appears to be using the NET::LDAP library.  I had a look at the
> > > documentation below and they recommend processing each entry as it
> is
> > > returned:  
> > > 
> > > http://search.cpan.org/~gbarr/perl-ldap-0.33/lib/Net/LDAP/FAQ.pod#USING_THE_CALLBACK_SUBROUTINE_APPROACH
> > > 
> > > If we want 100 users to be displayed on the page we print the first
> 100
> > > entries then quit.  If we want users 100-200 we would ignore the first
> > > hundred entries and print the second hundred, and so on.
> > > 
> > > Hope the above idea has merit and its possible for you to implement
> at
> > > some stage.
> > 
> > Actually, that is what the module already does. The main page does an
> > LDAP search for matching users, and then checks the number of results
> available.
> > If this is higher than the maximum set on the Module Config page, it
> just
> > displays the search form without fetching the full user list.
> > 
> > Of course, for this to be useful I assume that the LDAP server doesn't
> > fetch users from it's database until actually asked to..
> > 
> >  - Jamie
> > 
> 
> If it does things in the sequential manner then I am not sure why it is
> so slow for me.  There is nothing wrong with the performance of the ldap
> server.  I can do an ldapsearch from a client machine and dump the whole
> list of users into a text file in less than a second.  Going into the
> first ldap users and groups screen with the list of users takes > 10
> secs even with the amount of users to display set to 0. I am not sure
> where the bottleneck in performance is?

I took another look at the code for this module, and it seems that even
though Webmin just calls the Net::LDAP module's search function, this
takes a long time to respond on large LDAP databases even though no
actual results are requested!

However, I managed to find another way to reduce the search size, by
specifying a maximum result size to the LDAP server. This will go into
the next (1.330) Webmin release.

 - Jamie