Re: [webmin-l] LDAP Users and Groups module performance

[Jamie C. <jca...@we...>]

On 25/Feb/2007 22:19 Murray Trainer wrote ..
> On Sun, 2007-01-21 at 22:27 -0800, Jamie Cameron wrote:
> > On 21/Jan/2007 16:11 Murray Trainer wrote ..
> > > On Fri, 2007-01-19 at 00:23 -0800, Jamie Cameron wrote:
> > > > On 18/Jan/2007 22:46 Murray Trainer wrote ..
> > > > > Hi Jamie
> > > > > 
> > > > > We now have over 1000 users we manage with the LDAP Users module.
> > > I
> > > > > have noticed lately that it is becoming very slow bringing up the
> > > > > initial screen.  I can imagine things only getting worse as the
> > > > > directory gets larger.  I haven't checked the queries My guess
> is that
> > > > > the module tries to search the LDAP directory for the whole list
> of
> > > > > users, then after realising there are too many, it then says there
> > > are
> > > > > too many to display and fails.  Even if I set the maximum number
> of
> > > > > users or groups to display to 0 or 1 it still takes a fair few
> seconds
> > > > > to display the opening screen.  
> > > > > 
> > > > > The recommed method is to process each LDAP user entry one at a
> time
> > > as
> > > > > they are returned. The module would stop when it reached the limit
> > > of
> > > > > users for that page which could be a configuration value.  The
> module
> > > > > appears to be using the NET::LDAP library.  I had a look at the
> > > > > documentation below and they recommend processing each entry as
> it
> > > is
> > > > > returned:  
> > > > > 
> > > > > http://search.cpan.org/~gbarr/perl-ldap-0.33/lib/Net/LDAP/FAQ.pod#USING_THE_CALLBACK_SUBROUTINE_APPROACH
> > > > > 
> > > > > If we want 100 users to be displayed on the page we print the first
> > > 100
> > > > > entries then quit.  If we want users 100-200 we would ignore the
> first
> > > > > hundred entries and print the second hundred, and so on.
> > > > > 
> > > > > Hope the above idea has merit and its possible for you to implement
> > > at
> > > > > some stage.
> > > > 
> > > > Actually, that is what the module already does. The main page does
> an
> > > > LDAP search for matching users, and then checks the number of results
> > > available.
> > > > If this is higher than the maximum set on the Module Config page,
> it
> > > just
> > > > displays the search form without fetching the full user list.
> > > > 
> > > > Of course, for this to be useful I assume that the LDAP server doesn't
> > > > fetch users from it's database until actually asked to..
> > > > 
> > > >  - Jamie
> > > > 
> > > 
> > > If it does things in the sequential manner then I am not sure why it
> is
> > > so slow for me.  There is nothing wrong with the performance of the
> ldap
> > > server.  I can do an ldapsearch from a client machine and dump the
> whole
> > > list of users into a text file in less than a second.  Going into the
> > > first ldap users and groups screen with the list of users takes > 10
> > > secs even with the amount of users to display set to 0. I am not sure
> > > where the bottleneck in performance is?
> > 
> > I took another look at the code for this module, and it seems that even
> > though Webmin just calls the Net::LDAP module's search function, this
> > takes a long time to respond on large LDAP databases even though no
> > actual results are requested!
> > 
> > However, I managed to find another way to reduce the search size, by
> > specifying a maximum result size to the LDAP server. This will go into
> > the next (1.330) Webmin release.
> > 
> >  - Jamie
> 
> Hi Jamie,
> 
> I noticed that you had added the fix above to the 1.328 version.  I
> downloaded it and tested it.  It displays the initial list of
> users/groups immediately if it exceeds the limit for the page.  Thanks
> for your work on that - it will improve things a lot in our setup.

Excellent .. glad to hear that solved it.

> I noticed there is now a different Webmin theme by default.  I tried
> changing to the previous theme and couldn't find it.  All the themes I
> chose appeared the same to me.  I couldn't change back to the new
> default theme either.  Not sure the new theme stuff is right?

Did you use the Change Language and Theme module to switch themes? If 
you change the global setting in the Webmin Configuration module, it
can still be over-ridden on a per-user basis by the former module..

 - Jamie