Re: [webmin-l] LDAP Users and Groups module performance

[Jamie C. <jca...@we...>]

On 27/Feb/2007 22:21 Murray Trainer wrote ..
> On Sun, 2007-02-25 at 22:49 -0800, Jamie Cameron wrote:
> > On 25/Feb/2007 22:19 Murray Trainer wrote ..
> > > On Sun, 2007-01-21 at 22:27 -0800, Jamie Cameron wrote:
> > > > On 21/Jan/2007 16:11 Murray Trainer wrote ..
> > > > > On Fri, 2007-01-19 at 00:23 -0800, Jamie Cameron wrote:
> > > > > > On 18/Jan/2007 22:46 Murray Trainer wrote ..
> > > > > > > Hi Jamie
> > > > > > > 
> > > > > > > We now have over 1000 users we manage with the LDAP Users module.
> > > > > I
> > > > > > > have noticed lately that it is becoming very slow bringing
> up the
> > > > > > > initial screen.  I can imagine things only getting worse as
> the
> > > > > > > directory gets larger.  I haven't checked the queries My guess
> > > is that
> > > > > > > the module tries to search the LDAP directory for the whole
> list
> > > of
> > > > > > > users, then after realising there are too many, it then says
> there
> > > > > are
> > > > > > > too many to display and fails.  Even if I set the maximum number
> > > of
> > > > > > > users or groups to display to 0 or 1 it still takes a fair
> few
> > > seconds
> > > > > > > to display the opening screen.  
> > > > > > > 
> > > > > > > The recommed method is to process each LDAP user entry one
> at a
> > > time
> > > > > as
> > > > > > > they are returned. The module would stop when it reached the
> limit
> > > > > of
> > > > > > > users for that page which could be a configuration value. 
> The
> > > module
> > > > > > > appears to be using the NET::LDAP library.  I had a look at
> the
> > > > > > > documentation below and they recommend processing each entry
> as
> > > it
> > > > > is
> > > > > > > returned:  
> > > > > > > 
> > > > > > > http://search.cpan.org/~gbarr/perl-ldap-0.33/lib/Net/LDAP/FAQ.pod#USING_THE_CALLBACK_SUBROUTINE_APPROACH
> > > > > > > 
> > > > > > > If we want 100 users to be displayed on the page we print the
> first
> > > > > 100
> > > > > > > entries then quit.  If we want users 100-200 we would ignore
> the
> > > first
> > > > > > > hundred entries and print the second hundred, and so on.
> > > > > > > 
> > > > > > > Hope the above idea has merit and its possible for you to implement
> > > > > at
> > > > > > > some stage.
> > > > > > 
> > > > > > Actually, that is what the module already does. The main page
> does
> > > an
> > > > > > LDAP search for matching users, and then checks the number of
> results
> > > > > available.
> > > > > > If this is higher than the maximum set on the Module Config page,
> > > it
> > > > > just
> > > > > > displays the search form without fetching the full user list.
> > > > > > 
> > > > > > Of course, for this to be useful I assume that the LDAP server
> doesn't
> > > > > > fetch users from it's database until actually asked to..
> > > > > > 
> > > > > >  - Jamie
> > > > > > 
> > > > > 
> > > > > If it does things in the sequential manner then I am not sure why
> it
> > > is
> > > > > so slow for me.  There is nothing wrong with the performance of
> the
> > > ldap
> > > > > server.  I can do an ldapsearch from a client machine and dump
> the
> > > whole
> > > > > list of users into a text file in less than a second.  Going into
> the
> > > > > first ldap users and groups screen with the list of users takes
> > 10
> > > > > secs even with the amount of users to display set to 0. I am not
> sure
> > > > > where the bottleneck in performance is?
> > > > 
> > > > I took another look at the code for this module, and it seems that
> even
> > > > though Webmin just calls the Net::LDAP module's search function,
> this
> > > > takes a long time to respond on large LDAP databases even though
> no
> > > > actual results are requested!
> > > > 
> > > > However, I managed to find another way to reduce the search size,
> by
> > > > specifying a maximum result size to the LDAP server. This will go
> into
> > > > the next (1.330) Webmin release.
> > > > 
> > > >  - Jamie
> > > 
> > > Hi Jamie,
> > > 
> > > I noticed that you had added the fix above to the 1.328 version.  I
> > > downloaded it and tested it.  It displays the initial list of
> > > users/groups immediately if it exceeds the limit for the page.  Thanks
> > > for your work on that - it will improve things a lot in our setup.
> > 
> > Excellent .. glad to hear that solved it.
> > 
> > > I noticed there is now a different Webmin theme by default.  I tried
> > > changing to the previous theme and couldn't find it.  All the themes
> I
> > > chose appeared the same to me.  I couldn't change back to the new
> > > default theme either.  Not sure the new theme stuff is right?
> > 
> > Did you use the Change Language and Theme module to switch themes? If
> > you change the global setting in the Webmin Configuration module, it
> > can still be over-ridden on a per-user basis by the former module..
> > 
> >  - Jamie
> 
> I just tried the Change Language and Theme module and that seems to work
> better.  It's a bit confusing having both options though.

On is for the global default (Webmin Configuration), the other is for a 
per-user setting (Change Language and Theme).

 - Jamie