Re: [webmin-devel] DNS modules with large zones

[John H. <j....@pl...>]

On Thu, 2003-09-18 at 00:10, Jamie Cameron wrote:
>
> I can see why webmin would be slow with an 18000 record zone file. 
> Currently, it parses that entire file every time you open a page that 
> does anything within the zone, and reading such a large file with Perl 
> is going to take a long time..
> 
> Unfortunately, there isn't really any solution, except modifying 
> webmin to add some kind of indexing or caching code for BIND zone files.
> 
Agreed. However, caching is already in place in the module so the zone
file is only read once per cgi script. The problem, as you said, is that
the zone has to be read whenever it is being looked at, modified or
wahetever. I see no way around that since the information may well
change.

I have done some simple timing tests:

  to read the zone (either dynamic or static zones): 10 seconds
  to parse the zone into the required structures: 5 seconds

so to 'read' a zone takes about 15 seconds. Dynamic zones are using dig
whilst static ones are read direct from the file. There is no real
difference in time between the two.

  to remove the '$GENERATE' entries: 20 seconds

Ah! This one line is a 'grep' in the edit_master.cgi script. If I
comment that out, because we have no generates in there, and dynamic
zones certainly won't see them, then the time drops to 15 seconds again
- that is, just the time to read the zone.

  to format and display the zone at this point: 1 second

so the displaying of the number of different records in a zone is not a
problem.

Next bit is displaying particular record types:

   Removing the $generate entries: 20 seconds
   Selecting the particular records to display: 1 second
   Sorting and displaying the records: 14 seconds


Okay. We can get around the $GENERATE bit for dynamic zones, and for
static ones we can probably cache something as the zone file is parsed.
The perl 'grep' seems to be slow and perhaps could be replaced with
in-line code or a faster function. The actual reading of the zone and
its parsing may possibly be speeded up, and the same for the displaying
of the records. Of course, ultimately it could just be that there is no
faster way and a large zone will take this sort of time :-)




John.

-- 
---------------------------------------------------------------
John Horne, University of Plymouth, UK  Tel: +44 (0)1752 233914
E-mail: Joh...@pl...       Fax: +44 (0)1752 233839