Re: [htdig-dev] Solved! - Re: Performance issue with exclude_urls

[Christopher M. <chr...@mc...>]

On Wed, 2004-04-21 at 23:21, Gilles Detillieux wrote:
> > > 3) We may also need to determine if the repeated calls to config->Find()
> > > at each URL are having an impact on performance as well.  E.g. what is
> > > the performance cost of doing thousands of calls like this one?
> > > 
> > >      tmpList.Create(config->Find(&aUrl, "exclude_urls"), " \t");
> > 
> >  Easy thing to test. I'll give it a try later this week if I can,
> > perhaps tomorrow, and report back.
> 
> Great.  I'll try to get my fix to Regex.cc in by the end of the week too,
> so it would be great if you could give it a whirl.  It would probably
> mean having to back out your own patch, though, or it wouldn't really
> get tested.

 Here's what I did. I simply added a new variable called 'fooList' of
type StringList. Then I inserted extra Create() method calls:


//these lines are simply to test the CPU load of the Create() method
fooList.Create(config->Find(&aUrl, "exclude_urls"), " \t");
fooList.Destroy();

//these lines are simply to test the CPU load of the Create() method
fooList.Destroy();
fooList.Create(config->Find(&aUrl, "bad_querystr"), " \t");
fooList.Destroy();

 Then I re-compiled and ran with my normal excludes URL list. It didn't
seem to have much of an impact on performance. This means that the
performance hit is definitely in the HtRegexList::setEscaped method.

Cheers,

Chris

-- 
Christopher Murtagh
Enterprise Systems Administrator
ISR / Web Communications Group 
McGill University
Montreal, Quebec
Canada

Tel.: (514) 398-3122
Fax:  (514) 398-2017