Menu
▾
▴
Re: [htdig-dev] Almost there...
|
From: Joe R. J. <jj...@cl...> - 2003-11-13 21:08:10
|
On Thu, 13 Nov 2003, Gilles Detillieux wrote:
> Date: Thu, 13 Nov 2003 13:38:05 -0600 (CST)
> From: Gilles Detillieux <gr...@sc...>
> To: Joe R. Jah <jj...@cl...>
> Cc: htd...@li...
> Subject: Re: [htdig-dev] Almost there...
>
> According to Joe R. Jah:
> > Job well done! It configured/built/ran out of the box on my BSD/OS-4.3.1
> > with gcc 2.95.3 like a charm; It took only 96 minutes to index my site;)
>
> How does this compare to earlier 3.2.0b4 snapshots, and to 3.1.6?
> Is 3.2.0b5 significantly slower than 3.1 releases, and is it better or
> worse than earlier 3.2 betas?
First of all I should correct the indexing time; that one was sent in
hurry to express my joy;) and didn't realize that it was indexing the site
twice; once for http and again for https;( I added a rewrite rule:
url_rewrite_rules: https://(.*) http://\\1
And now 3.2.0b5 indexes my site, ~15,000 docs, in 54 minutes, even more
joyous;)) For comparison fully patched 3.1.6 indexes it in 12 minutes;
however, it indexes more pages because of the fileSpce.1 patch.
Unfortunately in our site we have many file names that include space in
them. Roughly about 5% more documents are indexed by my 3.1.6 than
3.2.0b5. I'd say it takes five times longer for 3.2.0b5 to index the
site.
I can't directly compare the results of 3.2.0b5 with 3.2.0b4 because my
old statistics were taken on a slower machine. Here is an old statistics
I have posted to the list:
Machine: 300 MHz PentiumII
RAM: 256 MB
SWAP: 768 MB
OS: BSDI 4.01
Documents: ~5,000
With different versions of htdig:
3.1.5 11 Minutes
3.2.0b3 9 1/2 hours
3.2.0b4-031201 29 hours and 20 minutes
3.2.0b4-040801 > 12 days
You can see that 3.2.0b5's performance has greatly improved.
Regards,
Joe
--
_/ _/_/_/ _/ ____________ __o
_/ _/ _/ _/ ______________ _-\<,_
_/ _/ _/_/_/ _/ _/ ......(_)/ (_)
_/_/ oe _/ _/. _/_/ ah jj...@cl...
|
