Re: [htdig-dev] Re: [htdig-members] Results of Possible Efficiency improvment for 3.2.0

[Joe R. J. <jj...@cl...>]

On Wed, 12 Jan 2005, Jim wrote:

> Date: Wed, 12 Jan 2005 23:02:12 -0700 (MST)
> From: Jim <li...@yg...>
> To: Neal Richter <ne...@ri...>
> Cc: htd...@li...
> Subject: [htdig-dev] Re: [htdig-members] Results of Possible Efficiency improvment for 3.2.0
> 
> Moved this to developers list.
> 
> On Wed, 5 Jan 2005, Neal Richter wrote:
> 
> > 	As far as speed testing I would love it if a couple of you could 
> > index a decent sized site (pefferable local to eliminate network latency) 
> > with and without the index compression enabled and report back your speed & 
> > size results.
> >
> > wordlist_compress:         false
> > wordlist_compress_zlib:    false
> 
> Here is what I got.
> 
> With compression:
>    2296.03 user
>    224.52 system
>    8:10:56 elapsed
> 
>    10592256 Jan 10 09:46 db.docdb
>    10461184 Jan 10 09:44 db.docs.index
>    90120192 Jan 10 09:44 db.excerpts
>    113465344 Jan 10 09:44 db.words.db
>       16384 Jan 10 09:44 db.words.db_weakcmpr

Not that it makes much of a difference, but it looks like you do not
compress the excerpts, compression_level: 0; right?

> Without compression:
>    450.46 user
>    123.33 system
>    4:56:01 elapsed
> 
>    Misplaced the file sizes for this run. I can rerun it some night if it
>    is important.

Add a `/bin/ls -l $DBDIR` line at the end of your rundig-test script, and
never misplace them;)

> These numbers are for a local dig on a dual Xeon box with RAID 5 and a
> couple GB of RAM. No swapping occurred.

Hmmm, no swapping!  That may be the deciding factor why the change in your
elapsed time is so different than mine; or may be your CPU speed;-/

My system is similar to yours, dual Xeon 2.4 GHz box RAID 5 10K RPM SCSI
drives, and 2.0 GB DDR RAM; What's your CPU speed?

Regards,

Joe
-- 
     _/   _/_/_/       _/              ____________    __o
     _/   _/   _/      _/         ______________     _-\<,_
 _/  _/   _/_/_/   _/  _/                     ......(_)/ (_)
  _/_/ oe _/   _/.  _/_/ ah        jj...@cl...