[htdig-dev] Re: [htdig-members] Results of Possible Efficiency improvment for 3.2.0

[Jim <li...@yg...>]

Moved this to developers list.

On Wed, 5 Jan 2005, Neal Richter wrote:

> 	As far as speed testing I would love it if a couple of you could 
> index a decent sized site (pefferable local to eliminate network latency) 
> with and without the index compression enabled and report back your speed & 
> size results.
>
> wordlist_compress:         false
> wordlist_compress_zlib:    false

Here is what I got.

With compression:
   2296.03 user
   224.52 system
   8:10:56 elapsed

   10592256 Jan 10 09:46 db.docdb
   10461184 Jan 10 09:44 db.docs.index
   90120192 Jan 10 09:44 db.excerpts
   113465344 Jan 10 09:44 db.words.db
      16384 Jan 10 09:44 db.words.db_weakcmpr

Without compression:
   450.46 user
   123.33 system
   4:56:01 elapsed

   Misplaced the file sizes for this run. I can rerun it some night if it
   is important.

These numbers are for a local dig on a dual Xeon box with RAID 5 and a
couple GB of RAM. No swapping occurred.

Jim