[htdig-dev] Compression related performance

[Jim <li...@yg...>]

Here is another performance comparison collected when the server was under 
less load. The data is essentially the same as that used for the last set 
of numbers.

Uncompressed
------------
416.28 user
97.72 system
1:30:23 elapsed

   10510336 Jan 15 00:00 db.docdb
   10444800 Jan 15 00:00 db.docs.index
   90488832 Jan 15 00:00 db.excerpts
   436043776 Jan 15 00:00 db.words.db

Compressed
----------
2008.63 user
133.35 system
2:40:16 elapsed

   10526720 Jan 14 09:35 db.docdb
   10444800 Jan 14 09:35 db.docs.index
   90611712 Jan 14 09:36 db.excerpts
   113773568 Jan 14 09:36 db.words.db
      16384 Jan 14 09:36 db.words.db_weakcmpr


I do have a large data set that I could test with if it would be of real
value. However I would need to try rebuilding ht://Dig with large file
support before I could consider dropping compression for that set.


Jim