Re: [htdig-dev] Compression related performance

[Joe R. J. <jj...@cl...>]

On Sat, 15 Jan 2005, Neal Richter wrote:

> Date: Sat, 15 Jan 2005 16:06:09 -0700 (MST)
> From: Neal Richter <ne...@ri...>
> To: Jim <li...@yg...>
> Cc: htd...@li...
> Subject: Re: [htdig-dev] Compression related performance
> 
> Jim,
>  	Thanks for the numbers.  Pretty different from Joe's.

Yes they are; can you explain it?

> Question #1)
> 
>     Could you download a fresh copy of BDB 3.0.55 from
> http://www.sleepycat.com/download/index.shtml
> 
> It's easy to build, I am most interested in the results when you run 
> db_stat on the uncompressed files and report back the fill-factor.  This 
> is a measure of wasted space in the BDB pages.  This won't work on the 
> compressed index.
> 
> Question #2)
>    What page_size are you using?
> 
> Question #3)
>    Do you ever see any errors when using the compressed index?
> 
>    Joe:  Do you?

No I don't.

Regards,

Joe
-- 
     _/   _/_/_/       _/              ____________    __o
     _/   _/   _/      _/         ______________     _-\<,_
 _/  _/   _/_/_/   _/  _/                     ......(_)/ (_)
  _/_/ oe _/   _/.  _/_/ ah        jj...@cl...

> Thanks
> 
> Neal
> 
> On Fri, 14 Jan 2005, Jim wrote:
> 
> > Here is another performance comparison collected when the server was under 
> > less load. The data is essentially the same as that used for the last set of 
> > numbers.
> >
> > Uncompressed
> > ------------
> > 416.28 user
> > 97.72 system
> > 1:30:23 elapsed
> >
> >  10510336 Jan 15 00:00 db.docdb
> >  10444800 Jan 15 00:00 db.docs.index
> >  90488832 Jan 15 00:00 db.excerpts
> >  436043776 Jan 15 00:00 db.words.db
> >
> > Compressed
> > ----------
> > 2008.63 user
> > 133.35 system
> > 2:40:16 elapsed
> >
> >  10526720 Jan 14 09:35 db.docdb
> >  10444800 Jan 14 09:35 db.docs.index
> >  90611712 Jan 14 09:36 db.excerpts
> >  113773568 Jan 14 09:36 db.words.db
> >     16384 Jan 14 09:36 db.words.db_weakcmpr
> >
> >
> > I do have a large data set that I could test with if it would be of real
> > value. However I would need to try rebuilding ht://Dig with large file
> > support before I could consider dropping compression for that set.
> >
> >
> > Jim
> >
> >
> > -------------------------------------------------------
> > The SF.Net email is sponsored by: Beat the post-holiday blues
> > Get a FREE limited edition SourceForge.net t-shirt from ThinkGeek.
> > It's fun and FREE -- well, almost....http://www.thinkgeek.com/sfshirt
> > _______________________________________________
> > ht://Dig Developer mailing list:
> > htd...@li...
> > List information (subscribe/unsubscribe, etc.)
> > https://lists.sourceforge.net/lists/listinfo/htdig-dev
> >
> 
> -- 
> Neal Richter 
> Knowledgebase Developer
> RightNow Technologies, Inc.
> Customer Service for Every Web Site
> Office: 406-522-1485