Re: [Exist-development] Benchmarking eXist

[Loren C. <lor...@gm...>]

The benchmark target is not listed in the "./build.sh -p" command results.

On Nov 12, 2009, at 03:13 PM, Wolfgang Meier wrote:

> Hi,
> 
>> I see that eXist has some performance tests in the test
>> suite, but I don't know whether they are still maintained or used as
>> guides for further development.
> 
> I wrote those tests and I still use them whenever I make larger
> changes to the query engine ("build.sh benchmark"). The tests take too
> many resources to be run as part of the standard test suite. Memory
> and cache settings need to be increased a bit (1gb main memory, 256mb
> cache), otherwise loading the 600MB dblp bibliography file will just
> take too long. Also, I don't always run all tests. However, it is a
> useful tool to make sure my changes do not result in a loss of
> performance.
> 
>> From my experience I would say that you need a large number of
> slightly different queries in every tested area to really measure
> performance. Benchmarks which are based on 10 or 20 queries are pretty
> useless (at least to me as a developer). XQuery often provides more
> than one way to address a problem. For example, it may make a
> difference for the query engine if you use a "where" clause or an
> XPath predicate, even though the result of both expressions will be
> the same. Minimal changes in the query formulation may cause the query
> to take 20 times as long as before. The underlying data structure also
> plays a major role, so you have to test with a variety of data sets.
> 
> As a developer, I certainly want all queries to be as fast as
> possible, but this requires a large number of slightly varying tests.
> I thus think the current benchmark test suite is still much to small.
> It only covers the basic indexes and queries, and only 3 data sets. I
> would certainly welcome some additions.
> 
> I also think the performance tests should be better integrated. It
> would be nice if we had a web page (similar to the one for the XQTS)
> where I could just select the test sets I want to run, execute them
> and get back the results.
> 
> Wolfgang
> 
> ------------------------------------------------------------------------------
> Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day 
> trial. Simplify your report design, integration and deployment - and focus on 
> what you do best, core application coding. Discover what's new with
> Crystal Reports now.  http://p.sf.net/sfu/bobj-july
> _______________________________________________
> Exist-development mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-development