[Firebird-test] Engine profiling and performance testing

[Leyne, S. <sl...@at...>]

All,

Mike N's conversation regarding FB2 threads/design/etc. is great!

I realized, however, that we are all (myself included) starting to
suggest solutions to a problem(S) which is not clearly defined.=20

Namely, what are *the* bottlenecks in the current engine design?

I'm not trying to suggest that the ideas put forward by Mike (or anybody
else) are not good or won't be ultimately adopted.  The problem,
however, is that we are 'guessing' at the source of the real problems.

So the question is then, how can we profile the engine in order to
determine where the performance 'sink holes' are?

Related to the issue of profiling is performance testing/baselines.

Also, I think we need to establish some baseline values for the engine,
which can then be used to judge the impact of any change made to the
engine/code. =20

We can't be working on 'gut' senses of performance -- look at the
ongoing debate about cache size > 10000 pages.  We need imperical data
to backup our assertions (FB vX is 100% faster than IB 7.0 -- work with
me here :-] )

I know that Tord Hammer has the TPC-R tests to running with the engine.


There are, however, a couple of issues:

- The TPC-R tests, are more "single user" tests unlike the TPC-C which
are multi-user.

- The TPC-R tests currently don't run on Windows.

So, while we can use these tests to benchmark such things as the
performance impact of page size and/or cache size, they aren't can't
mimic a "real life"/multi-user environment.

So the question the here is, what can we do to get some 'multi-user'
tests?


I'd be willing to dedicate a server box to allow for the testing to be
performed and a baseline established, and to keep this box unchanged (a
known/stable configuration) for as long as we need.  I also have a
number of Windows desktops which I could use for the necessary
"clients".

Finally, who would be interested in working on these problems?


Sean