The content that is up at the LBS web page seems reasonable as a first cut.
There are some refinements to make (e. g. clarifiying the publication issues wrt Volanomark, etc)
However, at the moment, all I would suggest is that the following links
http://directory.netscape.com/Computers/Performance_and_Capacity and also
would also be useful links for the LBS web page.
A more critical problem and the other thing that appears to be missing is a mission statement as to what this is about. A casual
inspection of the Source Forge site found the following other projects related to benchmarking (I've eliminated a few that are
obviously of no interest to the Linux scalability effort (LSE on Source Forge)). Also note that TIOBENCH did not show up in the
search, so it is quite possible there are bunch of other projects related to benchmarking that didn't show either):
-------------start list of Source Forge Benchmarks------------
Hammerhead is a web site coverage and stress testing tool. It's also
useful for benchmarking your web servers. It's been designed to emulate
multiple users from multiple IP addresses and designed for maximum speed
(not like it's interpreted counterparts).
lmbench is a suite of simple, portable, ANSI/C microbenchmarks for
UNIX/POSIX. In general, it measures two key features: latency and
bandwidth. lmbench is intended to give system developers insight into
basic costs of key operations.
openbench is a simple crossplattform processor benchmark made to ease the
comparison of cpu power across different architectures Linux Benchmark
Suite Develop benchmarks targeted toward Linux system developers.
ejbSPEC is a project that will provide a suite of benchmark programs
that are intended to test an EJB container's performance, integrity,
and compliance. Threaded I/O tester Portable, robust, fully-threaded
I/O benchmark program Postal/Rabid mail server benchmarks Program to
benchmark high performance mail servers.
VA Diagnostics & Reporting
(VADR) The VA Diagnostics & Reporting (pronounced 'va-der') suite is a set
of tools and standardized test suites for a variety of services. The first
suite to be release will be vadr-web, an http benchmark tool and suite.
Siege is an http regression testing and benchmarking utility. It was
designed to let web developers measure the performance of their code
Open Source Database Benchmark
The OSDB project is defining a database-independent, system-independent
benchmark to enable individuals to analyze the performance of a variety
of database and system configurations.
-----------------end list of Source Forge Benchmarks---------------------
So this begs the obvious question, what makes LBS unique among all of the above and what contribution do we think we are making by
putting effort into it? (Hence, the mission statement. What ground do we want to mark off here as the area we are trying to
contribute in?). And the home web page does not even address how we are different from the other benchmarking projects that are
listed on our home web page (e. g. the Linux Benchmarking Project). I wonder if we are doing more damage than good by adding to
this confusion by creating yet another benchmarking site that has no clear agenda or role? So, who wants to work on the mission
Anyway, one other comment on the LBS home page. One thing that we have identified that needs to be done wrt performance tools
(and about the only thing for LTC Kernel Performance to do) is to prepare a list of open source performance tools, description,
status, etc and make that available inside of the LTC. I have been nominated to do that task. Obviously, we would also like to
make it available outside the LTC. Perhaps the section on performance tools on the LBS page would be the appropriate place to put
this. What do you think?
One of the hard problems in starting a project like LBS is getting a core set of people involved who care about it. Given that
there are plausible alternatives (e. g. Linux Benchmarking Project) one thing I would like to understand is "Why not just join
them rather than starting a new project?
IBM Linux Technology Center
We are Linux. Resistance is an indication that you missed the point.
"...the Right Thing is more important than the amount of flamage you need
to go through to get there"
--Eric S. Raymond