A command-line benchmarking tool. Statistical analysis across multiple runs. Support for arbitrary shell commands. Constant feedback about the benchmark progress and current estimates. Warmup runs can be executed before the actual benchmark. Cache-clearing commands can be set up before each timing run. Statistical outlier detection to detect interference from other programs and caching effects.
The Phoronix Test Suite is the most comprehensive testing and benchmarking platform available that provides an extensible framework for which new tests can be easily added. The Phoronix Test Suite is designed to effectively carry out both benchmarks in a clean, reproducible, and easy-to-use manner.
GL O.B.S. is based around a PyGTK interface that launches OpenGL programs feeding them with a common CLI options set which will affect their behaviour, then receives back their statistical output. Benchmarks are easy to create and add.
Free Java monitoring library to easily instrument apps to measure performance detail over time with minimal overhead. Has dashboard, SVG graphs. Extends Jamon and its statistical data, same usage. http://www.e-peas.com/opensla/. JMX, SNMP planned.