Cbench-sf / News: Recent posts

The Cbench website, actually the read-only mirror of the live Cbench TRAC website, is migrating from the 'cbench-sf' Sourceforge project to the 'cbench' Sourceforge project. As of Friday 03-07-2008.

http://cbench.sourceforge.net and http://cbench.org are the correct URLs to access the homepage now.

This release is the final major release in the Cbench 1.1.X series. Any other releases will be bugfix only. All new Cbench development is in the head of the repository and is towards Cbench 1.2.

* Cbench release (1.1.5)
* Added the --existencecheck mode to nodehwtest_output_parse.pl
which is sort of a light version of checking node(s) parsed test
data against characterized data. In the normal mode, the parsed
data is compared against the characteristic data for deviation.
In this new mode, the parsed data is compared against
the characteristic for existence, i.e. to make sure the
test seems to have run on the node and generated data.
* udpated the little used --nodediag feature of output_parse_generic.pl
to find the nodelist a job ran on if it was in the embedded Cbench
info tags
* start grabbing the nodelist from Torque inside batch jobs
* fix and update output_parse modules that use the CBENCH
NOTICE capability
* the NPB ft.D.* mpi tests take wayyyyy to long if there isn't
enough memory on the nodes, so added a basic runtime check
to only run ft.D.* binaries on nodes with at least 6GB
* bypass lu.D on node-level testing since it seems to behave badly
* some attempts to get NPB to compile with fewer errors
* add --random option to allow random ordering of test
modules in node_hw_test
* fixed a bug that for some reason caused the target_values
file to be read wrong and threw off all the statistical
comparisons in nodehwtest_output_parse.pl
* added the prototype 'shakedown' testset using a new gen_jobs
scripting structure (install with sbin/install_shakedown)
* update Linpack and HPCC makefiles so that the HP_dlamch.c
file is compiled properly with the NOOPT_FLAGS from make.def
(CCNOOPT in the makes) to avoid optimzation related failures
(this was the desire of the hpcc and linpack makefiles, it
was just not quite right)
* in compute_N(), round up when calculating the number of nodes
* Marcus added some sorting so that job generation happens in a repeatable,
logical order
* make 1 processor the default starting size for generating and starting
jobs
* several tools/gen_jobs_generic.pl optimizations and cleanups from Marcus
* several more custom parsing error filters
* update tools/nodehwtest_output_parse.pl to be more consistent with
tools/output_parse_generic.pl
* assume openmpi 1.2.x is being used now when dealing with openmpi
* fix a bug with the HPL/HPCC N parameter calulation when using > 1ppn,
thanks to Jim Schutt
* make the default hpcc input deck match that of linpack
* Marcus Epperson added the mpiGraph benchmark, by Adam Moody of LLNL,
http://sourceforge.net/projects/mpigraph. The crunch_mpiGraph utility
can be used to generate an HTML report from mpiGraph run output.
* added OSU messsage rate benchmark
* changed the IO testset to be a stress test of the
parallel filesystem
* adding the IOSANITY to be used as a sanity checker
of parallel filesystems, but not necessarily stressful
* Build some openmp versions of streams if we are using Intel compilers
* honor RPATH and BLASLIB set in the environment if
found which allows one to recompile with a different
BLAS linkage w/o doing any editing
* some under the covers openmpi 1.2.X specific support
* added the --waitall option for --throttledbatch mode which will keep
start_jobs from exiting until all jobs have exited the batch system,
thanks to Jim Schutt for suggesting this
* add binary identifier support (--binident parameter) which analogous to
test identifiers but for organizing multiple sets binaries in a Cbench
testing tree
* update the --ident parameter to accept comma separated lists
* fixed a bug with --repeat when combined with with --throttledbatch
or --serialbatch modes of start_jobs*
* added a --polldelay option for --throttledbatch/--serialbatch mode
* added more variations to compile streams
* more improvements to the Cbench testing tree rpm capabilities
* Adding initial (but working) support for 'yod' job launcher and 'pbspro'
batch system
* Marcus Epperson added the very nice HPCC debug build which
allows one to select on the command line which hpcc tests to run
* change the running as root/non-root logic to be smarter in node-level
testing and not depend on privelege escelation

Cbench received an honorary mention in an HPCwire article about Woven Systems's 10 Gigabit Ethernet (GE) fabric solution. The articles can be found at http://www.hpcwire.com/hpc/1711839.html and http://www.linuxhpc.org/stories.php?story=07/08/07/3708989.

The changes post version 1.1.3 reached a critical mass after about 9 months and it was time to cut a new release.
There are a lot changes all over the board:
o upgraded NPB support to use version 3.2.1
o added PSNAP OS noise measurement code from LANL
o added STREAM2 benchmark
o added the 'stress' UNIX OS system stress testing utility and
included it in the standard NODEHWTEST node-level testing
o added NPB serial testing to NODEHWTEST test set
o added colorized text output to several key Cbench utilities
o switched Cbench self-contained (standalone) MPICH to use ch_shmem
o reverted to MPICH 1.2.6 for Cbench self-contained (standalone) MPI
o output_parse scripts --ident parameter accecpts comma separated test
identifier lists now
o output_parse scripts support --minprocs and --maxprocs params
o workaround in output_parse_generic.pl for Perl<->Glibc memory allocation
weirdness with large raw output files
o the Cbench test tree can now be used completely standalone:
- binaries are installed into a single 'bin' directory instead of scattering
the test binaries throughout the tree
- the 'tools' directory is installed into the test tree
- Cbench standalone MPI is installed into the test tree if built
o added tools/make_skel_testset.pl utility script to allow easy creation of
a test set skeleton with appropriate generic utility scripts
o upgraded IOR to version 2.9.0
o upgraded iozone to version 3_263
o added --maxmem parameter to node_hw_test script for overriding the free
memory detection algorithm
o added the capability to the NODEHWTEST test set to deal with hw_test modules
that require root privelges to run correctly (like memtester and iozone)
o added a new mode for starting jobs called --throttledbatch; it allows you to
control the number of simultaneous running+queued jobs Cbench will allow in
the batch system at once
o added the capability to plot statistical distributions (aka dplot) of data
using the 'dplot' utility from Egan Ford's STAB toolkit
o fixes to mpi_overhead.c to handle negative memory dececrease cases (i.e. memory
increase after MPI_init)
o several new capabilities added to output_parse_generic.pl such as: PNG file
output, persistent gnuplot windows, data normalization, additional plotlines
specified on the command line, range control, collapsing of multiple test
identifier data into a single set of data, key placement control...
o added a handly script tools/runsizes_list.pl that allows you to build comma-
separated lists of runsizes that you can use with the --runsizes parameter
of gen_jobs to generate custom runsize sets
o catch HPLinpack allocation failures and flag them during output parsing
o removed TODO file from source distribution (the TODO list is now on the Cbench
TRAC website)
o added doc/ListOfSoftware as a new file mirrored from the Cbench wiki
o Changed the implementation of the custom parse filters to be multple Perl module
files specified by the @parse_filter_include array in cluster.def; modules can
easily be excluded or included by commenting out/in
o added OSU mpi benchmarks to the BANDWIDTH test set
o added an MPI message rate data point to the BANDWIDTH test set
o added support for running HPlinpack as a node-level test (i.e. added an xhpl.pm
hw_test module)
o added --statsmode to output_parse_generic.pl to dump out a more statistically
oriented tabular output
o Added the --grepable option to output_parse_generic.pl to output gathered data
in a grep friendly fashion
o rename data metrics generated by the hpcc.pm output_parse module to be more
consistent with what the HPCC website names them
o make CBENCHTEST environment variable a requirement; if it's not defined and you do
a "make itests" as root, your /bin directory will be forcibly removed...this is not
good!
o undo several implicit path assumptions
o added the 'nodehwtest' top-level Makefile target which takes care of building
everything needed for node-level testing (i.e. the nodehwtest test set) and
nothing more (e.g. no MPI builds occur)
o added the output_parse_customparse.pl script to allow usage of the custom parse
filters without having to use the full output parsing framework
o added the rather nifty single_node_benchmark.pl script which uses various
benchmarks and capabilities in Cbench for the singular purpose of benchmarking
a single node and generating a nice PDF report of the data
o better support for the new realms of running greater than 2 ppn (processes per node)
and for the new multi-core landscape
o started adding support for 'binary identifiers' which are very similar in
concept to test identifiers but used for cleanly and dynamically managing multiple
sets of test binaries
o several minor changes and under-the-cover capabilities to support the new Fixit123
Hardware Qualification/Requalification Framework
o clean up some minor compile annoyances in various sources
o many other small bugfixes and tweaks....... read more

About 6 months worth of updates and bugfixes are in 1.1.3. Some very nice improvements especially in the area of the output parsing capabilities.

See the Changelog in the Files section for more details.