While implementing my simulator RoadRunner I wanted to be able to compare it using a wider range of parameters:
What I came up with is this application:
While it is primarily used on windows, it will work fine under Linux and OS X thanks to the Mono framework.
The runner can be used with any Test Runner that was wrapped to be called from the command line. If your simulator was wrapped for a previous version of the test suite, you can use the same wrapper.
Alternatively you could simply specify a directory that contains the results generated by your simulator. For more information see: [Setting up the Simulator].