on 2012-05-01 we were talking on IRC about changing the test runner so
that it uses different random numbers each time (to "explore more of the
space", quoting Christophe), provided there is an easy way to repeat a
run with the same numbers.
I have built a preliminary something to this effect, using Nikodemus's
suggestion to generate a random state from a random integer seed, and
uploaded it to github:
There are two commits, the first some cleanups, the second the new
functionality. I'd appreciate feedback especially on whether this does
what you'd like it to do and on the user interface.
It works with pure and impure tests. I have not yet looked into passing
the random seed/state to shell scripts called by the test runner, too,
but plan to do that.
To test it, just build a pure or impure test file with contents
(with-test (:name :random)
(assert (zerop (random 2))))
and run it a few times. See the comment at the top of run-tests.sh for
how to pass a random seed into the script.