#1 Auto-repro for mplayer

[David Molnar]

Create script to periodically update mplayer to latest subversion, then re-test each test case uploaded. For each test case, three outcomes:

1 - test case repro-tested, still has bug
2 - test case not yet repro-tested
3 - test case repro-tested, bug disappeared

Here "bug" means comparing on bug type only, not on stack hash (since stack hash changes).

Fix Metafuzz index to report test case outcomes, most recent version of mplayer tried, with color codes. Green - repro'd, yellow - not yet tested, red - bug disappeared.

[David Molnar]

Logged In: YES
user_id=896370
Originator: YES

Have autorepro script running on koschei.cs.berkeley.edu . Will eventually check into SVN. Unfortunately three main issues have popped up:

1) Some of the repro files take a long time to play (or hang). Repro script needs a timeout.
2) A single repro machine can't keep up with the stream of new test cases uploaded to metafuzz, even though there's not that many of them (maybe 100/day).
3) By the time koschei.cs.berkeley.edu gets through a chunk of test cases, mplayer has updated again.

Probably the way to go forward is come up with a way to do repro testing on the cluster or on EC2. I don't have time to do that right now. If someone else wants to pick it up, drop me a line.