if you get errors saying "Unsupported major.minor version 49.0", you are probably not using Java 1.5; but the binary release was compiled using Java 1.5.
1) Upgrade to java 1.5;
2) get the sources from the CVS repository, and compile them yourself. Follow the CVS link on the project page for instructions.
I have released the rating test tool that I developed during my PhD as open source. You can use it to design and carry out your own listening tests, by simply adapting a configuration file -- normally, no source code editing should be necessary.
Get the binary release here: