#100 Eclipse's use of rt.jar makes the workload jvm-specific.

[Steve Blackburn]

From Jon Bell:

What I meant was that it seems to me that the amount of work that’s done in the benchmark shouldn’t scale with the JVM that is being evaluated - it should remain constant regardless which JVM is being used to evaluate it, and the time needed to complete the benchmark should only vary with the JVM’s implementation of the optimizer, compiler, gc, class files, etc. However, if I have a JVM that has a ton of extra classes included in rt.jar, I don’t think that it should make the dacapo eclipse benchmark run slower - but this is the case, because the benchmark runs its tasks with regard to the JVM used to start it (index, search, etc.).

I think that someone else tried to think this one through - there is already a flag “-Declipse.java.home” that can be used to specify the JVM that the eclipse benchmark should use for compiling code against (but this doesn’t also apply to searching and indexing — the result is that both the JVM specified with -Declipse.java.home and the boot jvm are put on the class path internally for the projects that eclipse is compiling/searching/indexing. In my cases, I had been specifying a reference JVM, e.g. when comparing hotspot 7 and hotspot 7 w/ class file tweaks, I would specify the baseline hotspot 7 JVM with the -Declipse.java.home property, and the end result was that the benchmark took ~2x as long as it should have on the tweaked version — because it resulted in the benchmark indexing and searching over twice as large of a codebase (that is, searching and indexing both JVMs).

(from the 2007 commit message introducing the flag: “-

The 'eclipse' benchmark requires a regular, Sun-like jre
in order to be able to perform its build tasks (it needs to be able to compile against standard libraries such as 'rt.jar'). Some JVMs do not comply with eclipse's expectations. A simple work-around is to specify a jre for eclipse to compile against. If the
jvm used to run the benchmarks does not comply with eclipse's expectations, an error message will be produced with an explanation of the suggested work around (setting -Declipse.java.home=xxx on the command line to point to an orthodox jre, whose libraries
will be used). [Fixed in 2006-10-RC1]

”)

[Jonathan Bell]

In case someone else comes upon this and is in dire need of making it work, I wanted to post this here...

I made a javaagent that rewrites the relevant parts of the eclipse code that look up the boot path. https://github.com/jon-bell/dacapo-eclipse-hacker