CÚsar <divcesar@gmail.com> wrote on 11/14/2012 05:12:34 AM:
>
> Why multiple JIT threads is not the default?

>

I think mainly because when the recompilation system was being designed (circa 2000), cores were much scarcer than they are today.  Underlying the basic cost/benefit model used by the adaptive optimization system is the assumption that the CPUs are being fully utilized by the application. Therefore one must balance the expected benefit of recompiling a method at higher optimization level with the cost of taking away CPU cycles from the application while doing so.

Some of the obvious downsides of having a single JIT thread are at least partially mitigated by using a priority queue as the communication mechanism between the controller and the compilation thread. Thus, when there is a backlog of methods to recompile, the compilation thread is able to focus its attention on the recompilations that are expected to be most profitable (not the ones requested first).

Note that initial compilations (by the baseline compiler in a typical configuration) are executed on the application thread that is attempting to invoke the method, not on the compilation thread.  So they can proceed in parallel with each other (and with recompilation) in a multi-threaded application.

Also since, Jikes RVM doesn't use interpretation for initial execution the performance gap between the initial (baseline compiled) and optimized recompiled code is much smaller (we expect something between 4x and 6x according to the CompilerDNA).  In a system with initial interpretation the opportunity cost of delaying JIT compilation can be much more significant because the expected gain is typically more like 10x to 20x.

All that being said, it is certainly possible (and I think I have seen some published papers that looked at this for Jikes RVM) that on modern multi-core hardware using multiple JIT threads could result in a small improvement in start up performance by reducing the delay in JIT compilation.

I took a quick look through the code, and doing this isn't as simple as just creating multiple compilation threads in the controller.  One would also have to restructure the protocols/data structures in RuntimeCompiler (several of the key methods are synchronized).  

--dave