[Jscheme-devel] simple 1.3.1/1.4.0 benchmarking

[Ken A. <kan...@bb...>]

I've read that JDK 1.4 invoking a method through reflection is much faster 
than it was.  I also had never used Tim's compiler.  So, i thought i'd try 
it by writing a simple performance benchmark.

The bottom line is JDK 1.4.0 and jsint.Compile are a win, seperately or 
together.

Here's the benchmark:

(package performance)
(import "java.lang.Math")
(import "java.lang.System")
(define (f x) (if (= x 0) 0 (f (- x 1))))
(define (g x) (if (= x 0) 0 (g (abs (- x 1)))))
(define (h x) (if (= x 0) 0 (h (Math.abs (- x 1)))))

; (define time
;   (macro (exp . rest) `(first (second (time-call (lambda () ,exp) . 
,rest)))))

(define (minimum xs)
   (foldL (cdr xs) (lambda (x so-far) (if (< x so-far) x so-far)) (car xs)))

(define (times name f n sofar)
   (if (= n 0) (minimum sofar)
       (let ((result (first (second (time-call (lambda () (f 1000000)) 1))))
	    (version (System.getProperty "java.version")))
	(display {[version],[name],[result]\n})
	(times name f (- n 1) (cons result sofar)))))

(define (main args)
   (let* ((f (times 'f f 5 '()))
	 (g (times 'g g 5 '()))
	 (h (times 'h h 5 '()))
	 (abs (- g f))
	 (Mabs (- h f)))
     (display
      {[(System.getProperty "java.version")],[f],[g],[h],[abs],[Mabs]\n})
     ))

The output it produced on my machine is (header added):
compiled,JVM,f,g,h,abs,Mabs
no,1.3.1,6770,8202,19399,1432,12629
yes,1.3.1,4887,6079,6179,1192,1292
no,1.4.0,4206,5468,10876,1262,6670
yes,1.4.0,2474,3275,3364,801,890

THere are 3 functions, f, g, and h.  (f) is the simplest loop you can write 
in Jscheme.  (g) adds a call to the primitive (abs) and h adds a generic 
function call to (Math.abs).

I've enclosed a spreadsheet with this information and some additional 
computations.

There is a performance difference in the interpreter between primitive 
procedures, and generic methods which is 8.8 in JDK 1.3.1, but only 5.2 in 
JDK 1.4.0.  The compiler turns generic functions into Procedures, so the 
times are similar when compiled.

So, going to JDK 1.4 is definitely a good thing, and compiling your jscheme 
is also a good thing.

Also, reasonably fast scheme could be written in Jscheme and then compiled.

k