tcljava-dev

[tcljava-dev] Performance results for new TJC compiler

[Mo D. <md...@un...>]

Hello all

I have made the claim that the new TJC compiler for Jacl will improve runtime
performance, but until now I have been lacking real numbers to back up that
claim. I have now imported the tclbench framework into the tcljava/tests/tclbench
directory in CVS and modified the module so that it can be used to test Jacl.
This tclbench module will test just the execution time for a command, it ignores
shell startup and shutdown time and will take the hotspot compiler into account.

Proper benchmarking can be very tricky, but I think the following results
will stand the test of time.

http://www.modejong.com/tcljava/tjc_results_2_22_2006/jacl_tjc_results.html

These results compare Jacl 1.3.2 vs the current CVS version of Jacl with TJC
compiled commands.

cheers
Mo DeJong

Re: [tcljava-dev] Performance results for new TJC compiler

[Mo D. <md...@un...>]

Hello

I decided to try to get some "real world" numbers related to the TJC compiler.
What I did was to test how long the TJC compiler took to compile itself
and then compared that with how long a TJC compiled version of itself
took to compile itself. Just to make things even more interesting, I compared
those times to the amount of time it took to execute the TJC compiler in
the C version of Tcl which has an embedded compiler and byte code engine.
Here are the results.


C Tcl (Tcl byte code compiler)

$ time tjc -nocompile tjc.tjc
real    5m30s     (330 s)


Interpreted in Jacl (no compilation)

$ time tjc -nocompile tjc.tjc
real    21m17s    (1277 s)


TJC compiled in Jacl:

$ time tjc -nocompile tjc.tjc
real    2m42s     (162 s)


These results show that compiling with TJC lead to code
that executed 7.8 times faster. That is quite an improvement.
The TJC compiled Jacl version of this code even executes
about twice as fast as the same code running in the C version
of Tcl.

cheers
Mo DeJong

P.S.
I added support to the Makefile for a 2 stage build rule that
will recompile the TJC source with TJC. Just build the
normal way and then run "make install". After that, run
"make tjc2 tjc2.replace tjc.install" to 2 stage the compiler
and install the compiled versions of tjc.jar and tjcsrc.jar.
The numbers above show that this 2 stage will take
about 20 minutes to run the first time, but the compiler
will then execute about 8 times faster.

Re: [tcljava-dev] Performance results for new TJC compiler

[Tom P. <tpo...@ny...>]

On Thu, Mar 02, 2006 at 12:55:45AM -0800, Mo DeJong wrote:

> These results show that compiling with TJC lead to code
> that executed 7.8 times faster. That is quite an improvement.
> The TJC compiled Jacl version of this code even executes
> about twice as fast as the same code running in the C version
> of Tcl.

Very nice, Mo.  


-- 
Tom Poindexter
tpo...@ny...

Error during build: Re: [tcljava-dev] Performance results for new TJC compiler

[Tom P. <tpo...@ny...>]

Mo, I got this error while trying to compile tjc:

$ make tjc2
compile TJC Tcl source files
tjc tjc.tjc
error: cannot read: /home/tpoindex/src/java/tcljava.cvs/build/tjc2/bldTJC/source/tjc/*.java
1 error

real    9m39.976s
user    8m8.844s
sys     0m5.013s
make[1]: *** [tjc2/tjc.jar] Error 255
make: *** [tjc2] Error 2



I'm current on CVS, and ran:

cvs -q up -dP
./configure --enable-jacl --disable-tclblend
make
make tjc2


My Java is:

$ java -version
java version "1.5.0_06"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_06-b05)
Java HotSpot(TM) 64-Bit Server VM (build 1.5.0_06-b05, mixed mode)

and machine:

$ uname -a
Linux xxxxxxxx 2.6.12-10-amd64-generic #1 Mon Feb 13 12:14:05 UTC 2006 x86_64 GNU/Linux


Jacl ('make install' after above build) and TJC work otherwise:

$ jaclsh
% package require TJC
1.0
% proc foo {args} {puts "foo here"}
% info body foo
puts "foo here"
% foo
foo here
% TJC::compile foo
% after 10000
% info body foo
"foo" isn't a procedure
% foo
foo here


Small request:  how about a '-wait' option for TJC::compile that waits 
until compilation is completed, throwing a Tcl error on compile error?


-- 
Tom Poindexter
tpo...@ny...

[tcljava-dev] Re: Error during build

[Mo D. <md...@un...>]

On Thu, 2 Mar 2006 10:37:50 -0700
Tom Poindexter <tpo...@ny...> wrote:

> Mo, I got this error while trying to compile tjc:
> 
> $ make tjc2
> compile TJC Tcl source files
> tjc tjc.tjc
> error: cannot read: /home/tpoindex/src/java/tcljava.cvs/build/tjc2/bldTJC/source/tjc/*.java
> 1 error

Well, the command line to javac is something like:

javac source/tjc/*.java

It does that instead of passing a huge number of filenames on the command line
because the exec blows up under Windows when the command line is too long.
I wonder if the Unix version of javac can't accept a file name pattern like the
Windows version does.

You could try to disable that pattern logic in jdk.tcl and see if that fixes it.

> I'm current on CVS, and ran:
> 
> cvs -q up -dP
> ./configure --enable-jacl --disable-tclblend
> make
> make tjc2

You don't need to pass --disable-tclblend there since --enable-jacl
already does that implicitly. Also, you need to install before
running the tjc2 target.

make
make install
make tjc2 


> Jacl ('make install' after above build) and TJC work otherwise:
> 
> $ jaclsh
> % package require TJC
> 1.0
> % proc foo {args} {puts "foo here"}
> % info body foo
> puts "foo here"
> % foo
> foo here
> % TJC::compile foo
> % after 10000
> % info body foo
> "foo" isn't a procedure
> % foo
> foo here
> 
> 
> Small request:  how about a '-wait' option for TJC::compile that waits 
> until compilation is completed, throwing a Tcl error on compile error?

Notification is already implemented:

TJC::compile foo -readyvar rvar
vwait rvar
puts stderr "rvar result is \{$rvar\}"

cheers
Mo DeJong

Re: [tcljava-dev] Re: Error during build (FIX)

[Tom P. <tpo...@ny...>]

On Thu, Mar 02, 2006 at 11:05:50AM -0800, Mo DeJong wrote:
> On Thu, 2 Mar 2006 10:37:50 -0700
> Tom Poindexter <tpo...@ny...> wrote:
> 
> > Mo, I got this error while trying to compile tjc:
> > 
> > $ make tjc2
> > compile TJC Tcl source files
> > tjc tjc.tjc
> > error: cannot read: /home/tpoindex/src/java/tcljava.cvs/build/tjc2/bldTJC/source/tjc/*.java
> > 1 error
> 
> Well, the command line to javac is something like:
> 
> javac source/tjc/*.java
> 
> It does that instead of passing a huge number of filenames on the command line
> because the exec blows up under Windows when the command line is too long.
> I wonder if the Unix version of javac can't accept a file name pattern like the
> Windows version does.
> 
> You could try to disable that pattern logic in jdk.tcl and see if that fixes it.



See attached jdk.tcl.diff and jdk.tcl  (to patch/replace
./src/tjc/tjc/library/jdk.tcl)

The fix glob's files to expand wildcard patterns,
writes the resulting list of files to another file,
and uses javac's '@filename' to specify the file
containing the list of files.  

Also, I had to increase the memory for javac to
prevent out of memory errors during 'make tjc2'.


> > Small request:  how about a '-wait' option for TJC::compile that waits 
> > until compilation is completed, throwing a Tcl error on compile error?
> 
> Notification is already implemented:

A shortcut would still be nice to have,  perhaps:

proc TJC::proc {name arglist body} {
    uplevel #0 [list ::proc $name $arglist $body]
    set ::TJC::__DONE__ ""
    TJC::compile $name -readyvar ::TJC::__DONE__
    vwait ::TJC::__DONE__
}


-- 
Tom Poindexter
tpo...@ny...

Re: [tcljava-dev] Re: Error during build (FIX)

[Mo D. <md...@un...>]

On Tue, 7 Mar 2006 16:53:45 -0700
Tom Poindexter <tpo...@ny...> wrote:

> > > Mo, I got this error while trying to compile tjc:
> > > 
> > > $ make tjc2
> > > compile TJC Tcl source files
> > > tjc tjc.tjc
> > > error: cannot read: /home/tpoindex/src/java/tcljava.cvs/build/tjc2/bldTJC/source/tjc/*.java
> > > 1 error

...

> See attached jdk.tcl.diff and jdk.tcl  (to patch/replace
> ./src/tjc/tjc/library/jdk.tcl)
> 
> The fix glob's files to expand wildcard patterns,
> writes the resulting list of files to another file,
> and uses javac's '@filename' to specify the file
> containing the list of files.  
> 
> Also, I had to increase the memory for javac to
> prevent out of memory errors during 'make tjc2'.

This is a fine patch, but I think this javac function is getting too messy.
There is no real reason to support wildcard expansion and using the
@file thing will keep it from working with another compiler. What
I did was reimplement the [exec $JAVAC ...] bit so that files are
processed in batches of 40. This should fix the compile under Unix
while also avoiding the compiler out of memory problem.

The fix is checked into CVS as of today, please let me know if you
run into any more problems.

Mo DeJong

Re: [tcljava-dev] Re: Error during build (FIX)

[Tom P. <tpo...@ny...>]

On Wed, Mar 08, 2006 at 02:30:08PM -0800, Mo DeJong wrote:
> @file thing will keep it from working with another compiler. What
> I did was reimplement the [exec $JAVAC ...] bit so that files are
> processed in batches of 40. This should fix the compile under Unix
> while also avoiding the compiler out of memory problem.
> 
> The fix is checked into CVS as of today, please let me know if you
> run into any more problems.

Yep, latest CVS sources are now making tjc2 just fine, and no stack problems
during compile.

-- 
Tom Poindexter
tpo...@ny...

Re: [tcljava-dev] Performance results for new TJC compiler

[Mo D. <md...@un...>]

Hello all

I have recently implemented the "compiled locals" feature from the
C version of Tcl in the TJC compiler. This feature makes use
of an array of variable objects known at compile time instead of
using a hashtable to store the variable objects. This means that
a TJC compiled proc will now consume less memory and will execute
more quickly in many cases. Here are the highlights showing
performance improvements found when comparing TJC compiled
code before and after the "compiled locals" enhancement. These
times are all in ms, they come from the tclbench test
suite found in tcljava/tests/tclbench in the CVS.

Tcl Bench test name			Pre-locals	W-locals
invoke_1_to_10				3585		1873
invoke_1_to_10_loop			51770		18320
invoke_empty				42		20
bench_set_different_scalars		26		10
bench_set_same_scalar			10		6
bench_set_global_import			18		12
bench_set_array_variable_keys		70		28
bench_set_array_and_scalar_names	52		38
bench_incr_array_constant_50		100		50
data-create-array			5668		1652

These results show that the "compiled locals" feature has
resulted in a significant performance improvement for many
common cases.

cheers
Mo DeJong