tcljava-dev — discuss implementation and post patches here

[tcljava-dev] Performance results for new TJC compiler

[Mo D. <md...@un...>]

Hello all

I have made the claim that the new TJC compiler for Jacl will improve runtime
performance, but until now I have been lacking real numbers to back up that
claim. I have now imported the tclbench framework into the tcljava/tests/tclbench
directory in CVS and modified the module so that it can be used to test Jacl.
This tclbench module will test just the execution time for a command, it ignores
shell startup and shutdown time and will take the hotspot compiler into account.

Proper benchmarking can be very tricky, but I think the following results
will stand the test of time.

http://www.modejong.com/tcljava/tjc_results_2_22_2006/jacl_tjc_results.html

These results compare Jacl 1.3.2 vs the current CVS version of Jacl with TJC
compiled commands.

cheers
Mo DeJong

[tcljava-dev] Re: tjc runtime compiler

[Martti T. <ma...@da...>]

> From: Mo DeJong <md...@un...>
> 
 > The attached patch should fix these issues. The janino.jar file should
 > not actually appear on the CLASSPATH as it is loaded via the
 > TclClassLoader off of the env(TCL_CLASSPATH). Trouble was, the code
 > was >not looking in the right place. The attached patch should fix
 > that too.

Thanks Mo. The patch fixed up all the reported problems. I am now able 
to compile jacl commands on the fly in a server app that we are 
developing. The server runs a jacl interpreter for every client that 
logs in and each interpreter sources the individual jacl commands via 
the "unknown" command whenever they are required. Compiling each command 
as required inside the "unknown" command seems to work well. Because the 
"unknown" command also eval's the requested command immediately after 
starting compilation the first eval uses source interpretation. After a 
few seconds when the command is accessed again it executes up to 10 
times faster with some crude "time script" measurements. Since we are 
using 100's of jacl commands to provide user modifiable functionality it 
is fabulous that we can have them compiled, unloaded and recompiled 
dynamically without restarting the server.

The only minor problem so far is that the "-readyvar" option on 
TJC::compile never returns a result in the server. It does return 
correctly in jaclsh. We are running jacl in a beanshell environment and 
the interpreters are started by the beanshell manager. We also have some 
difficulty getting jacl exceptions back via the beanshell environment so 
we will investigate this area.

Cheers
-- 
Martti Tienhaara (ma...@da...)
DASH Software Ltd.

Re: [tcljava-dev] Re: tjc runtime compiler

[Mo D. <md...@un...>]

On Wed, 15 Feb 2006 09:59:30 -0800
Martti Tienhaara <ma...@da...> wrote:

> > From: Mo DeJong <md...@un...>
> 
> > A new runtime Tcl proc compiler API has been added to the CVS version of Jacl.
> > This runtime compiler will take a Tcl procedure as input and generate Java
> > byte code that will work the same way as the original Tcl proc. Previously,
> > the TJC compiler only worked as a separate ahead of time compiler.
> > 
> Great work.

Thanks for testing Martti

> I have come up with a few minor problems running on a Linux system. I 
> carried out the configure-make-install steps.
> 
> - jaclsh installed in /usr/local/bin requires the MINGW check to be 
> commented out. Also the JACL_CLASSPATH doesn't include the java compiler 
> janino.jar.
> 
> - jdk.cfg installed in /usr/local/bin uses ":" the linux path separator 
> but only works if you change it to use the Windows path separator ";". 
> Again janino.jar is missing.
>
> - running the hello world example doesn't work in jaclsh unless you 
> create a link to the janino.jar file in the current working directory
> even after adding it to all the CLASSPATH's. It is installed in 
> /usr/local/lib/tcljava1.3.2 by make install.

The attached patch should fix these issues. The janino.jar file should
not actually appear on the CLASSPATH as it is loaded via the TclClassLoader
off of the env(TCL_CLASSPATH). Trouble was, the code was not looking
in the right place. The attached patch should fix that too.

Mo DeJong

[tcljava-dev] Re: tjc runtime compiler

[Martti T. <ma...@da...>]

> From: Mo DeJong <md...@un...>

> A new runtime Tcl proc compiler API has been added to the CVS version of Jacl.
> This runtime compiler will take a Tcl procedure as input and generate Java
> byte code that will work the same way as the original Tcl proc. Previously,
> the TJC compiler only worked as a separate ahead of time compiler.
> 
Great work.

I have come up with a few minor problems running on a Linux system. I 
carried out the configure-make-install steps.

- jaclsh installed in /usr/local/bin requires the MINGW check to be 
commented out. Also the JACL_CLASSPATH doesn't include the java compiler 
janino.jar.

- jdk.cfg installed in /usr/local/bin uses ":" the linux path separator 
but only works if you change it to use the Windows path separator ";". 
Again janino.jar is missing.

- running the hello world example doesn't work in jaclsh unless you 
create a link to the janino.jar file in the current working directory
even after adding it to all the CLASSPATH's. It is installed in 
/usr/local/lib/tcljava1.3.2 by make install.

I was unable to get runtime compilation to work in an application 
because it can't find the janino.jar file. Maybe I am missing some step 
in the make procedure or some other PATH needs to be fixed up.

Cheers
-- 
Martti Tienhaara (ma...@da...)
DASH Software Ltd.

[tcljava-dev] New TJC runtime compiler implementation available via CVS

[Mo D. <md...@un...>]

Hi all

A new runtime Tcl proc compiler API has been added to the CVS version of Jacl.
This runtime compiler will take a Tcl procedure as input and generate Java
byte code that will work the same way as the original Tcl proc. Previously,
the TJC compiler only worked as a separate ahead of time compiler.

See the documentation for the "TJC::compile" command for more info.
The most basic usage is:

package require TJC
TJC::compile PROCNAME

The above will kick off a second thread to do the compilation and convert
the Tcl proc into Java source code. It will then use the embedded Janino
Java compiler to convert Java source code to a class file. Finally, the class
file will be loaded into the current interpreter, replacing the Tcl proc.

This is a very exciting development for Jacl, but it is also very new and in
need of much testing before it can be considered stable. Developers are
encouraged to download this new code and give it a test run.

cheers
Mo DeJong

[tcljava-dev] New TclClassLoader implementation needs testers

[Mo D. <md...@un...>]

Hi all

I have recently rewritten the TclClassLoader.java class used in Jacl and Tcl Blend
to take advantage of the JDK 1.2 class loader APIs. The Interp class now supports a
getClassLoader() API and the strange class loading bugs that some people were
running into with classes defined on env(TCL_CLASSPATH) should be solved.
I also added the ability to get resources defined in jars on the env(TCL_CLASSPATH).

While these bug fixes are good, the big change is that the TclClassLoader will now
delagate to the thread context class loader. This should fix any remaining problems
running Jacl in environments like WebSphere, Tomcat, JBoss, and others. Could
folks running Jacl in environments like these test out the current CVS head to
see if everything is working as expected?

thanks much
Mo DeJong

Re: [tcljava-dev] Using tclblend in a starkit

[Mo D. <md...@un...>]

On Tue, 24 Jan 2006 23:28:55 +0000
Matt Newman <mat...@hp...> wrote:

Hi Matt

> My question is this - how can I go about making the whole tclblend package
> as a starkit - scripts, dll and jars....

Humm, that sounds tricky. Tcl Blend loads the JVM from dlls and I know that
starkits do some tricky magic to make it possible to load a dll from a statkit
filesystem. Thing is, the Java API also does some tricky dll loading stuff and
I don't know if you are going to be able to work around that. The Java JVM needs
to load the Tcl Blend dll from Java code in order to link up the native methods.
See the Interp.java file for Tcl Blend for an example of this.

> The stumbling block is the class loader - I am no java expert, but I am
> aware that java apps can provide their own class loader implementation -
> yes?

Well, yes. But the class loader is for Java classes. This native library loading
issue is not related to Java classes.

> So what I had in mind was one that turns loader requests around as Tcl file
> system accesses - thereby benefiting from the VFS layer etc.
> 
> Has anyone done this?
> Could someone point me to any examples of simple class loaders and how one
> "installs" them (forgive my ignorance)...

See the TclClassLoader.java file for an example of Java class loading. I have
never heard or read of anything like what you propose. Please let us know if
it works out.

Mo DeJong

Re: [tcljava-dev] Query.

[Mo D. <md...@un...>]

On Wed, 1 Feb 2006 11:48:06 +0530
<pul...@wi...> wrote:

> 
> Hi Guys,
> 
>            I'll repeat my query. I'm having a product which is written
> in cgi.tcl package. Product does job like communicate with the hardware
> by sending a command in response of the command the hardware send a
> reply parse it then showing the replies in the web page. I want to know
> presently I'm using cgi.tcl package it takes too much time to form the
> web page.
> 
> Kindly tell me if I use Jacl or TclBlend will it takes less time. What
> other advantages I will have if I use jacl or TclBlend.

I have not used this cgi.tcl package myself, but I don't see why you
would want to use it with Jacl or Tcl Blend. Jacl and Tcl Blend are tools that
help on integrate Tcl and a Java VM. It really has nothing to do with
JavaScript, HTML, or HTTP processing.

Mo DeJong

Re: [tcljava-dev] cvs anonymous login is broken..

[Mo D. <md...@un...>]

On Tue, 07 Feb 2006 09:36:41 -0800
Bill Deegan <bil...@xi...> wrote:

> I try to login with empty password as directed on sourceforge and login 
> fails.
> 
> Is this a know problem?

SourceForge sometimes has login or load problems. I just tried the public CVS
and it seems to be working.

Mo DeJong

Re: [tcljava-dev] I have a patch (which only works with Java 1.5) which will yield full env() access to env variables

[Mo D. <md...@un...>]

On Tue, 07 Feb 2006 09:36:12 -0800
Bill Deegan <bil...@xi...> wrote:

> All,
> 
> I have a patch, but it only works with Java 1.5 which lets env() get all 
> the environment variables.
> 
> Is this something I can get into CVS?
> I could make a patch for the makefile/autoconf to substitute a 1.4 or 
> 1.5 version depending on which java you are using.

Best way to get it working is to use the reflection API to invoke the
method at runtime if running on a JDK 1.5 system. See ExecCmd.java
for an example of how this is done. I can check a reflection based
patch into the CVS, but a compile time 1.5 req is no good.

cheers
Mo DeJong

[tcljava-dev] I have a patch (which only works with Java 1.5) which will yield full env() access to env variables

[Bill D. <bil...@xi...>]

All,

I have a patch, but it only works with Java 1.5 which lets env() get all 
the environment variables.

Is this something I can get into CVS?
I could make a patch for the makefile/autoconf to substitute a 1.4 or 
1.5 version depending on which java you are using.

Thanks,
Bill

[tcljava-dev] cvs anonymous login is broken..

[Bill D. <bil...@xi...>]

I try to login with empty password as directed on sourceforge and login 
fails.

Is this a know problem?

Thanks,
Bill

RE: [tcljava-dev] Query.

[<pul...@wi...>]

Hi Guys,

           I'll repeat my query. I'm having a product which is written
in cgi.tcl package. Product does job like communicate with the hardware
by sending a command in response of the command the hardware send a
reply parse it then showing the replies in the web page. I want to know
presently I'm using cgi.tcl package it takes too much time to form the
web page.

Kindly tell me if I use Jacl or TclBlend will it takes less time. What
other advantages I will have if I use jacl or TclBlend.

Regards,
Pulkit Malhotra.              =20

-----Original Message-----
From: tcl...@li...
[mailto:tcl...@li...] On Behalf Of Neil
Madden
Sent: Tuesday, January 31, 2006 6:36 PM
To: tcl...@li...
Subject: Re: [tcljava-dev] Query.

pul...@wi... wrote:
>           Kindly help me i have some queries about the tcljava
package.=20
> How much is benefical to use tcljava instead of cgi.tcl package.  Can=20
> we make better GUI by using tcljava all kinds javascript can we use.=20
> If anybody is having performance related data kindly tell me.

Java is not the same thing as JavaScript. In particular, using the
TclJava package is unlikely to be helpful in creating web pages (which
seems to be what you want). You probably want to look at the "html",
"javascript", and "ncgi" packages in tcllib
(http://tcllib.sourceforge.net/).

-- Neil


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log
files for problems?  Stop!  Download the new AJAX search engine that
makes searching your log files as easy as surfing the  web.  DOWNLOAD
SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=3Dlnk&kid=3D103432&bid=3D230486&dat=3D=
121642
_______________________________________________
tcljava-dev mailing list
tcl...@li...
https://lists.sourceforge.net/lists/listinfo/tcljava-dev

Re: [tcljava-dev] RE: TclJava to do web pages [Was: Query.]

[Tom P. <tpo...@ny...>]

On Tue, Jan 31, 2006 at 07:51:56AM -0700, Tom Poindexter wrote:

> http://sourceforge.net/projects/gnutsp

I meant to include this link too, it has some more
info on the project:

http://www.rcreations.com/public/CoDisplayPage,0t,P=5.html


-- 
Tom Poindexter
tpo...@ny...

Re: [tcljava-dev] RE: TclJava to do web pages [Was: Query.]

[Tom P. <tpo...@ny...>]

On Tue, Jan 31, 2006 at 08:20:05AM -0500, Virden, Larry W. wrote:
> I don't know if the original poster was thinking of Javascript, Java
> Servlet Pages, writing web based Java apps, or writing Tcl CGI and
> making use of TclJava to invoke various Java functionality.
> 
> Ignoring Javascript as irrelevant, has anyone tried using Jacl with JSP
> or Java apps, or used TclJava with Tcl CGI ?

http://sourceforge.net/projects/gnutsp

Other than having a quick look at this some time ago, I haven't
done anything with it.  A quick look at CVS shows the
most recent CVS activity was 9 months ago.


-- 
Tom Poindexter
tpo...@ny...

[tcljava-dev] RE: TclJava to do web pages [Was: Query.]

[Virden, L. W. <lv...@ca...>]

I don't know if the original poster was thinking of Javascript, Java
Servlet Pages, writing web based Java apps, or writing Tcl CGI and
making use of TclJava to invoke various Java functionality.

Ignoring Javascript as irrelevant, has anyone tried using Jacl with JSP
or Java apps, or used TclJava with Tcl CGI ?


--=20
<URL: http://wiki.tcl.tk/ > Indescribable,uncontainable,all
powerful,untameable
Even if explicitly stated to the contrary, nothing in this posting
should be construed as representing my employer's opinions.
<URL: mailto:lv...@gm... > <URL: http://www.purl.org/NET/lvirden/
>
=20

-----Original Message-----
From: tcl...@li...
[mailto:tcl...@li...] On Behalf Of Neil
Madden


pul...@wi... wrote:
>           Kindly help me i have some queries about the tcljava
package.=20
> How much is benefical to use tcljava instead of cgi.tcl package.  Can
we
> make better GUI by using tcljava all kinds javascript can we use. If=20
> anybody is having performance related data kindly tell me. =20

Java is not the same thing as JavaScript. In particular, using the=20
TclJava package is unlikely to be helpful in creating web pages (which=20
seems to be what you want). You probably want to look at the "html",=20
"javascript", and "ncgi" packages in tcllib=20
(http://tcllib.sourceforge.net/).

-- Neil

Re: [tcljava-dev] Query.

[Neil M. <ne...@cs...>]

pul...@wi... wrote:
>           Kindly help me i have some queries about the tcljava package. 
> How much is benefical to use tcljava instead of cgi.tcl package.  Can we
> make better GUI by using tcljava all kinds javascript can we use. If 
> anybody is having performance related data kindly tell me.  

Java is not the same thing as JavaScript. In particular, using the 
TclJava package is unlikely to be helpful in creating web pages (which 
seems to be what you want). You probably want to look at the "html", 
"javascript", and "ncgi" packages in tcllib 
(http://tcllib.sourceforge.net/).

-- Neil

[tcljava-dev] Query.

[<pul...@wi...>]

hi,
=20
          Kindly help me i have some queries about the tcljava package.
How much is benefical to use tcljava instead of cgi.tcl package. Can we
make better GUI by using tcljava all kinds javascript can we use. If
anybody is having performance related data kindly tell me.    =20
=20

Regards,

Pulkit Malhotra.

=20

"Success is going from failure to failure without a loss of enthusiasm".
- Winston Churchill

=20

[tcljava-dev] Using tclblend in a starkit

[Matt N. <mat...@hp...>]

So having got tclblend to compile as a stub-enabled extension - which
required some trickery for the case when it is loaded from a JVM, as opposed
to being loaded from tcl - I'll provide more details on that adventure
later..

My question is this - how can I go about making the whole tclblend package
as a starkit - scripts, dll and jars....

The stumbling block is the class loader - I am no java expert, but I am
aware that java apps can provide their own class loader implementation -
yes?

So what I had in mind was one that turns loader requests around as Tcl file
system accesses - thereby benefiting from the VFS layer etc.

Has anyone done this?
Could someone point me to any examples of simple class loaders and how one
"installs" them (forgive my ignorance)...

Such a capability would be really useful for the kind of embedded apps I use
and I don't mind doing the leg work I'm just lost as to where to start.

Matt

PS. I am a long-time (15 yrs) Tcl user and developer just a weenie where
Java is concerned.

[tcljava-dev] Tcl Java Compiler timing results

[Martti T. <ma...@da...>]

Hi all

I finally got my test setup sorted out and I am happy to report that 
there is indeed a big improvement in compiled jacl speed.

jacl code = 134 seconds

compiled with no optimizations = 116 seconds

compiled with all optimizations (+O) = 63 seconds

Hence repetitive execution (1000's of times) of small jacl procedures in 
a large Java application definitely is improved by the new compiler.
Now that I can see substantial improvements I will be examining the jacl 
code as well to take better advantage of the compiler.

Thank you very much Mo DeJong.

Cheers

Martti

> 
> I sincerely apologize for my previous post. The timing result for the 
> compiled jacl code is wrong. I failed to recognize that the program was 
> sourcing the raw jacl file as well and hence was actually interpreting 
> the jacl code directly. I will investigate the setup some more to see if 
> I can get a real result for compiled jacl code.
> 
> 
> Martti Tienhaara wrote:
>>> Today's Topics:
>>>
>>>    1. A new Tcl to Java compiler for Xmas (Mo DeJong)
>>>
>> This is a great addition for the Jacl world. However I found that it 
>> doesn't provide any improvement when executing a large number of small 
>> procedures. I suspect speed improvements only occur when the overhead of 
>> setting up the environment for the compiled version at runtime is small 
>> compared to the execution time of the compiled code.
>>
>> For example 36,000 executions with various arguments of a simple 
>> procedure with about 50 lines of code gave total runtimes of
>>
>> raw jacl = 138 seconds
>> compiled jacl = 144 seconds
>> coded in java = 5 seconds
>>
>> This is embedded in a real application so a lot of execution time was 
>> consumed elsewhere in the Java code.
>>
>> I would be interested in hearing if anyone else has done any comparisons 
>> for larger jacl code procedures.
>>
>> Cheers
-- 
Martti Tienhaara (ma...@da...)
DASH Software Ltd.

Re: [tcljava-dev] Tcl Java Compiler

[Mo D. <md...@un...>]

On Fri, 13 Jan 2006 16:34:17 -0800
Martti Tienhaara <ma...@da...> wrote:

> 
> > Today's Topics:
> > 
> >    1. A new Tcl to Java compiler for Xmas (Mo DeJong)
> > 
> 
> This is a great addition for the Jacl world. However I found that it 
> doesn't provide any improvement when executing a large number of small 
> procedures.

Perhaps the documentation that I just added in the docs/TJC subdirectory
will help to explain how to use the TJC compiler. My own testing has
shown that most TJC compiled code executes 2 to 3 times faster while
some code could execute up to 10 times faster. It really depends on the
Tcl code you are compiling.

I hope that helps
Mo DeJong

[tcljava-dev] Re: Tcl Java Compiler

[Martti T. <ma...@da...>]

I sincerely apologize for my previous post. The timing result for the 
compiled jacl code is wrong. I failed to recognize that the program was 
sourcing the raw jacl file as well and hence was actually interpreting 
the jacl code directly. I will investigate the setup some more to see if 
I can get a real result for compiled jacl code.


Martti Tienhaara wrote:
> 
>> Today's Topics:
>>
>>    1. A new Tcl to Java compiler for Xmas (Mo DeJong)
>>
> 
> This is a great addition for the Jacl world. However I found that it 
> doesn't provide any improvement when executing a large number of small 
> procedures. I suspect speed improvements only occur when the overhead of 
> setting up the environment for the compiled version at runtime is small 
> compared to the execution time of the compiled code.
> 
> For example 36,000 executions with various arguments of a simple 
> procedure with about 50 lines of code gave total runtimes of
> 
> raw jacl = 138 seconds
> compiled jacl = 144 seconds
> coded in java = 5 seconds
> 
> This is embedded in a real application so a lot of execution time was 
> consumed elsewhere in the Java code.
> 
> I would be interested in hearing if anyone else has done any comparisons 
> for larger jacl code procedures.
> 
> Cheers

Martti Tienhaara (ma...@da...)
DASH Software Ltd.

[tcljava-dev] Tcl Java Compiler

[Martti T. <ma...@da...>]

> Today's Topics:
> 
>    1. A new Tcl to Java compiler for Xmas (Mo DeJong)
> 

This is a great addition for the Jacl world. However I found that it 
doesn't provide any improvement when executing a large number of small 
procedures. I suspect speed improvements only occur when the overhead of 
setting up the environment for the compiled version at runtime is small 
compared to the execution time of the compiled code.

For example 36,000 executions with various arguments of a simple 
procedure with about 50 lines of code gave total runtimes of

raw jacl = 138 seconds
compiled jacl = 144 seconds
coded in java = 5 seconds

This is embedded in a real application so a lot of execution time was 
consumed elsewhere in the Java code.

I would be interested in hearing if anyone else has done any comparisons 
for larger jacl code procedures.

Cheers

Martti Tienhaara (ma...@da...)
DASH Software Ltd.

Re: [tcljava-dev] A new Tcl to Java compiler for Xmas

[Tom P. <tpo...@ny...>]

On Tue, Dec 27, 2005 at 07:29:35PM -0800, Mo DeJong wrote:
> Yes, and the next step will be getting Jacl to compile procs into
> classfiles at runtime. That way, Jacl will compile Tcl procs into
> executable code as they are defined in the interp.

My 'Hyde' package might be of use for that (compile Java files at 
runtime),  I'd be glad to donate any/all of that code.  I can mail
you (or anyone else) a copy if you'd like.

Too much fun with the family to do much looking at your new code, 
hopefully in the next week or two.

-- 
Tom Poindexter
tpo...@ny...

Re: [tcljava-dev] A new Tcl to Java compiler for Xmas

[Mo D. <md...@un...>]

On Thu, 22 Dec 2005 08:48:04 -0700
Tom Poindexter <tpo...@ny...> wrote:

> On Wed, Dec 21, 2005 at 11:49:13AM -0800, Mo DeJong wrote:
> 
> > Tis the season for giving and because Jacl users have been
> > very good this year, the gift of a shiny new compiler is
> > now available via the tcljava CVS. The compiler is
> 
> Saaaaweeett!  And mostly implemeted in Tcl, no less. 

Yes, and the next step will be getting Jacl to compile procs into
classfiles at runtime. That way, Jacl will compile Tcl procs into
executable code as they are defined in the interp.

Interested developers might also want to check out the new "tjcruntime"
module that I just added. It provides a huge array of regression test
cases that check the correctness of tjc compiled code.

cheers
Mo DeJong