sablevm-developer — SableVM developer mailing list

[Sablevm-developer] Documentation on SableVM

[Etienne M. G. <eg...@j-...>]

Hi!

I have been quiet for a little while, as I was participating to OOPSLA
2000, then I spent some time writing about SableVM.  A have made the
resulting document public; it is the Sable Technical Report number
2000-3.

You can find the postscript version of this report at: 
 http://www.sable.mcgill.ca/publications/#report2000-3

If you prefer an html version, I have made one available on:
 http://www.j-meg.com/~egagnon/sable-report_2000-3/

The document explains the objectives and design of SableVM.  Based on
this document, it id now be possible to share tasks in the completion of
the Core SableVM engine (if any of you is interested).

Please let me know of any comment you have.

Thanks,

Etienne
-- 
----------------------------------------------------------------------
Etienne M. Gagnon, M.Sc.                     e-mail: eg...@j-...
Author of SableVM:                             http://www.sablevm.org/
----------------------------------------------------------------------

[Sablevm-developer] After "Hello World"...

[Etienne M. G. <eg...@j-...>]

Hi there!

With a running version of SableVM, it should be feasible to split tasks
around.  Using a good debugger, you should be able to look a little
around and see how the beast works.  In my last code, I have taken a few
shortcuts (like not worrying too much about memory error conditions,
etc.) to get the thing up and running, but the core structures are in
place and working.

I would ideally like to put my energy in the core interpreter parts.  I
have to implement the few missing bytecodes, add threads and GC (and
make the JNI implementation safe in their presence), and patch all the
shortcuts taken.

If you would like to work in a particular area, please let me know. 
Also, for starters, there are a few relatively easy cleaning tasks
available like:
 - putting all debugging code in functions (debug.[ch])
 - making debugging output conditional (2 levels):
    - level 1: vm options (verbose jni/gc/class, 
      Runtime.traceInstructions/traceMethodCalls, etc.)
    - level 2: static compiler conditional (#ifdef DEBUG)
 - verifying that all necessary "volatile" modifiers are there (and 
   no spurious ones), and checking that prototypes are consistent with
   definitions on their use of modifiers

Have fun!

Etienne
----------------------------------------------------------------------
Etienne M. Gagnon, M.Sc.                     e-mail: eg...@j-...
Author of SableVM:                             http://www.sablevm.org/
----------------------------------------------------------------------

Re: [Sablevm-developer] sablevm

[John L. <je...@pi...>]

> John Leuner wrote:
> > So this effectively acquires a mutex for that class and for that method. A
> > very good solution. My mutex blocks any thread in any class trying to
> > resolve a virtual method.
> 
> The drawback of my solution is space: 1 mutex/method...  If this becomes
> critical, we can revert to your solution.

It shouldn't be a problem. But even if it does become a problem, it's
better to try a more complex way of maintaining yours than to impose the
performance penalty that my solution has. 

Re: [Sablevm-developer] sablevm

[Etienne M. G. <eg...@j-...>]

John Leuner wrote:
> So this effectively acquires a mutex for that class and for that method. A
> very good solution. My mutex blocks any thread in any class trying to
> resolve a virtual method.

The drawback of my solution is space: 1 mutex/method...  If this becomes
critical, we can revert to your solution.

Etienne
-- 
----------------------------------------------------------------------
Etienne M. Gagnon, M.Sc.                     e-mail: eg...@j-...
Author of SableVM:                             http://www.sablevm.org/
----------------------------------------------------------------------

[Sablevm-developer] Back to distcheck state:-)

[Etienne M. G. <eg...@j-...>]

Hi guys.

As promised, I have put back the CVS snapshot in a compilable state. 
But, don't try to run anything with it yet or it will segfault on you...

Etienne
-- 
----------------------------------------------------------------------
Etienne M. Gagnon, M.Sc.                     e-mail: eg...@j-...
Author of SableVM:                             http://www.sablevm.org/
----------------------------------------------------------------------

Re: [Sablevm-developer] sablevm

[John L. <je...@pi...>]

> John Leuner wrote:
> > As a quick solution I just placed a recursive mutex around the method,
> > how are you planning to do it?
> 
> In my design, the mutex is in the body of the preparation opcode.  As
> soon as this opcode is eliminated, no more synchronization happens.  

So this effectively acquires a mutex for that class and for that method. A
very good solution. My mutex blocks any thread in any class trying to
resolve a virtual method.

John

Re: [Sablevm-developer] sablevm

[Etienne M. G. <eg...@j-...>]

John Leuner wrote:
> As a quick solution I just placed a recursive mutex around the method,
> how are you planning to do it?

In my design, the mutex is in the body of the preparation opcode.  As
soon as this opcode is eliminated, no more synchronization happens.  

Etienne
-- 
----------------------------------------------------------------------
Etienne M. Gagnon, M.Sc.                     e-mail: eg...@j-...
Author of SableVM:                             http://www.sablevm.org/
----------------------------------------------------------------------

RE: [Sablevm-developer] reflection classes for constant pool and bytecode data

[John L. <je...@pi...>]

> > From: John Leuner <je...@pi...>
> > I'm working on a separate little project to compile kissme into
> > the linux kernel, this is of course completely non-portable, but
> > provides excellent opportunity for using Java as a systems-level
> > programming language, eg writing device drivers, managing memory
> > etc.
> > When combined with some clever run-time optimisation such as
> > inlining, you could get a system that performs as fast as today's 
> > monolithic kernel OSes (NT / Linux), but with all the benefits of
> > the Java language.
> 
> Maybe NT. :)  Not Linux.
> 
> Seriously, I suppose you can really crunch during optimization and
> end up with Java-derived native code that is nearly as fast as C-
> derived native code, provided:
> 
>   - You never use virtual methods.
> 
>   - Either a) you never use exceptions; or b) your native code 
>     never has to check an exception type before branching.
> 
>   - You never have to specify exact alignment or packing of a
>     structure (neither of which can be guaranteed or even
>     specified by the Java spec). Hmm... This can probably be
>     restated as: You don't need 'C'-like structure accesses at
>     all.

Let me restate that. Instead of saying you'll get a kernel "as fast" as a
C monolithic kernel, I think that the advantages of dynamic optimisation
(run-time profiling etc) and removal of a userspace/kernelspace barrier
will allow such a system to approach the overall performance of a
monolithic C kernel. 

But you're right, as long as you're using Virtual Methods, relying on
automatic GC and exceptions you won't be able to achieve the same speed. 

I didn't understand the bit about alignment and packing?
 
> This, however, does not give you much of the benefits of the Java
> language. It doesn't let you write network protocol stacks or
> device drivers, either.

Well you can write code of that kind by providing magic methods
which are compiled from assembly to provide access to raw hardware.

John Leuner

[Sablevm-developer] sablevm

[John L. <je...@pi...>]

> > I like your ideas very much so far. I've downloaded the CVS but I can't
> > build bootstrap_class_loader.c:
> 
> The CVS isn't in a ggod shape.  I was using it to work from different
> locations.  As, at this point, I'm the only active developer, I didn't
> bother making sure it compiled...  I will fix this situation.  I promise
> not to repeat that;-)  [As I'll be asking everyone to always keep the
> repository in a "make distcheck" stable state].
> 
> You should, at this point play using version 0.1.4 (downloadable).

Ok, it's not a problem. I just wanted to get a feel for it.
 
> > I have a problem at the moment with multithreading and class resolution.
> > My interpreter wasn't designed for multiple threads to call the resolution
> > methods, so I'm dealing with that now.
> 
> You have to use synchronization, but obviously, you don't want all calls
> to the method to be synchronized, so you must work up a little magic. 
> The design is done, implementation of it is pending...

As a quick solution I just placed a recursive mutex around the method,
how are you planning to do it?
 
> Maybe, at this time, you should try to look a little at how 0.1.4
> works.  I am working pretty hard at putting back SableVM in a compilable
> state, as introducing exceptions broke many things.  I'll let you know
> as soon as it compiles.
> 
> Most likely, you will be able to help filling JNI interface holes.  But
> you'll need a few implemented examples to see how it's done.

Ok, I'll have a look at that. My JVM has a fairly brute-force
implementation and I wouldn't mind improving it .

> You should be able to figure out the stack magic using a debugger and
> 0.1.4:-)  My preferred debugger is DDD, which can graphically display
> the data structures.  Very informative, and helpful.

Yes I've installed it on my new box, haven't used it yet though.

John 

RE: [Sablevm-developer] reflection classes for constant pool and bytecode data

[Andrew P. <apu...@ac...>]

> From: John Leuner <je...@pi...>
> I'm working on a separate little project to compile kissme into
> the linux kernel, this is of course completely non-portable, but
> provides excellent opportunity for using Java as a systems-level
> programming language, eg writing device drivers, managing memory
> etc.
> When combined with some clever run-time optimisation such as
> inlining, you could get a system that performs as fast as today's 
> monolithic kernel OSes (NT / Linux), but with all the benefits of
> the Java language.

Maybe NT. :)  Not Linux.

Seriously, I suppose you can really crunch during optimization and
end up with Java-derived native code that is nearly as fast as C-
derived native code, provided:

  - You never use virtual methods.

  - Either a) you never use exceptions; or b) your native code 
    never has to check an exception type before branching.

  - You never have to specify exact alignment or packing of a
    structure (neither of which can be guaranteed or even
    specified by the Java spec). Hmm... This can probably be
    restated as: You don't need 'C'-like structure accesses at
    all.

This, however, does not give you much of the benefits of the Java
language. It doesn't let you write network protocol stacks or
device drivers, either.


Andy

RE: [Sablevm-developer] reflection classes for constant pool and bytecode data

[Andrew P. <apu...@ac...>]

> From: Etienne M. Gagnon <eg...@j-...>
> > John Leuner wrote:
> > I want to point you guys to some papers ( I read some of these last
> > weekend and they gave me lots of food for thought):
> > 
> > http://www-4.ibm.com/software/developer/library/jalapeno/index.html
>
> Mainly:  just keep your mind open and try to push beyond what
> you see other have done!  But, I agree: it is important to know
> about what others have done;-)

I found a related paper ("An Empirical Study of Selective
Optimization"; Matthew Arnold, Michael Hind, and Barbara G.
Ryder; accessable from the link John provided) very interesting.
The result is the conclusion that coarse-grained profiling is
not only sufficient for determining which methods to aggresively
optimize, but is closer to optimal than all other considered
alternatives.

This is good... It means that we can select methods that will
likely benefit from JIT in a simple, low-overhead manner from
within the SableVM interpreter:

  1.  Interrupt normal processing every 10? 100? ms. 

      a.  Determine which method is currently executing.

      b.  If it is native, do nothing further.

      c.  Increment a counter which records how often this
          method was seen executing during profiling. If this
          counter exceeds a minimum value, continue; otherwise
          do nothing further.

      d.  Find (or create) an entry on the JIT's global "to
          do" list for this method (a simple doubly-linked 
          list).

      e.  Look at the previous list entry. If current->count 
          > prev->count; swap prev <-> current. (Only one swap,
          though, not bubblesort!)

      f.  Signal JIT if it is suspended.


  2.  In the JIT thread:

      a.  Remove top entry of "to do" list. If empty, suspend.

      b.  JIT compile.

      c.  goto a


Andy

[Fwd: RE: [Sablevm-developer] reflection classes for constant pool and bytecode data]

[Andrew P. <apu...@ac...>]

> From: "John Leuner" <je...@pi...>
> I was wondering if this interface needs to be sablevm 
> specific? All JVMs will be able to provide the information 
> you're asking for here, and probably in exactly the same format.

No, generally this interface is not SableVM-specific, and should
be easily ported to any VM. The obvious exception is the method(s)
which prompt the VM to replace a method with new native code.

> I'm thinking here of cloning this work with my JVM, ultimately
> I would like to have whatever JIT framework we develop here
> running on both JVMs. 

That would be great. Also, I expect there are useful things in your
JVM which could be rolled in as well.
 
> access flags. But if you're only looking for the standard ACC_PUBLIC,
> ACC_PROTECTED etc, it should be ok.

Yes, and the ACC_* flag values will be static constants.
 
> >       public int exception_table[][];
> 
> How does this store the exception type?

for (int i = 0; i < exception_table.length; i++) {
    int ia[] = exception_table[i];
    String name;
    if (ia[3] == 0) {
        name = "all";
    } else {
        int name_index = 
          ((Constant_Class)clazz.getConstant(ia[3])).name_index;
        name = ((Constant_UTF8)clazz.getConstant(name_index)).value;
    }
    System.out.println(
        "catch " + name +
        " from L" + ia[0] +
        " to L" + ia[1] + 
        " using L" + ia[2]
        );
}

> As regards my own work on my primitive bytecode-to-assembly 
> translator, I've managed to encode exception handlers within
> the native(optimised) code now.

Very cool.


Andy

[Sablevm-developer] Gone fishing...

[Etienne M. G. <eg...@j-...>]

Hi there.

I'll be away from my computer for a few days.  I'll be in the woods, up
north of Quebec city, fishing.

Unfortunately, I didn't have time to get the cvs snapshot back to a
compile state.  You'll have to continue playing with the 0.1.4 version
(unless you feel adventurous, and want to give a look at the latest
development).

Meanwhile, have fun!  I'll try to have some;-)

Etienne
-- 
----------------------------------------------------------------------
Etienne M. Gagnon, M.Sc.                     e-mail: eg...@j-...
Author of SableVM:                             http://www.sablevm.org/
----------------------------------------------------------------------

Re: [Sablevm-developer] reflection classes for constantpoolandbytecode data

[Etienne M. G. <eg...@j-...>]

Hi John,

John Leuner wrote:
> Ok, but if we called it jvm it would be ok, because that acronym in not a
> trademark?


OK. I conceed that I was a little over zealous on this one.  The acronym
is not trademarked, and thus should be fine to use.  Mainly, I wanted to
raise a flag to be careful with trademarks.  We certainly don't want to
infringe on anybody's rights.


> It doesn't interpret the language, it interprets bytecode, which I want to
> call the java platform, but we need another name for it.

The best I could come with was "bytecode interpreter".  There isn't
anything trademarkable about it;-)  You could possibly push that to
"Java bytecode interpreter"...

> I like your ideas very much so far. I've downloaded the CVS but I can't
> build bootstrap_class_loader.c:

The CVS isn't in a ggod shape.  I was using it to work from different
locations.  As, at this point, I'm the only active developer, I didn't
bother making sure it compiled...  I will fix this situation.  I promise
not to repeat that;-)  [As I'll be asking everyone to always keep the
repository in a "make distcheck" stable state].

You should, at this point play using version 0.1.4 (downloadable).

> So you insert a PREPARE bytecode for every method that hasn't run yet?

I don't insert it;  I simply point the code pointer to it (a single
copy).

> I have a problem at the moment with multithreading and class resolution.
> My interpreter wasn't designed for multiple threads to call the resolution
> methods, so I'm dealing with that now.

You have to use synchronization, but obviously, you don't want all calls
to the method to be synchronized, so you must work up a little magic. 
The design is done, implementation of it is pending...

> How do you know it runs faster if you haven't got it running yet? (Or are
> you saying theoretically).

Theoretically and not.  If you run 0.1.4, you'll notice that many
bytecodes are executed.  Simply grab a debugger and place a brakpoint in
the interpreter.  You'll see.

> Well I'm very interested, please tell if you think there is something I
> could help with.

Maybe, at this time, you should try to look a little at how 0.1.4
works.  I am working pretty hard at putting back SableVM in a compilable
state, as introducing exceptions broke many things.  I'll let you know
as soon as it compiles.

Most likely, you will be able to help filling JNI interface holes.  But
you'll need a few implemented examples to see how it's done.

You should be able to figure out the stack magic using a debugger and
0.1.4:-)  My preferred debugger is DDD, which can graphically display
the data structures.  Very informative, and helpful.

Etienne
-- 
----------------------------------------------------------------------
Etienne M. Gagnon, M.Sc.                     e-mail: eg...@j-...
Author of SableVM:                             http://www.sablevm.org/
----------------------------------------------------------------------

[Sablevm-developer] Public statement re: Sun's java licences

[John L. <je...@pi...>]

Through this public statement I wish to indicate that I have not signed
any agreement or licence with Sun Microsystems regarding the use of their
source code for Java Virtual Machine implementations or class library
implementations. 

I have not worked for any employer that has signed such an agreement
either. I have not signed any non-disclosure agreement with anyone
regarding source code for JVM technology or java class libraries.

I acknowledge that any contributions I make to the sablevm project will be
licensed under that project's license. Contributions to sablePath will
also be licensed under that project's license.

John Leuner

Re: [Sablevm-developer] reflection classes for constant pool andbytecode data

[Etienne M. G. <eg...@j-...>]

Hi all!

First, a small detail I've learned lately:  we are not talking of a JVM,
but a Jvm;-)  What's the difference?  Big.  A Java Virtual Machine is a
virtual machine for the Sun's Java platform (trademarked).  On the other
hand, a Java virtual machine is an interpreter for the "Java language"
(not platform!).  I certainly do not claim SableVM to be a JVM!  In
fact, I prefer to stick to unambiguous terms like "vm, interpreter, ..."
which do not even remotely relate to the trademark.  It is accepted the
the "Java language" is the name of a programming language, so might be
freely used.  But you must be careful to make clear that you really mean
"the language" whenever you use the term "Java".  Other uses might be
restricted.

John Leuner wrote:
> I was wondering if this interface needs to be sablevm specific? All JVMs
> will be able to provide the information you're asking for here, and
> probably in exactly the same format. I'm thinking here of cloning this
> work with my JVM, ultimately I would like to have whatever JIT framework
> we develop here running on both JVMs.

The interpreter could simply make public the raw bytes of the class
file.  This should be suitable for all interpreters, and could easily be
provided by any vm.

> 
> Of course there are things that might differ, eg the
> > >       public org.sablevm.vm.Class clazz;     // class containing this method
> > >       public int access_flags;

The difficult part is not gathering class information, but hooking back
with the vm.  E.g., how does the jit hook back to the right
runtime_package/class?  How does the jit handle exceptions that should
be forwarded beyond the compiled methods, etc.  I doubt these things
could be really done in a vm independant way, but you're allowed to
surprize me;-)

> I want to point you guys to some papers ( I read some of these last
> weekend and they gave me lots of food for thought):
> 
> http://www-4.ibm.com/software/developer/library/jalapeno/index.html
> 
> Is an excellent overview of Jalapeno, it discusses locking (which I want
> to reimplement on my JVM), has a good high-level description of the whole
> Optimiser.
> 
> An article on the optimiser itself:
> 
> "THe Jalapeno Dynamic optimizing compiler for Java" -
> http://www.research.ibm.com/jalapeno/publication.html#grande99
> 
> And on runtime selection of methods to optimise:
> 
> "An empirical study of Selective optimisation" -
> http://www.research.ibm.com/jalapeno/publication.html#lcpc00
> 

Thanks for these additional pointers.  

Keep in mind that, althought IBM's work is impressive, it often depends
on very machine/OS specific features.  There's much assembly level
things involved in building Jalapeno (even if it is written in Java, it
generates machine code).  This is something a jit can't avoid, I agree. 
But, SableVM (the interpreter) tries to achieve efficiency without being
too machine/OS dependent, e.g.: very little dependency, which should be
relatively easy to adapt, like a max of 2~10 assembly lines in the whole
interpreter (light locks), very little OS dependency beyond POSIX
compliance, mostly ANSI C (+ GNU C labels as values)...  A jit could try
to fit its interface within this framework, and thus could be quite
different fom Jalapeno.  Mainly:  just keep your mind open and try to
push beyond what you see other have done!  But, I agree: it is important
to know about what others have done;-)

I have been working on SableVM's exception framework, lately, which
turns out difficult to avoid for interpreting HelloWorld.  (The JNI
interface is too much dependent on it...)

Have fun with the JIT!

Etienne
-- 
----------------------------------------------------------------------
Etienne M. Gagnon, M.Sc.                     e-mail: eg...@j-...
Author of SableVM:                             http://www.sablevm.org/
----------------------------------------------------------------------

Re: [Sablevm-developer] reflection classes for constant pool and bytecode data

[John L. <je...@pi...>]

Hi Guys

> Anybody else has comments?  (I think we should CC John Leuner, which I
> am doing now).  Maybe he'll be interested to being added as a developer,
> too? (This last question is directed to John;-)

Yes please. (I'm on the sablvm-developer list already).

> > Hi, SableVM developers!
> > 
> > I've been looking over the SableVM sources and I think the first step
> > toward creating a JIT should be the addition of a simple class hierarchy
> > containing native methods for exposing SableVM's internal representations
> > of the constant pool and (prepared) bytecodes to the Java environment.
> > I've been working this week with some classes I've cobbled together for
> > parsing class files directly from disk, but keeping the class data in
> > memory solely for a JIT's use is wasteful.

I was wondering if this interface needs to be sablevm specific? All JVMs
will be able to provide the information you're asking for here, and
probably in exactly the same format. I'm thinking here of cloning this
work with my JVM, ultimately I would like to have whatever JIT framework
we develop here running on both JVMs. 

Of course there are things that might differ, eg the 
> >       public org.sablevm.vm.Class clazz;     // class containing this method
> >       public int access_flags;

access flags. But if you're only looking for the standard ACC_PUBLIC,
ACC_PROTECTED etc, it should be ok.

> >       public int max_stack;
> >       public int max_locals;
> >       public int exception_table[][];

How does this store the exception type?

> >       public org.sablevm.vm.Opcode code[];   // method opcodes
> >       ...
> >   }


As regards my own work on my primitive bytecode-to-assembly translator,
I've managed to encode exception handlers within the native(optimised) 
code now.
Thus:

try {
	int[] array = null;
	array[3] = 2;
} catch(NullPointerException npe)
 {
	return 2;
 }

will return 2 from the native (optimisedd) method without invoking a very
expensive search of the stack frames for execption handlers. 

Also, if native code generates an exception (these usually come from
bytecodes that use object / array references, or method invocations) it
will pass it upwards for the calling method to deal with. 

My next problem to solve is to make sure that I match the type of the
exception correctly. The problem is that the exception must be
type-checked at runtime, so I have to generate the equivalent of:

exception object in eax

if( DEREF(eax)->pstType == type in exception handler)
  jump to handler
carry on

So for me this means parsing the C structure that represents the tObject
structure, and finding at what offset the pstType member is. 

This has led me to write a C preprocessor which strips comments, and one
which processes the #define #ifdef, #endif statements so that I can start
manipulating the C structure within the runtime optimiser. All of this is
in Java, I'm using the ANTLR compiler-compiler to generate lexers and
parsers. ( I noticed SableCC just now, will look at that).

Once I complete this I hope my optimiser will be pretty much finished ... 
I suppose I still have to monitors and there are issues with invokespecial
and others, but these are just a matter of rewriting the code.




I want to point you guys to some papers ( I read some of these last
weekend and they gave me lots of food for thought):

http://www-4.ibm.com/software/developer/library/jalapeno/index.html

Is an excellent overview of Jalapeno, it discusses locking (which I want
to reimplement on my JVM), has a good high-level description of the whole
Optimiser. 

An article on the optimiser itself:

"THe Jalapeno Dynamic optimizing compiler for Java" -
http://www.research.ibm.com/jalapeno/publication.html#grande99

And on runtime selection of methods to optimise:

"An empirical study of Selective optimisation" -
http://www.research.ibm.com/jalapeno/publication.html#lcpc00




John Leuner

RE: [Sablevm-developer] reflection classes for constant pool and bytecode data

[Andrew P. <apu...@ac...>]

> Etienne M. Gagnon <eg...@j-...> wrote:
> >   public class org.sablevm.vm.Constant_UTF8
> >   public class org.sablevm.vm.Constant_Unicode
>
> I am unsure about these two...  Why not keep a single
> "org.sablevm.vm.Constant_String", as the interpreter has
> presumably already a native global reference to the string?

I just wanted to make the source encoding explicit in the constant
class type.

> >       public int size() { return 2; }
>
> Why the "size"?  Don't you assume a long is 64 bits, as per
> specification?  On a 64 bit processor, this would be a single
> word.  We shouldn't make the JIT interface specific to an
> architecture.

Size means "number of 32 bit words". I guess its not really
necessary.
 
> >   public class org.sablevm.vm.Opcode  { ... };
>
> As I said earlier, it would be possible to keep the original
> code array around, to shield the JIT from SableVM specific
> internal codes,as much as possible. SableVM encodes the
> destination of branches using absolute addresses.  You
> certainly don't want to reverse engineer those addresses!

Hmm. I was thinking I might gain insight by looking at the internal
codes (although I haven't anything specific yet). However, I
suppose use of absolute addressing in the internal codes isn't
going to be the only problem down the road (and it is a problem 
now), so I'll stick to using the original Java bytecode. 

I haven't had much time this week to work on this. Hopefully next
week will be kinder...


Andy

Re: [Sablevm-developer] reflection classes for constant pool and bytecode data

[Etienne M. G. <eg...@j-...>]

Andrew Purtell wrote:
> 
> I've been working this week with some classes I've cobbled together for
> parsing class files directly from disk, but keeping the class data in
> memory solely for a JIT's use is wasteful.

It's like you want... It would be easy to keep the raw code attribute
around, if necessary.

> How does this look?


>   public class org.sablevm.vm.Constant_UTF8
>   public class org.sablevm.vm.Constant_Unicode

I am unsure about these two...  Why not keep a single
"org.sablevm.vm.Constant_String", as the interpreter has presumably
already a native global reference to the string?  You can use
"java.lang.String" methods to get the length and characters.  Unless I
missed the point competely?...

>   public class org.sablevm.vm.Constant_Long
>       extends org.sablevm.vm.Constant
>   {
>       public long value;
>       public int size() { return 2; }
>   };


Why the "size"?  Don't you assume a long is 64 bits, as per
specification?  On a 64 bit processor, this would be a single word.  We
shouldn't make the JIT interface specific to an architecture.

>   public class org.sablevm.vm.Opcode  { ... };

As I said earlier, it would be possible to keep the original code array
around, to shield the JIT from SableVM specific internal codes,as much
as possible. SableVM encodes the destination of branches using absolute
addresses.  You certainly don't want to reverse engineer those
addresses!  

> I've only jotted down read-only methods so far. Eventually there will need
> to be an org.sablevm.vm.Class method for attaching prepared native code to
> a given method.
> 
> Note that these are NOT clases I plan to keep JIT state in during
> translation. There's an "org.sablevm.vm.jit.*" hierarchy I'm thinking over
> at the moment.
> 
> I was thinking the native methods implementing these classes could be added
> to libsablevm. 

Yes.

> The associated classes will need to be added to sablepath.

Yes.

> Seeing how both are pulled in only if they are part of the transitive
> closure of referenced classes/functions/etc., they cost nothing if the "use
> JIT" configure option is not enabled.

True.  (Exception of keeping the code array around, but we could
"#ifdef" this section of the class loader pretty easily).

Etienne
-- 
----------------------------------------------------------------------
Etienne M. Gagnon, M.Sc.                     e-mail: eg...@j-...
Author of SableVM:                             http://www.sablevm.org/
----------------------------------------------------------------------

Re: [Sablevm-developer] reflection classes for constant pool and bytecode data

[Etienne M. G. <eg...@j-...>]

Hi Andrew.

I have to give this a careful look.  I'll get back tomorrow about it.  

Anybody else has comments?  (I think we should CC John Leuner, which I
am doing now).  Maybe he'll be interested to being added as a developer,
too? (This last question is directed to John;-)
  
Etienne

Andrew Purtell wrote:
> 
> Hi, SableVM developers!
> 
> I've been looking over the SableVM sources and I think the first step
> toward creating a JIT should be the addition of a simple class hierarchy
> containing native methods for exposing SableVM's internal representations
> of the constant pool and (prepared) bytecodes to the Java environment.
> I've been working this week with some classes I've cobbled together for
> parsing class files directly from disk, but keeping the class data in
> memory solely for a JIT's use is wasteful.
> 
> How does this look?
> 
>   public abstract class org.sablevm.vm.Constant
>   {
>       public int size() { return 1; }
>   };
> 
>   public class org.sablevm.vm.Constant_UTF8
>       extends org.sablevm.vm.Constant
>   {
>       public String value;
>   };
> 
>   public class org.sablevm.vm.Constant_Unicode
>       extends org.sablevm.vm.Constant
>   {
>       public char[] value;
>   };
> 
>   public class org.sablevm.vm.Constant_Integer
>       extends org.sablevm.vm.Constant
>   {
>       public int value;
>   };
> 
>   public class org.sablevm.vm.Constant_Float
>       extends org.sablevm.vm.Constant
>   {
>       public float value;
>   };
> 
>   public class org.sablevm.vm.Constant_Long
>       extends org.sablevm.vm.Constant
>   {
>       public long value;
>       public int size() { return 2; }
>   };
> 
>   public class org.sablevm.vm.Constant_Double
>       extends org.sablevm.vm.Constant
>   {
>       public double value;
>       public int size() { return 2; }
>   };
> 
>   public class org.sablevm.vm.Constant_String
>       extends org.sablevm.vm.Constant
>   {
>       public String value;
>   };
> 
>   public class org.sablevm.vm.Constant_Class
>       extends org.sablevm.vm.Constant
>   {
>       public int class_index;
>   };
> 
>   public class org.sablevm.vm.Constant_Fieldref
>       extends org.sablevm.vm.Constant
>   {
>       public int class_index;
>       public int name_and_type_index;
>   };
> 
>   public class org.sablevm.vm.Constant_Methodref
>       extends org.sablevm.vm.Constant
>   {
>       public int class_index;
>       public int name_and_type_index;
>   };
> 
>   public class org.sablevm.vm.Constant_InterfaceMethodref
>       extends org.sablevm.vm.Constant
>   {
>       public int class_index;
>       public int name_and_type_index;
>   };
> 
>   public class org.sablevm.vm.Constant_NameAndType
>       extends org.sablevm.vm.Constant
>   {
>       public int name_index;
>       public int signature_index;
>   };
> 
>   public class org.sablevm.vm.Opcode  { ... };
> 
>   public class org.sablevm.vm.Code
>   {
>       public org.sablevm.vm.Class clazz;     // class containing this method
>       public int access_flags;
>       public int max_stack;
>       public int max_locals;
>       public int exception_table[][];
>       public org.sablevm.vm.Opcode code[];   // method opcodes
>       ...
>   }
> 
>   public class org.sablevm.vm.Field
>   {
>       public int access_flags;
>       public String name;
>       public String signature;
>       ...
>   }
> 
>   public class org.sablevm.vm.Class
>   {
>       public org.sablevm.vm.Class superclass;     // superclass, if any
>       public int access_flags;
>       public org.sablevm.vm.Field fields[];       // class fields
>       public org.sablevm.vm.Field methods[];      // class/interface methods
> 
>       // constructs a Class object from the internal representation of
>       // class or interface 'name'
>       public native org.sablevm.vm.Class (String name);
> 
>       // retrieve a Constant object for the constant located at index 'i'
>       // for Class 'name'
>       public static native org.sablevm.vm.Constant getConstant (String name,
>                                                                 int i);
> 
>       // returns a Code object encapsulating the given method's bytecodes,
>       // exception table, max stack size, and max locals
>       public static native org.sablevm.vm.Code getCode (String class,
>                                                         String name,
>                                                         String signature);
> 
>       // object method version of getConstant()
>       public native org.sablevm.vm.Constant getConstant (int i);
> 
>       // object method version of getCode()
>       public native org.sablevm.vm.Code getCode (String name,
>                                                  String signature);
>   }
> 
> I've only jotted down read-only methods so far. Eventually there will need
> to be an org.sablevm.vm.Class method for attaching prepared native code to
> a given method.
> 
> Note that these are NOT clases I plan to keep JIT state in during
> translation. There's an "org.sablevm.vm.jit.*" hierarchy I'm thinking over
> at the moment.
> 
> I was thinking the native methods implementing these classes could be added
> to libsablevm. The associated classes will need to be added to sablepath.
> Seeing how both are pulled in only if they are part of the transitive
> closure of referenced classes/functions/etc., they cost nothing if the "use
> JIT" configure option is not enabled.
> 
> Andrew Purtell
> from home -- apu...@ac...
> 
> _______________________________________________
> Sablevm-developer mailing list
> Sab...@li...
> http://lists.sourceforge.net/mailman/listinfo/sablevm-developer

-- 
----------------------------------------------------------------------
Etienne M. Gagnon, M.Sc.                     e-mail: eg...@j-...
Author of SableVM:                             http://www.sablevm.org/
----------------------------------------------------------------------

[Sablevm-developer] reflection classes for constant pool and bytecode data

[Andrew P. <apu...@ac...>]

Hi, SableVM developers!

I've been looking over the SableVM sources and I think the first step
toward creating a JIT should be the addition of a simple class hierarchy
containing native methods for exposing SableVM's internal representations
of the constant pool and (prepared) bytecodes to the Java environment.
I've been working this week with some classes I've cobbled together for
parsing class files directly from disk, but keeping the class data in
memory solely for a JIT's use is wasteful.

How does this look?

  public abstract class org.sablevm.vm.Constant 
  {
      public int size() { return 1; }
  };

  public class org.sablevm.vm.Constant_UTF8
      extends org.sablevm.vm.Constant 
  {
      public String value;
  };

  public class org.sablevm.vm.Constant_Unicode
      extends org.sablevm.vm.Constant
  {
      public char[] value;
  };

  public class org.sablevm.vm.Constant_Integer
      extends org.sablevm.vm.Constant
  {
      public int value;
  };

  public class org.sablevm.vm.Constant_Float
      extends org.sablevm.vm.Constant
  {
      public float value;
  };

  public class org.sablevm.vm.Constant_Long
      extends org.sablevm.vm.Constant
  {
      public long value;
      public int size() { return 2; }
  };

  public class org.sablevm.vm.Constant_Double
      extends org.sablevm.vm.Constant
  {
      public double value;
      public int size() { return 2; }
  };

  public class org.sablevm.vm.Constant_String
      extends org.sablevm.vm.Constant
  {
      public String value;
  };

  public class org.sablevm.vm.Constant_Class
      extends org.sablevm.vm.Constant
  {
      public int class_index;
  };

  public class org.sablevm.vm.Constant_Fieldref
      extends org.sablevm.vm.Constant
  {
      public int class_index;
      public int name_and_type_index;
  };

  public class org.sablevm.vm.Constant_Methodref
      extends org.sablevm.vm.Constant
  {
      public int class_index;
      public int name_and_type_index;
  };

  public class org.sablevm.vm.Constant_InterfaceMethodref
      extends org.sablevm.vm.Constant
  {
      public int class_index;
      public int name_and_type_index;
  };

  public class org.sablevm.vm.Constant_NameAndType
      extends org.sablevm.vm.Constant
  {
      public int name_index;
      public int signature_index;
  };

  public class org.sablevm.vm.Opcode  { ... };

  public class org.sablevm.vm.Code
  {
      public org.sablevm.vm.Class clazz;     // class containing this method
      public int access_flags;
      public int max_stack;
      public int max_locals;
      public int exception_table[][];
      public org.sablevm.vm.Opcode code[];   // method opcodes
      ...
  }

  public class org.sablevm.vm.Field
  {
      public int access_flags;
      public String name;
      public String signature;
      ...
  }

  public class org.sablevm.vm.Class
  {
      public org.sablevm.vm.Class superclass;     // superclass, if any
      public int access_flags;
      public org.sablevm.vm.Field fields[];       // class fields
      public org.sablevm.vm.Field methods[];      // class/interface methods

      // constructs a Class object from the internal representation of
      // class or interface 'name'
      public native org.sablevm.vm.Class (String name);

      // retrieve a Constant object for the constant located at index 'i'
      // for Class 'name'
      public static native org.sablevm.vm.Constant getConstant (String name,
                                                                int i);

      // returns a Code object encapsulating the given method's bytecodes,
      // exception table, max stack size, and max locals
      public static native org.sablevm.vm.Code getCode (String class,
                                                        String name,
                                                        String signature);

      // object method version of getConstant()
      public native org.sablevm.vm.Constant getConstant (int i);

      // object method version of getCode()
      public native org.sablevm.vm.Code getCode (String name, 
                                                 String signature);
  }

I've only jotted down read-only methods so far. Eventually there will need
to be an org.sablevm.vm.Class method for attaching prepared native code to
a given method.

Note that these are NOT clases I plan to keep JIT state in during
translation. There's an "org.sablevm.vm.jit.*" hierarchy I'm thinking over
at the moment.

I was thinking the native methods implementing these classes could be added
to libsablevm. The associated classes will need to be added to sablepath.
Seeing how both are pulled in only if they are part of the transitive
closure of referenced classes/functions/etc., they cost nothing if the "use
JIT" configure option is not enabled.


Andrew Purtell
from home -- apu...@ac...

Re: [Sablevm-developer] Continuations (Part 1)

[Etienne M. G. <eg...@j-...>]

Hi Brent.

Brent Fulgham wrote:
> Sounds good.  I've got to spend a little time doing some research
> before I can fully quantify the problem.

Have you given a look at LISC (ftp://ftp.gamora.org/pub/gamora/lisc/)? 
It is a small Scheme interpreter written in Java.

It might give you some ideas...

Etienne
-- 
----------------------------------------------------------------------
Etienne M. Gagnon, M.Sc.                     e-mail: eg...@j-...
Author of SableVM:                             http://www.sablevm.org/
----------------------------------------------------------------------

RE: [Sablevm-developer] Continuations (Part 1)

[Brent F. <bre...@xp...>]

> Brent Fulgham wrote:
> >... 
> > (define (my-resumable-proc)
> >   (do-something)
> >   (display (call-with-current-continuation my-abortable-proc))
> >   (do-some-more))
> > ...
> 
> This is getting way over my head.  My knowledge of Scheme is 
> near null.  I know LISP a little, and I have been told that 
> Scheme is like LISP but with static binding in places where LISP
> implements dynamic binding...
> 
> I'll let you research the literature, and come with suggestions, when
> you are ready.
>
Sounds good.  I've got to spend a little time doing some research
before I can fully quantify the problem.

Thanks,

-Brent
 

Re: [Sablevm-developer] Continuations (Part 1)

[Etienne M. G. <eg...@j-...>]

Hi Brent!

Brent Fulgham wrote:
>... 
> (define (my-resumable-proc)
>   (do-something)
>   (display (call-with-current-continuation my-abortable-proc))
>   (do-some-more))
> ...

This is getting way over my head.  My knowledge of Scheme is near null. 
I know LISP a little, and I have been told that Scheme is like LISP but
with static binding in places where LISP implements dynamic binding...

I'll let you research the literature, and come with suggestions, when
you are ready.

Etienne
-- 
----------------------------------------------------------------------
Etienne M. Gagnon, M.Sc.                     e-mail: eg...@j-...
Author of SableVM:                             http://www.sablevm.org/
----------------------------------------------------------------------

RE: [Sablevm-developer] Continuations (Part 2)

[Brent F. <bre...@xp...>]

> 1- At the bytecode level, exception handling is expressed quite
> differently from Java source.

Right, but there's a bytecode (Returnsub or something like that)
that might be usable?

> 2- If I remember my high level programming languages course correctly,
> the whole point of a continuation is to capture a snapshot of the
> "current" stack, so that you can later execute code in this same
> context, even if the current stack is unwinded.  In ML (or some
> dialect?) that allowed implementing "static binding" instead 
> of "dynamic binding" a la LISP.  The consequence is that the method
> in which the "create snapshot" occurs might return before you 
> "resume" and execute the continuation.  In which case your scheme
> wouldn't work, unfortunately.
> 
Understood.  I hadn't thought this through fully.  Drat.

> Please let me know if I am completely off track. (Quite possible!)
> 
> In fact, can you write for me a simple example that requires
> continuation?
> 
(Simple in Scheme, but ugly in Java -- see Part1).

> See below for additional questions.
> 
> > 3. JVM creates an exception that holds the current state of
> >    this routine (i.e., must make a copy of the current variables)
> 
> What is in the "saved" state?
> 
> - current stack frame?
> - stack (all frames) of current thread?
> - global variables (e.g. static fields)?
> - other stacks (stack of other threads)?
> - heap state (object instances?
> - other vm internal state (locks, ...)?
> 

Unknown.  It's almost like taking a snapshot of the VM at the moment
you hit the continuation.  But I'm not totally sure how it's typically
implemented.  I need to read through the http://www.nightmare.com/
(schintro-v14/schintro_3.html) stuff because it explains one implementaion
of this.

Anyway, probably not really worth worrying about just now.  :-)

-Brent