sablevm-developer

[Sablevm-developer] using a Soot-transformed classpath

[Chris P. <chr...@ma...>]

Hi,

I want to make SableVM use Soot-transformed versions of the non-native 
portion of Classpath.  I have the output from Soot in the application 
directory where I try to run SableVM, but the classes aren't getting 
loaded even with '-c .' specified as an option.

What is the best way to do this?

For an application that transforms just gnu.* and java.* I tried:

cd ~/lib/sablevm/classpath
mv gnu gnu-old
mv java java-old
ln -s ~/path-to-sootOutput/java
ln -s ~/path-to-sootOutput/gnu

but that didn't work (cannot create vm).

Also, on a related note -- does anyone know of a good way to catch those 
classes which are only referenced from native code and thus not found by 
Soot?  (I'm assuming such classes exist for certain applications). 
Profiling SableVM for the application and then comparing against the 
Soot output for whole-program-including-classpath-transformation seems 
like the only realistic choice, but maybe I'm missing something (it's 
not difficult to do, but it's not a real analysis).

Cheers,
Chris

Re: [Sablevm-developer] using a Soot-transformed classpath

[Chris P. <chr...@ma...>]

Chris Pickett wrote:

> Hi,
> 
> I want to make SableVM use Soot-transformed versions of the non-native 
> portion of Classpath.  I have the output from Soot in the application 
> directory where I try to run SableVM, but the classes aren't getting 
> loaded even with '-c .' specified as an option.
> 
> What is the best way to do this?
> 
> For an application that transforms just gnu.* and java.* I tried:
> 
> cd ~/lib/sablevm/classpath
> mv gnu gnu-old
> mv java java-old
> ln -s ~/path-to-sootOutput/java
> ln -s ~/path-to-sootOutput/gnu
> 
> but that didn't work (cannot create vm).

I also tried

cd ~/lib/sablevm/classpath
mv gnu gnu-old
mv java java-old
cp -R ~/path-to-sootOutput/gnu .
cp -R ~/path-to-sootOutput/java .

and still fails, which indicates that it isn't a problem related to 
native code calling methods in untransformed classes and not finding 
them (I thought for a minute the answer to my second question was also 
the answer to my first question).

Is there some reason why inserting an empty method call (which actually 
gets recognized and transformed by me) before and after every method 
call would make SableVM fail if the transformation was applied to all of 
classpath?

Anyway, in the meantime, I'll look into the details of why I "cannot 
create vm" with gdb.

Cheers,
Chris

Re: [Sablevm-developer] using a Soot-transformed classpath

[Grzegorz B. P. <ga...@de...>]

W li=B6cie z pi=B1, 05-03-2004, godz. 23:45, Chris Pickett pisze:=20
> Is there some reason why inserting an empty method call (which actually=
=20
> gets recognized and transformed by me) before and after every method=20
> call would make SableVM fail if the transformation was applied to all o=
f=20
> classpath?

I am not sure what exactly you're trying to do, but be warned that
SableVM (just as probably any VM) is very picky about what can be
executed at bootstrap time. So if you could avoid doing the
transformations to the classes loaded during the bootstrap, it
could save you a lot of troubles.

> Anyway, in the meantime, I'll look into the details of why I "cannot=20
> create vm" with gdb.

try to strace it first (it's faster)

Does it fail after loading the first .class ?

					GBP

--=20
Grzegorz B. Prokopski <ga...@de...>
Debian GNU/Linux      http://www.debian.org
SableVM - LGPLed JVM  http://www.sablevm.org
Why SableVM ?!?       http://devel.sablevm.org/wiki/WhySableVM

Re: [Sablevm-developer] using a Soot-transformed classpath

[Etienne G. <gag...@uq...>]

On Fri, Mar 05, 2004 at 11:45:07PM -0500, Chris Pickett wrote:
> >I want to make SableVM use Soot-transformed versions of the non-native 
> >portion of Classpath.  I have the output from Soot in the application 
> >directory where I try to run SableVM, but the classes aren't getting 
> >loaded even with '-c .' specified as an option.

How about compiling a version of SableVM with
--enable-debugging-features and then launching sablevm like this:

 sablevm -v --property sablevm.verbose.methods=true ...

This should show you the complete sequence of called methods.  I guess
we should really document these boot-time properties somewhere.  Any
volunteer?

Etienne

-- 
Etienne M. Gagnon, Ph.D.             http://www.info.uqam.ca/~egagnon/
SableVM:                                       http://www.sablevm.org/
SableCC:                                       http://www.sablecc.org/

Re: [Sablevm-developer] using a Soot-transformed classpath

[Grzegorz B. P. <ga...@de...>]

W li=B6cie z sob, 06-03-2004, godz. 21:41, Etienne Gagnon pisze:=20
> On Fri, Mar 05, 2004 at 11:45:07PM -0500, Chris Pickett wrote:
> > >I want to make SableVM use Soot-transformed versions of the non-nati=
ve=20
> > >portion of Classpath.  I have the output from Soot in the applicatio=
n=20
> > >directory where I try to run SableVM, but the classes aren't getting=
=20
> > >loaded even with '-c .' specified as an option.
>=20
> How about compiling a version of SableVM with
> --enable-debugging-features and then launching sablevm like this:
>=20
>  sablevm -v --property sablevm.verbose.methods=3Dtrue ...
>=20
> This should show you the complete sequence of called methods.  I guess
> we should really document these boot-time properties somewhere.  Any
> volunteer?

Ehrm...! does _anybody_ read manual pages?! ;-)

$ man sablevm

(though suggestions/improvements are welcomed)

			GBP

PS: Assuming you have SableVM installed in some standard place so that
man command knew about sablevm.1 manual page... 'man ./doc/sablevm.1'
in SableVM sources should also work AFAIR.

--=20
Grzegorz B. Prokopski <ga...@de...>
Debian GNU/Linux      http://www.debian.org
SableVM - LGPLed JVM  http://www.sablevm.org
Why SableVM ?!?       http://devel.sablevm.org/wiki/WhySableVM

Re: [Sablevm-developer] using a Soot-transformed classpath

[Chris P. <chr...@ma...>]

Grzegorz B. Prokopski wrote:

>W liście z sob, 06-03-2004, godz. 21:41, Etienne Gagnon pisze: 
>  
>
>>On Fri, Mar 05, 2004 at 11:45:07PM -0500, Chris Pickett wrote:
>>    
>>
>>>>I want to make SableVM use Soot-transformed versions of the non-native 
>>>>portion of Classpath.  I have the output from Soot in the application 
>>>>directory where I try to run SableVM, but the classes aren't getting 
>>>>loaded even with '-c .' specified as an option.
>>>>        
>>>>
>>How about compiling a version of SableVM with
>>--enable-debugging-features and then launching sablevm like this:
>>
>> sablevm -v --property sablevm.verbose.methods=true ...
>>
>>This should show you the complete sequence of called methods.  I guess
>>we should really document these boot-time properties somewhere.  Any
>>volunteer?
>>    
>>
>
>Ehrm...! does _anybody_ read manual pages?! ;-)
>
>$ man sablevm
>
>(though suggestions/improvements are welcomed)
>  
>

not sure if you last changed it very recently, but for me, only 
sablevm.verbose.methods and sablevm.verbose.instructions are there.  
perhaps the best solution is to generate that portion of the file from 
vm_args.(m4?).c -- that way, it always stays up to date.

cheers,
chris

p.s.  java.lang.Object superclass problem is solved for me, but soot and 
d-java are still broken, for those who aren't on the soot list.

Re: [Sablevm-developer] using a Soot-transformed classpath

[Chris P. <chr...@ma...>]

Etienne Gagnon wrote:
> On Fri, Mar 05, 2004 at 11:45:07PM -0500, Chris Pickett wrote:
> 
>>>I want to make SableVM use Soot-transformed versions of the non-native 
>>>portion of Classpath.  I have the output from Soot in the application 
>>>directory where I try to run SableVM, but the classes aren't getting 
>>>loaded even with '-c .' specified as an option.
> 
> 
> How about compiling a version of SableVM with
> --enable-debugging-features and then launching sablevm like this:
> 
>  sablevm -v --property sablevm.verbose.methods=true ...

i use vm->verbose_methods and vm->verbose_instructions all the time :(

i should have reported more details at first; i was just wondering if 
somebody knew off the top of their head why it fails.

anyway, to make it quick:  i followed it through with gdb and it dies 
loading java.lang.Object because apparently java.lang.Object has a 
superclass (relevant code follows).

if i just replace java.lang.Object with the orignal, then it dies 
whenever it tries to load Spmt.class (which can be deferred by copying 
over more non-transformed classes).

i found that if i keep the original Object.class, VMClass.class, and 
Class.class files, and also put Spmt.class in the root of classpath, 
then it works fine.  anything less is insufficient.

however, i would really like to have my special instructions inserted 
around every method call -- i want to gather statistics on the number of 
calls we can successfully predict when the whole of classpath is 
transformed.

is this a soot bug?  generating code for java.lang.Object that declares 
java.lang.Object as a superclass?  or is there something particular to 
what i'm doing that forces java.lang.Object as a superclass (i don't see 
it ...)

i checked the jasmin disassembly of the transformed java.lang.Object 
classfile and it does indeed declare '.super java/lang/Object'

cheers,
chris

==========================================

static jint
_svmf_bootcl_resolve_super_class (_svmt_JNIEnv *env, _svmt_class_info 
*class)
{
   _svmt_JavaVM *vm = env->vm;
   _svmt_CONSTANT_Class_info **cp_super_class = class->super_class;

   assert (class->class_loader_info->class_loader == NULL);

   /* no super class -> we're done */
   if (CANNOT_DREF (cp_super_class))
     {
       /* this must be "java/lang/Object" */
       /* and it must be a public non-abstract non-final class */
       if (strcmp (class->name, "java/lang/Object") != 0 ||
           class->class_loader_info->class_loader != NULL ||
           (!_svmf_is_set_flag (class->access_flags, SVM_ACC_PUBLIC)) ||
           _svmf_is_set_flag (class->access_flags, SVM_ACC_FINAL) ||
           _svmf_is_set_flag (class->access_flags, SVM_ACC_INTERFACE) ||
           _svmf_is_set_flag (class->access_flags, SVM_ACC_ABSTRACT))
         {
           _svmf_error_VerifyError (env);
           return JNI_ERR;
         }

       return JNI_OK;
     }

   if (DREF (cp_super_class, tag) != SVM_CONSTANT_Class ||
       CANNOT_DREF (DREF (cp_super_class, name)) ||
       DREF (DREF (cp_super_class, name), tag) != SVM_CONSTANT_Utf8 ||
       DREF (DREF (cp_super_class, name), value)[0] == '[')
     {
       _svmf_error_ClassFormatError (env);
       return JNI_ERR;
     }

   {
     _svmt_type_node type = { NULL, NULL, NULL, NULL, NULL };

     type.name = DREF (DREF (cp_super_class, name), value);

     if (_svmm_tree_find_type
         (vm->class_loading.boot_loader.partially_derived_type_tree,
          &type) != NULL)
       {
=>      _svmf_error_ClassCircularityError (env);
         return JNI_ERR;
       }
   }

Re: [Sablevm-developer] using a Soot-transformed classpath

[Clark V. <cl...@CS...>]

> is this a soot bug?  generating code for java.lang.Object that declares 
> java.lang.Object as a superclass?  or is there something particular to 
> what i'm doing that forces java.lang.Object as a superclass (i don't see 
> it ...)
> 
> i checked the jasmin disassembly of the transformed java.lang.Object 
> classfile and it does indeed declare '.super java/lang/Object'

The super_class field of the classfile for java.lang.Object should be 0
(ie invalid constant-pool ref), so that sounds like a soot bug.

-- 
ttfn,
clark
cl...@cs...


On Sun, 7 Mar 2004, Chris Pickett wrote:

> Etienne Gagnon wrote:
> > On Fri, Mar 05, 2004 at 11:45:07PM -0500, Chris Pickett wrote:
> > 
> >>>I want to make SableVM use Soot-transformed versions of the non-native 
> >>>portion of Classpath.  I have the output from Soot in the application 
> >>>directory where I try to run SableVM, but the classes aren't getting 
> >>>loaded even with '-c .' specified as an option.
> > 
> > 
> > How about compiling a version of SableVM with
> > --enable-debugging-features and then launching sablevm like this:
> > 
> >  sablevm -v --property sablevm.verbose.methods=true ...
> 
> i use vm->verbose_methods and vm->verbose_instructions all the time :(
> 
> i should have reported more details at first; i was just wondering if 
> somebody knew off the top of their head why it fails.
> 
> anyway, to make it quick:  i followed it through with gdb and it dies 
> loading java.lang.Object because apparently java.lang.Object has a 
> superclass (relevant code follows).
> 
> if i just replace java.lang.Object with the orignal, then it dies 
> whenever it tries to load Spmt.class (which can be deferred by copying 
> over more non-transformed classes).
> 
> i found that if i keep the original Object.class, VMClass.class, and 
> Class.class files, and also put Spmt.class in the root of classpath, 
> then it works fine.  anything less is insufficient.
> 
> however, i would really like to have my special instructions inserted 
> around every method call -- i want to gather statistics on the number of 
> calls we can successfully predict when the whole of classpath is 
> transformed.
> 
> is this a soot bug?  generating code for java.lang.Object that declares 
> java.lang.Object as a superclass?  or is there something particular to 
> what i'm doing that forces java.lang.Object as a superclass (i don't see 
> it ...)
> 
> i checked the jasmin disassembly of the transformed java.lang.Object 
> classfile and it does indeed declare '.super java/lang/Object'
> 
> cheers,
> chris
> 
> ==========================================
> 
> static jint
> _svmf_bootcl_resolve_super_class (_svmt_JNIEnv *env, _svmt_class_info 
> *class)
> {
>    _svmt_JavaVM *vm = env->vm;
>    _svmt_CONSTANT_Class_info **cp_super_class = class->super_class;
> 
>    assert (class->class_loader_info->class_loader == NULL);
> 
>    /* no super class -> we're done */
>    if (CANNOT_DREF (cp_super_class))
>      {
>        /* this must be "java/lang/Object" */
>        /* and it must be a public non-abstract non-final class */
>        if (strcmp (class->name, "java/lang/Object") != 0 ||
>            class->class_loader_info->class_loader != NULL ||
>            (!_svmf_is_set_flag (class->access_flags, SVM_ACC_PUBLIC)) ||
>            _svmf_is_set_flag (class->access_flags, SVM_ACC_FINAL) ||
>            _svmf_is_set_flag (class->access_flags, SVM_ACC_INTERFACE) ||
>            _svmf_is_set_flag (class->access_flags, SVM_ACC_ABSTRACT))
>          {
>            _svmf_error_VerifyError (env);
>            return JNI_ERR;
>          }
> 
>        return JNI_OK;
>      }
> 
>    if (DREF (cp_super_class, tag) != SVM_CONSTANT_Class ||
>        CANNOT_DREF (DREF (cp_super_class, name)) ||
>        DREF (DREF (cp_super_class, name), tag) != SVM_CONSTANT_Utf8 ||
>        DREF (DREF (cp_super_class, name), value)[0] == '[')
>      {
>        _svmf_error_ClassFormatError (env);
>        return JNI_ERR;
>      }
> 
>    {
>      _svmt_type_node type = { NULL, NULL, NULL, NULL, NULL };
> 
>      type.name = DREF (DREF (cp_super_class, name), value);
> 
>      if (_svmm_tree_find_type
>          (vm->class_loading.boot_loader.partially_derived_type_tree,
>           &type) != NULL)
>        {
> =>      _svmf_error_ClassCircularityError (env);
>          return JNI_ERR;
>        }
>    }
> 
> 
> 
> 
> 
> 
> -------------------------------------------------------
> This SF.Net email is sponsored by: IBM Linux Tutorials
> Free Linux tutorial presented by Daniel Robbins, President and CEO of
> GenToo technologies. Learn everything from fundamentals to system
> administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
> _______________________________________________
> Sablevm-developer mailing list
> Sab...@li...
> https://lists.sourceforge.net/lists/listinfo/sablevm-developer
> 
> 

Re: [Sablevm-developer] using a Soot-transformed classpath

[Chris P. <chr...@ma...>]

Clark VERBRUGGE wrote:
>>is this a soot bug?  generating code for java.lang.Object that declares 
>>java.lang.Object as a superclass?  or is there something particular to 
>>what i'm doing that forces java.lang.Object as a superclass (i don't see 
>>it ...)
>>
>>i checked the jasmin disassembly of the transformed java.lang.Object 
>>classfile and it does indeed declare '.super java/lang/Object'
> 
> 
> The super_class field of the classfile for java.lang.Object should be 0
> (ie invalid constant-pool ref), so that sounds like a soot bug.


Eric Bodden on the Soot list said it might be because two copies of 
java.lang.Object are in my classpath.  I tried the most minimal 
classpath I could find, and still had this problem.

What's weird is that with D-Java from

http://www.netsw.org/softeng/lang/java/bytecode/disassembler/djava/

I get:

$ cd ~/lib/sablevm/classpath
$ D-Java java/lang/Object.class -o jasmin

;>> java/lang/Object.class <<
;
; Output created by D-Java (Mar  8 2004)
; mailto:ums...@cc...
; Copyright (c) 1996-1997 Shawn Silverman
;

;Classfile version:
;    Major: 45
;    Minor: 3

.source Object.java
.class  public java/lang/Object
; ACC_SUPER bit is set
.super  java/lang/Object
^^^^^^^^^^^^^^^^^^^^^^^^

[...]

on a newly-installed classpath, which really doesn't make sense to me. 
Furthermore, when I step through this with gdb and sablevm, it doesn't 
think java.lang.Object has a superclass.  I installed D-Java to get 
straight jasmin output and try to pin down this problem -- Soot does 
some rearranging of instructions before spitting out Jasmin.

Can somebody else with sablevm-classpath installed verify this?  It 
takes less than 5 minutes.  I'm going a bit crazy with all these 
classpath issues.

Cheers, and thanks,
Chris

Re: [Sablevm-developer] using a Soot-transformed classpath

[Grzegorz B. P. <ga...@de...>]

W li=B6cie z pon, 08-03-2004, godz. 15:07, Chris Pickett pisze:=20
> Clark VERBRUGGE wrote:
> >>is this a soot bug?  generating code for java.lang.Object that declar=
es=20
> >>java.lang.Object as a superclass?  or is there something particular t=
o=20
> >>what i'm doing that forces java.lang.Object as a superclass (i don't =
see=20
> >>it ...)
> >>
> >>i checked the jasmin disassembly of the transformed java.lang.Object=20
> >>classfile and it does indeed declare '.super java/lang/Object'
> >=20
> >=20
> > The super_class field of the classfile for java.lang.Object should be=
 0
> > (ie invalid constant-pool ref), so that sounds like a soot bug.
>=20
>=20
> Eric Bodden on the Soot list said it might be because two copies of=20
> java.lang.Object are in my classpath.  I tried the most minimal=20
> classpath I could find, and still had this problem.

> Can somebody else with sablevm-classpath installed verify this?  It=20
> takes less than 5 minutes.  I'm going a bit crazy with all these=20
> classpath issues.

I've just checked that with Kaffe's java compiler (KCJ) and Sun's
javac and they all dissasemble to this:

.source Object.java
.class  public java/lang/Object
; ACC_SUPER bit is set
.super  java/lang/Object

I imagine that by default, if there's no superclass, it's assumed to
be java/lang/Object, no matter what. I guess it might need a workaround
in Soot then.

HTH

					GBP

--=20
Grzegorz B. Prokopski <ga...@de...>
Debian GNU/Linux      http://www.debian.org
SableVM - LGPLed JVM  http://www.sablevm.org
Why SableVM ?!?       http://devel.sablevm.org/wiki/WhySableVM

Re: [Sablevm-developer] using a Soot-transformed classpath

[David <db...@cs...>]

On Fri, Mar 05, 2004 at 10:16:01PM -0500, Chris Pickett wrote:
> Hi,
>=20
> I want to make SableVM use Soot-transformed versions of the non-native=20
> portion of Classpath.  I have the output from Soot in the application=20
> directory where I try to run SableVM, but the classes aren't getting=20
> loaded even with '-c .' specified as an option.
>=20

The -c options is for the app class loader.  The bootstrap classes path
is a compiled constant.


> What is the best way to do this?
>=20
> For an application that transforms just gnu.* and java.* I tried:
>=20
> cd ~/lib/sablevm/classpath
> mv gnu gnu-old
> mv java java-old
> ln -s ~/path-to-sootOutput/java
> ln -s ~/path-to-sootOutput/gnu
>=20
> but that didn't work (cannot create vm).

It would help if you would run with -v to see how far it goes.

Note that some of the Soot generated classes need to follow SableVM
requirements for some classes.  See bootstrap.m4.c.


>=20
> Also, on a related note -- does anyone know of a good way to catch thos=
e=20
> classes which are only referenced from native code and thus not found b=
y=20
> Soot?  (I'm assuming such classes exist for certain applications).=20
> Profiling SableVM for the application and then comparing against the=20
> Soot output for whole-program-including-classpath-transformation seems=20
> like the only realistic choice, but maybe I'm missing something (it's=20
> not difficult to do, but it's not a real analysis).
>=20

I guess you could do:
find . -name '*.c' | xargs grep 'FindClass'
if you have the source code of the apps.


David

---

David B=E9langer
Graduate Student
School of Computer Science
McGill University
Office: MC226

Web page:   http://www.cs.mcgill.ca/~dbelan2/
Public key: http://www.cs.mcgill.ca/~dbelan2/public_key.txt