jython-dev

Re: [Jython-dev] Jython and native java merged with simplicity

[Samuele P. <pe...@in...>]

Hi.

I'm still a bit confused, how far is this (up to PyObject subclassing and
other not directly related issues) from what you would like?

<jc.py>

import new

class X:
  def __init__(self,c):
    self.c = c
    
  def add(self,a,b):
  	return self.c*(a+b)
  	
  def mul(self,a,b):
  	return self.c*a*b

x=X(2)
print repr(x.__class__)
print X.add
print X.mul
print x.add(1,2)
print x.mul(3,2)

def jmerge(cls,jskel):
  new_cls = new.classobj(cls.__name__,(jskel,cls),{})
  new_cls.__module__ = cls.__module__
  return new_cls

import Xj

X=jmerge(X,Xj)
x=X(2)
print repr(x.__class__)
print X.add
print X.mul
print x.add(1,2)
print x.mul(3,2)
  	
print x.describe()

</jc.py>

Output:
<class __main__.X at 5783932>
<unbound method X.add>
<unbound method X.mul>
6
12
<class __main__.X at 3462044>
<java function add at 3199804>
<unbound method X.mul>
6
12
Xj(c=2)


Consider, Xj is the following java class:


public class Xj {

 public int c;
 
 public int add(int a,int b) {
   return c*(a+b);
 }


 public String describe() {
   return "Xj(c="+c+")";
 } 

}



regards, Samuele Pedroni.

Re: [Jython-dev] Jython and native java merged with simplicity

[Samuele P. <pe...@in...>]

Of course there is a problem with multiple inheritance from java classes,
but in any case I think this "skeleton: trick to be used parsimously and
with leaf classes.

About the idea of re-enabling subclassing from java classes in general:
what is the right thing to do with classes inherited along multiple paths?

Maybe it is possible to weaken the rule a bit:

if pyclass inherits from JClass
then 

class newclass(pyclass,JClass, unrelavant pure python classes)

is OK if JClass2 subclasses JClass.

regards, Samuele Pedroni.

Re: [Jython-dev] Jython and native java merged with simplicity

[Samuele P. <pe...@in...>]

...

> 
> 
> For demonstration purposes I'm embedding the feature
> in directly into the PyInstance.  I define a new
> private method called setSkeleton in PyInstance.java.
Make sense for demo, but for the long run is a no-go.

> 
> 
> I add a bit of code in __setattr__ and __findattr__
> which will search the skeleton instance first before
> self.  When setting the attribute I check to see if it
> is a "basis type" (i.e. float int string).  If it is
> not I must check that object to see if it has a
> skeleton.  If so, set the ghost attribute with the
> ob's skeleton.
> 
> 
> I include an extra optional argument to init args
> called __skeleton__.  If it's None I do nothing, if it
> is defined, I set the PyInstance's skeleton class
> before we assign attributes to the instance.  This way
> the skeleton will automatically be initialized since
> __setattr__ and __findattr__ have been slightly
> modified.  
> 
> 
> What is important about the nature of ghosting is that
> information is stored in both the java native skeleton
> and the real PyObject.
Why this two-places approach? once you have a skeleton
you can store things there, unless an attribute
is not a pre-existent java field.

> 
> This seems funny, and alot of work, but really isn't. 
> Once you see the speed boost obtained from doing bulk
> processing you may deem this feature worthwhile.
OK, but you stil pay reflection for calling java methods,
so you get a real speedup maybe only if your java methods
are coarse-grain.

> 
> 
> It does retain a great deal of python flexibility.  
> Provided a small set of attributes are not deleted. 
> That should throw an exception.
> 
> 
> thanks for your comments. have been very helpful.
> -john
You're welcome. 
For the momement I see this as an interesting experiment,
but to have such a functionality be distributed with jython
I think we need the 2.2 framework, in order to make
this a flavor of java subclassing under the control of a special-purpose
meta-class.
At least I hope that these words will make sense at that point.

regards, Samuele Pedroni.

Re: [Jython-dev] Jython and native java merged with simplicity

[john c. <joh...@ya...>]

thanks for the encouraging remarks!

these dev discussions have been really good.

lots of thinking going on...


I already have something preliminary whipped up.
And it is pretty simple like I though it would be.  As
expected the ghosting does not slow the interpreter
down since the method calls on the java skeleton are
fast.

Now the real speed up will be when I do benchmarking
of algorithms done in python against the same
algorithms implemented in the ghost class on ghost
attributes.


> Why this two-places approach? once you have a
> skeleton
> you can store things there, unless an attribute
> is not a pre-existent java field.

Your right!  No duplication at all.  A bit of tweaking
to dir function and __dict__.keys() things like that.
Doesn't look like this will break anything!


So far I haven't needed to use reflection.  I don't
think I will, since that step has already been done
for me by way of the interpreter.

The way that I am doing it now, for each attribute foo
in the skeleton class, one must define accessor
methods set_foo and get_foo.  Perhaps this is
cumbersome.


-john


__________________________________________________
Do You Yahoo!?
Make international calls for as low as $.04/minute with Yahoo! Messenger
http://phonecard.yahoo.com/

RE: [Jython-dev] Jython and native java merged with simplicity

[Garcia, M. <mg...@Bu...>]

I'm confused about this topic you guys are discussing.  Can you please
elaborate as to what benefits this would have or what purpose this will
serve?

thanks,
Mick 

-----Original Message-----
From: john coppola
To: Samuele Pedroni
Cc: jyt...@li...
Sent: 8/7/01 9:56 PM
Subject: Re: [Jython-dev] Jython and native java merged with simplicity

thanks for the encouraging remarks!

these dev discussions have been really good.

lots of thinking going on...


I already have something preliminary whipped up.
And it is pretty simple like I though it would be.  As
expected the ghosting does not slow the interpreter
down since the method calls on the java skeleton are
fast.

Now the real speed up will be when I do benchmarking
of algorithms done in python against the same
algorithms implemented in the ghost class on ghost
attributes.


> Why this two-places approach? once you have a
> skeleton
> you can store things there, unless an attribute
> is not a pre-existent java field.

Your right!  No duplication at all.  A bit of tweaking
to dir function and __dict__.keys() things like that.
Doesn't look like this will break anything!


So far I haven't needed to use reflection.  I don't
think I will, since that step has already been done
for me by way of the interpreter.

The way that I am doing it now, for each attribute foo
in the skeleton class, one must define accessor
methods set_foo and get_foo.  Perhaps this is
cumbersome.


-john


__________________________________________________
Do You Yahoo!?
Make international calls for as low as $.04/minute with Yahoo! Messenger
http://phonecard.yahoo.com/

_______________________________________________
Jython-dev mailing list
Jyt...@li...
http://lists.sourceforge.net/lists/listinfo/jython-dev

[Jython-dev] What is a Skeleton?

[john c. <joh...@ya...>]

Hello Michael.

What is a skeleton?

In much the same way a skeleton supports our body, a
skeleton java class will support the native python
class.  Python is amorphous and highly dynamic.  With
this flexibility there comes a penalty, lower speed. 
For most things, speed is not important but for other
things is it.


A skeleton gives our Python class a backbone. 
Even-though we can bend our bodies in many different
shapes, our skeleton prevents us from turning inside
out.


A skeleton is a set of attributes and or methods
implemented in the native java backbone.   They should
in theory do exactly the same thing as they do in the
python class (ghosting).  Of course this is up to the
programmer to verify by use of a good test suite.


Because of the way the Jython interpreter works (and
my implementation), Jython has absolutely no way of
knowing the difference between values stored in the
the java skeleton or the native Python object.

The benefit we get from this is that computationally
intensive algorithms could be ported to native java
while preserving the Python and Jython
interchangeability.  We implement that functionality
in the true python class anyway but if a skeleton
class exists, search the skeleton first for that
attribute.

Things are faster because:
  A) the skeleton class is native java.
  B) data is already available in native java form

The nice thing about it is that the Python class
should still work without the backbone.  The skeleton
has access to the data in native java form without
going through api layers.  We use the skeleton to
address efficiency issues.

Speed boost is enormous.  For a example, here are some
numbers for a simple program:

from time import time
def test():
    start = time()
    total = 0.0
    for i in range(0,1000000):
        total = total + i + i*2.0
    print time()- start

This test on my computer:
  python2.0:                     2.77 seconds
  jython2.1a1:                   7.40 seconds
  native java:                   0.014 seconds

Native java is:
* 197 times faster than Python
* 528 times faster than Jython
 # perhaps I should have used xrange (o:

A big light bulb went off.  I needed a way to harness
this power without sacrificing too much python
flexibility.


Here is an example of how they might be used together.

#a python class
class Spam:
    def __init__(self,aFloat,bInteger,cString):
        self.a = aFloat
        self.b = bInteger
        self.c = cString


//java Skeleton class
public class SpamSkeleton extends PyObject {
    private double a;
    private int b;
    private String c;

    public void set_c(PyString ob) {
        c=ob.internedString();
    }

    public String get_c() {
        return c;
    }

    public void set_b (PyInteger ob) {
        b=ob.getValue();
    }

    public int get_b () {
        return b;
    }

    public void set_a (PyFloat ob) {
        a = ob.getValue();
    }

    public double get_a() {
        return a;
    }
}
    

The concept is still in its conceptual stages.

from Spam import Spam
import SpamSkeleton
a=Spam(__skeleton__=SpamSkeleton())

I intercept the __skeleton__ keyword in the
constructor in PyInstance.  So even though it is not
defined in the __init__ function of Spam it will work.

To make this code portable:
from Spam import Spam

import sys
if sys.platform == "java":
   import SpamSkeleton
   a=Spam(__skeleton__=SpamSkeleton())
else:
   a=Spam()

Even though in the class self.a is defined, because
the skeleton has this attribute, it is never stored in
the PyInstance but instead the skeleton instance via
the getter and setter methods.

Thats all for now.  To really utilize the power I must
implement functionality that will allow nesting.  So a
list of objects with the restriction that they are of
the same class will becomes a list of skeletons in the
native java skeleton.  That's where the real
performance boost will be.

Will have something whipped up by next week with some
hard numbers for performance comparisons.


have fun,
John Coppola


__________________________________________________
Do You Yahoo!?
Make international calls for as low as $.04/minute with Yahoo! Messenger
http://phonecard.yahoo.com/

Re: [Jython-dev] Jython and native java merged with simplicity

[Samuele P. <pe...@in...>]

[Finn]
> I guess that you get a speed increase because the the test for skeleton
> is located early in the __findattr__ method. The result of such an early
> test is all non-skeleton instances will run a little bit slower, rigth?.
> If that is the case, you will have a tough job selling it.
> 
Yes, but the short-cutting PyInstance should not be the general PyInstance,
that's why I said maybe it can go in, once it will be clear how to express this
in terms of meta-classes.

My concern is that you still have to use reflection to call java code from
jython, so depending on the granularity of the logic that you have rewritten
in java you can get a speed-up or nothing you could even measure.

[Side-note: In general some kind of caching logic can give some similar benefits 
- in a more general setting - but then you have to deal with dynamic changes to 
classes, memory use and multi threading issues. ]

I agree with your opinion on the benchmark but I don't think it (the benchmark) 
is relevant to the problem. It s just a bit of partial sad "truth" exposed in an 
unfair way.

My initial understanding is that this could be used somehow to make some
final optimization using Java code over a design prototyped in pure jython.
But it is still fully unclear whether this buy us something concretely,
or is just (neat) hacking.

Further the idea of nesting, also skeletons all down the way, seems not 
workable.

regards, Samuele Pedroni.

Re: [Jython-dev] Jython and native java merged with simplicity

[john c. <joh...@ya...>]

Yes, something of this nature, but not quite like that
since several layers of code must still be traversed
before getting to the meat and potatoes slowing it
down. 


It is important that this interface be transparent
(like a ghost) since that is largely the beauty of it.



For demonstration purposes I'm embedding the feature
in directly into the PyInstance.  I define a new
private method called setSkeleton in PyInstance.java.


I add a bit of code in __setattr__ and __findattr__
which will search the skeleton instance first before
self.  When setting the attribute I check to see if it
is a "basis type" (i.e. float int string).  If it is
not I must check that object to see if it has a
skeleton.  If so, set the ghost attribute with the
ob's skeleton.


I include an extra optional argument to init args
called __skeleton__.  If it's None I do nothing, if it
is defined, I set the PyInstance's skeleton class
before we assign attributes to the instance.  This way
the skeleton will automatically be initialized since
__setattr__ and __findattr__ have been slightly
modified.  


What is important about the nature of ghosting is that
information is stored in both the java native skeleton
and the real PyObject.  The same information in two
places.  Somewhat of an OO no no.  But I am willing to
make this trade off to get 500x performance boost. 
Since all it requires is a bit a maintenance work in
well tucked away places.


This seems funny, and alot of work, but really isn't. 
Once you see the speed boost obtained from doing bulk
processing you may deem this feature worthwhile.


It does retain a great deal of python flexibility.  
Provided a small set of attributes are not deleted. 
That should throw an exception.


thanks for your comments. have been very helpful.
-john

--- Samuele Pedroni <pe...@in...> wrote:
> Hi.
> 
> I'm still a bit confused, how far is this (up to
> PyObject subclassing and
> other not directly related issues) from what you
> would like?
> 
> <jc.py>
> 
> import new
> 
> class X:
>   def __init__(self,c):
>     self.c = c
>     
>   def add(self,a,b):
>   	return self.c*(a+b)
>   	
>   def mul(self,a,b):
>   	return self.c*a*b
> 
> x=X(2)
> print repr(x.__class__)
> print X.add
> print X.mul
> print x.add(1,2)
> print x.mul(3,2)
> 
> def jmerge(cls,jskel):
>   new_cls =
> new.classobj(cls.__name__,(jskel,cls),{})
>   new_cls.__module__ = cls.__module__
>   return new_cls
> 
> import Xj
> 
> X=jmerge(X,Xj)
> x=X(2)
> print repr(x.__class__)
> print X.add
> print X.mul
> print x.add(1,2)
> print x.mul(3,2)
>   	
> print x.describe()
> 
> </jc.py>
> 
> Output:
> <class __main__.X at 5783932>
> <unbound method X.add>
> <unbound method X.mul>
> 6
> 12
> <class __main__.X at 3462044>
> <java function add at 3199804>
> <unbound method X.mul>
> 6
> 12
> Xj(c=2)
> 
> 
> Consider, Xj is the following java class:
> 
> 
> public class Xj {
> 
>  public int c;
>  
>  public int add(int a,int b) {
>    return c*(a+b);
>  }
> 
> 
>  public String describe() {
>    return "Xj(c="+c+")";
>  } 
> 
> }
> 
> 
> 
> regards, Samuele Pedroni.
> 



__________________________________________________
Do You Yahoo!?
Make international calls for as low as $.04/minute with Yahoo! Messenger
http://phonecard.yahoo.com/

Re: [Jython-dev] Jython and native java merged with simplicity

[Joseph G. <oc...@se...>]

Just a quick uneducated question:

I don't suppose there's anyway to use the java machinery by default but kick
in the full flexibility of python on an as-needed basis?  The reason I ask
is that I recall (over 10 years ago) one of the main Smalltalk implementers
(Douglas Engelbart?) giving a great talk on his Smalltalk.  They apparently
did just that: provided the illusion of a full object model while using raw
ints (etc.) under the covers.  According to his talk, when you really got
into the nitty-gritty, that emulation (which appeared hard on the surface)
was not a major challenge; instead, some unexpected stuff became hard!

So, I don't know how hard it is to emulate the full object model while
cheating behind the scenes, but it may be worth trying based on some decade
old Smalltalk experience.  :-)

Just a thought,

= Joe =


john coppola wrote:

> Yes, something of this nature, but not quite like that
> since several layers of code must still be traversed
> before getting to the meat and potatoes slowing it
> down.
>
> It is important that this interface be transparent
> (like a ghost) since that is largely the beauty of it.
>
> For demonstration purposes I'm embedding the feature
> in directly into the PyInstance.  I define a new
> private method called setSkeleton in PyInstance.java.
>
> I add a bit of code in __setattr__ and __findattr__
> which will search the skeleton instance first before
> self.  When setting the attribute I check to see if it
> is a "basis type" (i.e. float int string).  If it is
> not I must check that object to see if it has a
> skeleton.  If so, set the ghost attribute with the
> ob's skeleton.
>
> I include an extra optional argument to init args
> called __skeleton__.  If it's None I do nothing, if it
> is defined, I set the PyInstance's skeleton class
> before we assign attributes to the instance.  This way
> the skeleton will automatically be initialized since
> __setattr__ and __findattr__ have been slightly
> modified.
>
> What is important about the nature of ghosting is that
> information is stored in both the java native skeleton
> and the real PyObject.  The same information in two
> places.  Somewhat of an OO no no.  But I am willing to
> make this trade off to get 500x performance boost.
> Since all it requires is a bit a maintenance work in
> well tucked away places.
>
> This seems funny, and alot of work, but really isn't.
> Once you see the speed boost obtained from doing bulk
> processing you may deem this feature worthwhile.
>
> It does retain a great deal of python flexibility.
> Provided a small set of attributes are not deleted.
> That should throw an exception.
>
> thanks for your comments. have been very helpful.
> -john
>
> --- Samuele Pedroni <pe...@in...> wrote:
> > Hi.
> >
> > I'm still a bit confused, how far is this (up to
> > PyObject subclassing and
> > other not directly related issues) from what you
> > would like?
> >
> > <jc.py>
> >
> > import new
> >
> > class X:
> >   def __init__(self,c):
> >     self.c = c
> >
> >   def add(self,a,b):
> >       return self.c*(a+b)
> >
> >   def mul(self,a,b):
> >       return self.c*a*b
> >
> > x=X(2)
> > print repr(x.__class__)
> > print X.add
> > print X.mul
> > print x.add(1,2)
> > print x.mul(3,2)
> >
> > def jmerge(cls,jskel):
> >   new_cls =
> > new.classobj(cls.__name__,(jskel,cls),{})
> >   new_cls.__module__ = cls.__module__
> >   return new_cls
> >
> > import Xj
> >
> > X=jmerge(X,Xj)
> > x=X(2)
> > print repr(x.__class__)
> > print X.add
> > print X.mul
> > print x.add(1,2)
> > print x.mul(3,2)
> >
> > print x.describe()
> >
> > </jc.py>
> >
> > Output:
> > <class __main__.X at 5783932>
> > <unbound method X.add>
> > <unbound method X.mul>
> > 6
> > 12
> > <class __main__.X at 3462044>
> > <java function add at 3199804>
> > <unbound method X.mul>
> > 6
> > 12
> > Xj(c=2)
> >
> >
> > Consider, Xj is the following java class:
> >
> >
> > public class Xj {
> >
> >  public int c;
> >
> >  public int add(int a,int b) {
> >    return c*(a+b);
> >  }
> >
> >
> >  public String describe() {
> >    return "Xj(c="+c+")";
> >  }
> >
> > }
> >
> >
> >
> > regards, Samuele Pedroni.
> >
>
> __________________________________________________
> Do You Yahoo!?
> Make international calls for as low as $.04/minute with Yahoo! Messenger
> http://phonecard.yahoo.com/
>
> _______________________________________________
> Jython-dev mailing list
> Jyt...@li...
> http://lists.sourceforge.net/lists/listinfo/jython-dev

Re: [Jython-dev] Jython and native java merged with simplicity

[Kevin D. <kda...@we...>]

----- Original Message -----
From: "john coppola" <joh...@ya...>
To: <pe...@in...>; <jyt...@li...>
Sent: Tuesday, August 07, 2001 6:08 PM
Subject: Re: [Jython-dev] Jython and native java merged with simplicity


[snip]

> What is important about the nature of ghosting is that
> information is stored in both the java native skeleton
> and the real PyObject.  The same information in two
> places.  Somewhat of an OO no no.  But I am willing to
> make this trade off to get 500x performance boost.
> Since all it requires is a bit a maintenance work in
> well tucked away places.
>
>
> This seems funny, and alot of work, but really isn't.
> Once you see the speed boost obtained from doing bulk
> processing you may deem this feature worthwhile.

Hmm... It seems like what you're talking about can be already accomplished
with ExtensionClass (today) and possibly 2.2's class/type fix (tomorrow). I
haven't read much about the 2.2 functionality.

phabric's Acquisition module uses the ExtensionClass mechanism.

For convenience, Acquisition is implemented as a Python module:

http://cvs.phabric.org/index.cgi/phabric/py/Acquisition.py?rev=1.5&content-t
ype=text/x-cvsweb-markup

But, if you look at the ImplicitAcquisitionWrapper, the heavy lifting is
done completely in Java and with Java variables. When you have an instance
of an ImplicitAcquisitionWrapper, what you have is not an
org.python.core.PyInstance, but an org.phabric.ec.AqWrapper:

http://cvs.phabric.org/index.cgi/phabric/src/org/phabric/ec/AqWrapper.java?r
ev=1.3&content-type=text/x-cvsweb-markup

Currently, the ExtensionClass package will use reflection to automatically
find public methods, and it adds those methods to the class __dict__. It
would be possible to add support to ExtensionClass to do something similar
for attributes.

So, if you call foo.aq_acquire it looks in foo's __dict__, finds aq_acquire
and calls it. The search through the containment hierarchy happens
completely in Java, using Java's aq_self and aq_parent variables.

Rather than changing PyInstance itself, ExtensionClass just instantiates a
class that extends PyInstance.

Kevin

p.s. What I said above still stands, but I just realized that I haven't yet
committed the more complete form of aq_acquire that I've recently
finished...