jython-dev

[Jython-dev] work in progress

[Samuele P. <pe...@in...>]

Hi.

I'm working on the precedence issue.
I hope to have finished for ~monday.

Everything we have discussed for it should be implemented, plus some
other bug fixes.

I also have implemented PackageManager through a hier. of abstract classes,
this is an hook for future support for load sets.

The patch/rewriting will be something finished but not definitive,
something we can try and discuss on.

We will have a symm. behaviour for dir/jar pkgs, I have merged
PyJavaDir- and PyJavaPackage.

Some questions:

I would like to distinguish the support for dir() for java packages from
the support for from <pkg> import *. I dislike the fact that dir
force lazy loading, because __dir__ is used by from import * code.
Lazy loading should be limited to the from import * case.
The fact is that I want to minimize the "potential" robustness issues
of the - in any case - necessary cache patch.
Any remark or hint on this is welcome?

I have read the archive of import-sig and looked at the version of imputil
distributed with Python 2.0.

Finn or somebody else knows what is the actual state of the issue?

will imputil substitute ihooks, the distributed version seems not finished
and contains todos and referees and (Finn's) notes about what should be done
to make it possibly work under J(P)ython.

If it will be the future imp hooks framework, not ignoring it for the design
of load sets is maybe a not bad design idea, but clearly imp hooks
and load sets are in some sense orthogonal and the latter very java specific.


[into the future]
My first look impression, is that imputil design and import-sig discussion
have ignored the jython side of the problem.

imputil should support RExec and the import practice of major py pkgs
and extensions plus freeze and try to offer the primitives for mimicking
the java load behaviours (url loading)....

jython imp should support jpythonc, maybe somekind of poor freeze, java 
reloading, mixing of classpath and sys.path ...

My impression is that both world have something to learn from each other,

for example jython is maybe the right ground to decide what could make sense
on sys.path (urls vs. 'importers' ...) and given the constraints for load sets
design, that the concept of virtual pkg can sometimes be useful.

On the other side jython lacks something like RExec, ok we have
java security but it is (very) difficult to use it for python code directly,
maybe we need an extension of RExec concepts that fits with java model:
but that's a real challenge (completely aware of that!) and maybe not that
much useful (very low prio ;)).

regards, Samuele

[Jython-dev] work in progress

[Samuele P. <pe...@in...>]

Hi.

Basically I have finished the tentative (open for discussion) patch/rewr.
for loading precedence and merging the PyJava*Package classes.
(I have setup also some hooks in PackageManager impl. and PyJavaPackage
for the future load sets).
I have to test it and then it will appear here.

I have some questions on assumptions I have made:

(I) For use as a 1st arg to a two args File ctr, an empty string is always
explicitly expanded in jython codebase, sometimes with
 if( dir.equals("") ) dir=".";
sometimes with
 if( dir.equals("") ) dir=<the value of System.getProperty("user.dir")>;
The two methods should be aquivalent, does someone know how could they
be different? which one should be preferred?
(e.g. are there problems with the first method on a Mac jvm?)

(II) a) I have fixed (I hope) the relative import in package modules bug
(runtime side, not jythonc yet),
I had to rewrite the three args imp.importName was code dealt only with
relative module import and not packages and was buggy wrt. to the top flag,

the runtime enforce and the old code assumed that if x.y.z is a key in 
sys.modules (x.y.z loaded) also any subseq (e.g. x,x.y) will be present as a key 
in sys.modules.
I have used that assumption too, and took care that the runtime enforce its
validity. Any one sees problem with that?

b) Is the following correct for CPython?, I have tryed to enforce it also
in jython:

p0,p1 are py packages

if I do the following in a p1.a module:

 import sys
 import p0.a.b.x
 from p0.a.b.y import val

the keys p1.sys and p1.p0 will be added to sys.modules with value None.
But no key of the form p1.p0.a... etc. (clearly the plain keys p0... and sys,
binding to the real modules will be added too).


regards, Samuele

Re: [Jython-dev] work in progress

[<bc...@wo...>]

[Samuele Pedroni]

>Basically I have finished the tentative (open for discussion) patch/rewr.
>for loading precedence and merging the PyJava*Package classes.
>(I have setup also some hooks in PackageManager impl. and PyJavaPackage
>for the future load sets).
>I have to test it and then it will appear here.
>
>I have some questions on assumptions I have made:
>
>(I) For use as a 1st arg to a two args File ctr, an empty string is always
>explicitly expanded in jython codebase, sometimes with
> if( dir.equals("") ) dir=".";
>sometimes with
> if( dir.equals("") ) dir=<the value of System.getProperty("user.dir")>;
>The two methods should be aquivalent, does someone know how could they
>be different? which one should be preferred?
>(e.g. are there problems with the first method on a Mac jvm?)

I think it would be better to let the File() ctor create and handle the
aspect of relative files. I.e. 

            if (dirName.length() == 0)
                dirName = null;
            File dir = new File(dirName, name);

It is not well defined, what will happen if the dirName == "", but a
null directory value will create a relative file.
Clearly both imp.loadFromPath and BytecodeLoader.open should use the
same algorithm. That they doesn't just show that noone have had the Big
Picture.

>(II) a) I have fixed (I hope) the relative import in package modules bug
>(runtime side, not jythonc yet),
>I had to rewrite the three args imp.importName was code dealt only with
>relative module import and not packages and was buggy wrt. to the top flag,
>
>the runtime enforce and the old code assumed that if x.y.z is a key in 
>sys.modules (x.y.z loaded) also any subseq (e.g. x,x.y) will be present as a key 
>in sys.modules.
>I have used that assumption too, and took care that the runtime enforce its
>validity. Any one sees problem with that?

No. From my understanding that is what sys.modules should contain.

>b) Is the following correct for CPython?, I have tryed to enforce it also
>in jython:
>
>p0,p1 are py packages
>
>if I do the following in a p1.a module:
>
> import sys
> import p0.a.b.x
> from p0.a.b.y import val
>
>the keys p1.sys and p1.p0 will be added to sys.modules with value None.
>But no key of the form p1.p0.a... etc. (clearly the plain keys p0... and sys,
>binding to the real modules will be added too).

That is indeed what CPython does. From the source, the None values
appears to some kind of optimizations, inserted just to show that these
module names does not exists. But it is difficult to be sure, the
dotted-import source code in CPython is not much better than Jython's.

regards,
finn

Re: [Jython-dev] work in progress

[<bc...@wo...>]

[cc'ed to Greg Stein]

[Samuele Pedroni]

>[..]
>I have read the archive of import-sig and looked at the version of imputil
>distributed with Python 2.0.
>
>Finn or somebody else knows what is the actual state of the issue?

It is my impression and recollection, that Greg Stein (the author of
imputil) wanted to move the implementation of dotted import into python
code. That should make it easier to change and experiment with. That
would have been a good and noble goal which could have helped JPython as
well. That was why I was watching the import-sig. 

Some design disagreements could not quite be worked out between Greg and
GvR. One of the issues that GvR wanted was the ability to use the
imputil to fake an import. Rather than running the code object and
creating an module object, the fake should just return the
text/identification of the imported module. That way freeze tools would
not have to re implement the entire dotted import algorithm all over
again. 

>will imputil substitute ihooks, the distributed version seems not finished
>and contains todos and referees and (Finn's) notes about what should be done
>to make it possibly work under J(P)ython.

I don't think that imputil will replace anything. At least not at the
moment. It is a useful frame work for making interesting import
features, like f.ex importing from a zip file or downloading module from
a CPAN library. 

>If it will be the future imp hooks framework, not ignoring it for the design
>of load sets is maybe a not bad design idea, but clearly imp hooks
>and load sets are in some sense orthogonal and the latter very java specific.

In its current state it is not the future of anything. It will stay a
somewhat obscure and undocumented module until Greg starts pushing it
again.

>[into the future]
>My first look impression, is that imputil design and import-sig discussion
>have ignored the jython side of the problem.

Yes. But then it is just our task to keep reminding the nice CPython
people that it also have to work with jython.  

>imputil should support RExec and the import practice of major py pkgs
>and extensions plus freeze and try to offer the primitives for mimicking
>the java load behaviours (url loading)....
>
>jython imp should support jpythonc, maybe somekind of poor freeze, java 
>reloading, mixing of classpath and sys.path ...
>
>My impression is that both world have something to learn from each other,

I agree.

>for example jython is maybe the right ground to decide what could make sense
>on sys.path (urls vs. 'importers' ...) and given the constraints for load sets
>design, that the concept of virtual pkg can sometimes be useful.
>
>On the other side jython lacks something like RExec, ok we have
>java security but it is (very) difficult to use it for python code directly,
>maybe we need an extension of RExec concepts that fits with java model:
>but that's a real challenge (completely aware of that!) and maybe not that
>much useful (very low prio ;)).

regards,
finn

Re: [Jython-dev] work in progress

[<bc...@wo...>]

[Samuele Pedroni ]

>I'm working on the precedence issue.
>I hope to have finished for ~monday.
>
>Everything we have discussed for it should be implemented, plus some
>other bug fixes.
>
>I also have implemented PackageManager through a hier. of abstract classes,
>this is an hook for future support for load sets.
>
>The patch/rewriting will be something finished but not definitive,
>something we can try and discuss on.
>
>We will have a symm. behaviour for dir/jar pkgs, I have merged
>PyJavaDir- and PyJavaPackage.

I'm looking forward to see it.

>Some questions:
>
>I would like to distinguish the support for dir() for java packages from
>the support for from <pkg> import *. I dislike the fact that dir
>force lazy loading, because __dir__ is used by from import * code.
>Lazy loading should be limited to the from import * case.
>The fact is that I want to minimize the "potential" robustness issues
>of the - in any case - necessary cache patch.
>Any remark or hint on this is welcome?

In principle, I don't see a problem if the "dir()" builtin on a java
package initialized the java classes, but your argumentation for doing
so is weak. The cache code must be robust in all cases. If it isn't, its
a bug that must be fixed.

Implementationwise, both "from xx import *" and "dir()" uses the
__dir__() hook, so do you propose make them diffrent?

regards,
finn