[Jython-dev] a java load (...) patch

[Samuele P. <pe...@in...>]

Hi.

I'm posting a (promised)

java load patch:

the attachments are both gzip compressed context diff patches

core.patch is for org/python/core
 * deletes PyJavaDirPackage.java
 * adds
    CachedJarsPackageManager.java
    PathPackageManager.java
    SysPackageManager.java
jythonc.patch is for Tools/jythonc
 * adds yapm.py

(sorry, the following is descriptive, not much technical, nor complete)

what it implies: (most of these things have been discussed here on the list)

 * should fix the (by now mostly broken)
 relative import from package modules to be more python compliant.

 * Sets up a new (more coherent) precedence for import/loading:

  py pkg/modules > jclasses from classpath > jclasses from sys.path .

 Virtually one can think that the dynamic value of sys.path is appended
 to classpath. Clearly this is true up to the fact that classes in a "same"
 package split over classpath and sys.path do not have mutual
 access to their package protected member. That because loading
 from classpath and from sys.path involve two different classloaders.
 There is no fix for this but such a splitting is just a bit ill-minded.
 Classes in a java package shadowed by a python pkg can be accessed through
 the python pkg.

 *There is just one internal representation for java packages
  (PyJavaPackage -- PyJavaDirPackage is no longer used/necessary):

  all the (public) classes in a package - that can span over jars in
  classpath and directories both in classpath and sys.path - are retrieved.

  dir(jpkg) reports all their names and all subpackages names (just
  meaning subdirs, any hint for a better  policy is welcome)
  but does not change the package __dict__.

  from jpkg import * also retrieves all classes/subpkgs.

  As previously jars are scanned for classes at startup, but
  one can add info about more packages and jars through
  sys.add_package and sys.packageManager.addJarToPackages at any time.
  Here jars can be specified also trhough urls.

  PyJavaPackage has an attribute __file__ which is either None
  or the filename of the jar that contains the package, if this is
  unique (dirs do not count). Better policies suggestions are welcome here
too.

 * hooks for the future:

   given the changes for java packages handling, PackageManager
   had to be partly rewritten.
   More radically I have re-factored it as a hierarchy of
   classes:
    PackageManager (abstract)
    CachedJarsPackageManager (abstract)
    PathPackageManager (abstract)
    SysPackageManager (somehow the equivalent of the old PackageManager).

   These should help implementing the future architecture for reloading.
   The design can change a bit but the public interfaces should be ok
   as they are, maybe just incremented. In any case they are not intended
   to be really visible from the jython-side.
   I have also put some introductive javadoc comments in front of them and
   of their new members.
   Also looking forward in this direction each PyJavaPackage
   can be constructed giving a controlling PackageManager
   to which it delegates most of the logic it needs.

  * where necessary, Class.forName has been substituted to deal with
sys.path java loading.

jythonc-side:
 I have fixed jythonc to work with the new runtime. This fixes
 are more "tentative" and maybe a better design is possible, or I could
 have missed something.
 The fixes for depend.py and jar.py are based on
ClassLoader.openResourceAsStream
 (such a method has also been added to BytecodeLoader) and on a parallel
 to system-wide package manager that gathers information about the
 non-public/non-plain classes in packages.
 This has been constructed in jython using the introduced abstract classes
for
 package managers. Its definition is in the new file yapm.py,
 yapm just stands for "yet another package manager",
 suggestions for a more explicative name are welcome.
 These two "hacks" avoided me the rewriting of classes info
gathering/reading logic.
 As consequence, when building a jar (-jar opt)
the --core/--all/--addpackages
 classes are collected from the whole classpath/sys.path respecting the
ususal
 precedence for java loading, if someone is working with a class that
patches
 a jar by coming first in classpath, the patched version will end up in the
 builded jar.

 Important: the jythonc patch prepend python.jythonc.classpath to the
classpath passed
 to java compiler, this should make more sense than whole substitution,
 and also sys.path - coherently with the new rules - is appended at the end.

Notice:

1) I have not yet fixed the package relative import in jythonc.
2) I have left around some comments of the form "// ??pending: ...", what
   they point out should not be critical (e.g how to deal
   with some minimal diff between the jvms? what should/should
   not be supported? etc.) hints on them are welcome.
3) the patch has been tested with Lib/test, a selection of Python 2.0 tests
   (e.g. test_pkg is ok), some of the demo (a little applet, javaclasses,
embed ...) and
   jythonc, plus some ad-hoc examples related to rel pkg import & java
loading.
4) I hope I avoided nasty new bugs ;)

regards, Samuele Pedroni