[Jython-dev] MINI-PEP: Alternative to jar scanning.

[<bc...@wo...>]

The problem of scanning jar files to discover the names of available
java packages comes up again and again in different embedded situations.
We have added some more sys.add_XXX() methods to aid with the task and
these methods have helped some, but we still have situations where the
application which embed jython doesn't have access to the file system.

Here I'll try to describe a solution that tries to improve the situation
further. It should be noted that I don't a solution that fixes all
problem and closes this FAQ forever.

I'll focus on the discovery of java packages. It is IMO far less
important to correctly discover the full contents of java packages. As a
result of my priorities, there are situations where dir(javapackage) and
"from java package import *" will return only a subset of the available
java classes defined in the javapackage.


The basis of my suggestion is a process of pre-indexing jar files for
the benefit of jython. The pre-indexing of performed by a tool that
re-writes the jar file with additional entries added in the META-INF
folder.

An index entry for some java package is the same as used in the cachedir
directory. If consists of a dictionary where the javapackage name is the
key and a comma separated lists of java classes is the string value. 

   { 'some.java.package' : 'Foo,Bar,Baz' }

(There is also support for differentiate between public and non-public
classes to support the respectJavaAccessibility option, but that is
encoded in the value string and we can ignore that for now)


The indexer tool will write a zip-entry for each java package discovered
with the name:

   META-INF/JYTHON-PACKAGE-some.java.package

The contents of the entries is the comma separated list of class names.


The JYTHON-PACKAGE-* entries will be used by the package-manager's
packageExists()


The tool can either index jar files one by one or it can index a series
of jar files and put the resulting index into one output jar.

Benefits:

A pre-indexed jar file can be used from jython without requiring that
the auto-indexer can see the .jar file on the class path.

The RunJar startup utility shows how /applications/ can be deployed
using the "java -jar" syntax. An index is generated of all the java
packages in the java system and stored in the main jar file together
with manifest file like this:

    Name: python.main
    Main-Script: mytest.py

    Name: python.options
    python-security-respectJavaAccessibility: true
    python-skipcachedir: true
    python-prepath:
    python-path: mytest.jar

(notice that the jar file that includes the $py.class files must be
available on sys.path. We can't load python modules from resources yet)

An application deployed this way does not have to create a cachedir.


Problems:

- Because the JYTHON-PACKAGE-* files is found as resources, only one
version of a package index can be loaded. If the classes in a
javapackage is split over more than one jar file, only the class files
in the first jar will be loaded with "import *" and show up in
dir(javapackage).

- Subpackages does not show with "import *" or dir(javapackage).

- The tool only scan jar and zip files when making the index. It does
not scan directories on the class path.

- Python module can not be loaded as resources, so none of this helps
with the applet situation.


A preliminary patch is available here:

> http://sourceforge.net/tracker/index.php?func=detail&aid=454329&group_id=12867&atid=312867

Comments are welcome.

regards,
finn