[Jython-dev] poor man freezing and loading from other sources: proposals and ideas

[Samuele P. <pe...@in...>]

Hi.

Jython 2.0 more clean semantic wrt import and java loading
was an improvement over JPython. 
Now often more features have been requested in the directions of what has
been called:

- "poor man freezing" (the ability to take interpreter $py.class files
  and package them somehow for deployment)
- (java) loading from other sources (the possibility to have
   import working from jars or classloaders specified at runtime) .

Here are some ideas and proposals toward something concrete in those
directions for 2.1.

Note: jython 2.0 'jreload' module in principle can be used to enable
 imports from jars at runtime.

i) allow to have jars specified in sys.path from which both
   java classes or python modules can be imported.
 open issues:
  - will that produce comp. problem with pre-existent code
    that expects only dirs in sys.path?  
  - should the jars added to sys.path be kept open for performance
    reasons as long as they are in sys.path?
  - when should the "slow" scan for java packages be scheduled, when
    the jar is added to sys.path (more deterministic) or at the next 
    import, should the results be cached (IMHO no), should there be
    a tool to prescan a jar and store the information? in the
    normal jython cache? or together or inside the jar?
  
  Some of this behaviour require sys.path changes to have side effects,
  this could be difficult to obtain in a clean way.
  
ii) with i) jython -jar foo.jar will put foo.jar in sys.path ;)

iii) make jython behaviour even more regular:
   (some definitions:
     rt-classloader: is the classloader of jython runtime,
     sys.classLoader is sys.classLoader ;) )
   actually jython behaviour is as follows:
   sys.classLoader not set:
     java loading: rt-classloader > sys.path
     python loading: sys.path
   sys.classLoader set:
     java loading: sys.classLoader
     python loading: sys.path
   special case: jythonc compiled code (=> sys.classLoader set)
     java loading rt-classloader
     python loading: special from rt-classloader > sys.path
     but in this case as required sys.path==[]
     
   ideas:
     applicative code should be able to set sys.classLoader
     obtaining a meaningful behaviour (typically this should not
     be done or at most once but this should not be enforced),
     there should be a version of PythonInterpreter.initialize
     that takes a classLoader and set it in sys.classLoader
     
   new behaviour: always:
    java loading: rt-classloader > sys.classLoader (if set) > sys.path
    python loading: special* from rt-classloader & sys.classLoader > sys.path

   *) by now jythonc compiled code comes with an hardcoded list that specifies
     which python modules/packages should be loaded from java classloaders,
     this should be mantained and extended, see iv)
     
 iv) <poor man freezing>
 how to deal with java -jar foo.jar where foo.jar contains jython runtime,
 java classes using PythonInterpreter API and a set of packaged $py.class
 or .py files ; or the similar applet case, etc ?

 First it should be noted that in general we cannot assume that we can scan
 the jar at runtime.
 
 idea: have a tool that take a jar or a set of directories
 and collect which java packages-classes/python packages-modules are defined
 and produce a report in a textual format (for maximal flexibility).

 This report can be put in the jar and be read (impl/expl) by jython runtime
 as a _resource_  in    
 order to initiliaze its internal tables for special and java loading (see *)).
  
 open issues: $py.class files still require that proxies and adapters
 are built at runtime, so classloader creation permission is required,
 this should not be an issue for server side deployment or trusted
 applets, for normal applets jythonc solve this. Is this a serious 
 problem?

Note: the open issue is somehow related  (could be partially solved by) to the 
possibility of loading jythonc compiled code in the interpreter (this actually 
fails), some of the proposals here could help to construct a framework for that
but for the moment I'm not addressing such a situation.


v) eventually jreload.makeLoadSet can be extended to accept
 a classloader (this will not allow reloading but makes sense for
 the loading from other srcs issue) and load-sets could have add_package-
 like features.

I would appreciate some feedback about what is/is not necessary,
design ideas and input about the open issues.

Please, no pure "it would be nice if" requests.

regards,  Samuele Pedroni