Re: [Jython-dev] *loading: somehow last mail for this week

[Samuele P. <pe...@in...>]

Hi.
This is very likely the last mail of this week.
From tomorrow up to monday I will not be online.
Maybe I will be able to read the mails through the mailing lists
pages at sourceforge.

This discussions has treated the future support for reloading,
solving the broken sys.path java loading, fixing compiler.*Maker bugs,
and defining a cleaner semantic for loading, in particular sys.path java
and python loading.

(a)
For the sys.path java loading there were(are) 2 options:
things should work as if sys.path were virtually appended to
CLASSPATH or we should use the dynamic __path__ values.

Finn and I agreed on the first option, and we have further developed it.
Barry?

Precedence should also be fixed.


(a''...')
[Finn]
> >But there is another issue: I have not yet analyzed it in depth.
> >
> >One can compile a set of py modules/py pkg under a user-defined
> >java pkg (package jythonc option) so for example py mod x.y
> >will be userpkg.x.y.
> >
> >If the code in interpreted mode load a java class x.JClass what should do
> >at runtime the same code when compiled and put in the pkg userpkg,
> >just still try to load x.JClass or userpkg.x.JClass, try both.
> 
> x.JClass IMO. The --package option to jpythonc should only be for the
> generated code, not for the entire world.

Rethinking about the problem I agree with you, just x.JClass.
There is no need to stress on compatibility for sys.path loading porting from 
interp. to compiled mode when one uses --package, this would be difficult
to achieve and I feel in any case confusing for somenone+bad.
This is also coherent with the principle that sys.path java loading
is just a feature for people in hurry and prototyping.

(b)
[Finn]
> I can make this change. Py.initProxy will then get a new "String[]
> modules" argument. I'll make it a patch,, i.e I will not commit this
> until we have settled the complete semantic change.

Great. (I'm sure you already know it) Also Py.runCode
and Py.initProperties - I think - are concerned by this.

(cache options: technical)
This will introduce the cache fixing issue:

i) we cannot just use a Class->PyJavaClass cache, because this will
not support lazy loading (from jpkg import *).

ii) there are 2 options
  1) just a (classloader,classname)->PyJavaClass cache
  2) a Class->PyJavaClass and a cache like 1) that should be kept synchronized.
     The (classloader,classname) cache will only be filled with caches
     that can incur in lazy loading.
     
  (1) is simpler to implent, but less efficient than (2)
  (2) is more complicated and error-prone
  
iii) the main problem is that we should be able to anticipate which classes
can incur in lazy loading and their "classloaders":

  jclass=loader.loadClass("...)

in general we will not have jclass.getClassLoader()==loader (*)

In java2 the classloaders are organized in parent-child hierarchies,
with java11 classloaders world is flat, but this does not mean that
(*) always holds.

If we can have full control on things (f.ex. the load sets) we can
assume (*) to be true but that's not the case in general:
a way to proceed is the following:

when we anticipate the "classloader" setting up things for lazy loading
we anticipate loader.

But when we receive a loaded jclass we should be able to map its classloader
back to the guesses we make in the lazy loading setup case. And we have
also just the classloader to decide if a class can incur in lazy loading
because classes can just arrive from the pure java side of the world.

In order to do this we should make some assumptions (no solution in this
context can be strictly robust, with (2) we can add an option
that disable lazy loading and so the guess machinary when one knows
that things can go wrong) and should work for the common and
usual cases.

What we need is a way to guess and a mapping from actual classloaders
to the guesses:
here is a proposal and discussion starting point:

I assume that sys.classLoader is not a hook (but in this case we should
better protect it) and is set only once, if this is not the case things
become more complicated (maybe impossible).

 normal lazy loading (the one that we have now and that will remain): 
  guess = None // null
 lazy loading from the context of a load sets (in the future):
  guess = classloader-of-the-load-set
  // the framework should force (*) to be true
  
  
 mapping: 
   we need the helper: 
    
    fromhier(jclass,classloader):
     java11:
      return jclass.classLoader == classloader or jclass.classLoader == None
     java2:
      try:
       cl=jclass.classLoader
      except <security>:
       return 1
      if classloader == '<secured>':
       return 0
      try:
       while 1:
        if cl == classloader:
         return 1
        classloader = classloader.parent
      except <security>:
       pass
      return 0

   sys.classLoader is sys.classLoader
   rtloader is the loader of Python runtime (Py.class.getClassLoader(),
   '<secured>' if one cannot get it for security reasons in java2)
      
   the_mapping(jclass):
    if fromhier(jclass,rtloader) or fromhier(jclass,sys.classLoader):
     return None
    if jclass.classLoader is a load-set-classloader:
     return it
    return 'cannot incur in lazy loading'
      

I have the feeling we have clarified a lot of things, but maybe still
missed something. Thanks.

I'm sorry to have produced just bad english and no line of code,
but I hope this will change in the near future.

regards, Samuele