[Jython-dev] weak reference support in JyNI

[Stefan R. <Ste...@gm...>]

Hello,

I am currently working out the support of garbage collection and weak references in JyNI. For weak references, I would use the _weakref module that is already included in Jython, but I need to tweak it a bit, because JyNI needs to know, whether a PyCPeer (i.e. a subclass of PyObject that JyNI uses to wrap native objects in some cases) is weakly referenced. The reason for this is - roughly speaking - that in case of weak references, the native object should keep the peer alive (i.e. use a global ref in JNI), while normal references should use a peer that keeps the native object alive and not vise versa (i.e. use a weak global ref in JNI). So there will be potentially 2 versions of PyCPeer having the keep-alive relation
PyCPeer ---keeps alive---> native object ---keeps alive---> PyCPeer
Normal references should refere to the leftmost one while weak references should refere to the rightmost one (the python level equality function will report them as equal).

Of course the default implementation of _weakref is not aware of this. So I made up roughly two (three) competing ideas how to do it.

0) The official way to get those weak references that point to a specific object is WeakrefModule.getweakrefs(...)
However I would have to poll that method, which is an unprecise and unefficient solution. (So I don't count it as solution)
Once a new weak reference to a PyCPeer would show up, I would replace its referent by the right version. Since GlobalRef extends java.lang.ref.WeakReference, which does not allow to modify the referent, this would involve creating a new GlobalRef and replacing the old one. This is rather tweaky since that is stored in a private field (i.e. I would use reflection with setAccessible(true); or let native code do the operation).

1) Use org.python.modules._weakref.GlobalRef.references to keep track.
I would write a List implementation that wraps some other list as backend but reports modifications to a listener. I would use the original org.python.modules._weakref.GlobalRef.references as backend and insert my custom list into the org.python.modules._weakref.GlobalRef.references-field. Since it is a private field, this would be tweaky again, but could be done the same way as mentioned above. The rest would be like in "solution" 0) but the polling would be avoided.

2) Write an adjusted version of the _weakref-module (i.e. that checks for PyCPeer BEFORE creating GlobalRefs - so it works completely without fumbling with private fields and avoids useless recreation of GlobalRefs).
I tried this first, because I consider it the least tweaky solution, but ran into problems I can't resolve. For testing and a proof of concept, I took the _weakref source code, put it into a new package JyNI._weakref and only modified the doc-string to be able to distinguish the modules.

Then I adjusted the JyNI-initializer as follows:

public void initialize(Properties preProperties, Properties postProperties, String[] argv, ClassLoader classLoader, ExtensiblePyObjectAdapter adapter)
{
		//Customize the _weakref module to JyNI-needs:
		String blti = (String) postProperties.get("python.modules.builtin");
		postProperties.put("python.modules.builtin", blti == null ? "_weakref:JyNI._weakref.WeakrefModule" : blti+",_weakref:JyNI._weakref.WeakrefModule");
		
		//init PySystemState:
		PySystemState initState = PySystemState.doInitialize(preProperties, postProperties, argv, classLoader, adapter);
		
//further initialization stuff...


This would overwrite the _weakref-builtin with my custom version. If I do

import _weakref
print _weakref.__doc__

it works nicely and indeed prints the doc-string of my custom module.
But if I do

from _weakref import ref
print ref
bla = "bla"
test = ref(bla)
print test

original Jython outputs:

<type 'weakref'>
<weakref at 0x2; to 'str' at 0x3>

(i.e. it works fine), while the output using my initializer is:

<type 'JyNI._weakref.ReferenceType'>
Traceback (most recent call last):
  File "/home/stefan/eclipseWorkspace/JyNI/JyNI-Demo/src/JyNIWeakRefTest.py", line 41, in <module>
    test = ref(bla)
TypeError: JyNI._weakref.ReferenceType(): expected 2-3 args; got 1


So obviously, my tweak fails to make up the right type for ref and yields no appropriate mro. However I have no clue how this comes, since Jython should initialize the custom module just the same way it would do with or.python.modules._weakref.WeakrefModule - it is absolutely identical apart from the doc-string and the package path (but that is adjusted in the properties).

Can someone tell me why approach 2) fails and maybe how to fix it? Or should I work out approach 1) instead? Or do you even have a better idea how to reach my goal?

Thanks in advance.

-Stefan