I'm the author of that tiny patch for 1522 (http://bugs.jython.org/issue1522). As far I understand it, there shouldn't be a need to do anything more elaborate to fix this bug than weakening the reference being held in class_to_type through a WeakHashMap.
Solution #3 might be still desirable to prevent monkeypatching of a given Java proxy from being visible to other instances of an interpreter. This was done earlier with the shadowing support for builtins (and somewhat, if minimally, documented by the tests in test_sys_jy.py). Such monkeypatching might be more common in the future, especially once custom proxy making is better supported (such as through clamp or other tools).
Hi, we have seen the same problems and implemented a work-around for
now. It is similar to solution #2 you mentioned.
You are right that it can cause problems clearing out this data. I
tried keeping track of all classes loaded in a particular interpreter
and then clearing it out after the interpreter has been cleaned up in
cleanup() but this caused problems, I didn't take time to dig deeper
to investigate more
Then I realized the main problem in our situation was the generated
java proxy classes. So, rather than tracking all the classes added to
class_to_type during an interpreter existence I only kept track of the
java proxy classes generated and cleared all those out when the
interpreter is cleaned up. This seems to be working well at the moment
Looks like someone just added a patch to issue 1522 related to
this...curious to know if that would work too...I also like the sound
of #3, seems like the most appropriate solution
On Mon, Jun 21, 2010 at 1:22 PM, Matt Brinkley <firstname.lastname@example.org> wrote:
> Hi there,
> In the past I have made some mailing list posts and filed bugs against
> jython w.r.t. its behavior in dynamic classloader environments like Tomcat
> or OSGi where classloaders can come and (more importantly) go at runtime.
> Not having heard any significant feedback from the jython dev community on
> this topic, I have been digging into the source code to try and find the
> root cause(s) and potential solutions.
> As I have stated previously, one of the biggest problem areas seems to be
> PyType.class_to_type - this is a static map that is added to but never
> removed from. It contains class objects both as keys in the map as well as
> referenced by some of the values (e.g. a python type that implements a java
> interface will hold a java proxy object which references that java interface
> by class.) In case it isn't obvious, this is not an OK thing to do in
> environments like Tomcat or OSGi when reloading classes - in this case the
> old classloader is removed and a new classloader is created to load the new
> versions of the classes. The old classloader cannot be garbage collected
> until all references to its classes are garbage collected, and this cannot
> happen if they are being referenced by a static map like class_to_type.
> Beyond leaking memory (and consuming permgen space) references to the old
> classes can lead to buggy behavior, since certain object instances may be
> operating on a mix of old and new classes.
> I have been thinking of a few workarounds/solutions here. Maybe someone on
> this mailing list can comment on these ideas or suggest other ones:
> (1) Make the "key" in the PyType.class_to_type map be a string rather than a
> class object: since there is no longer a reference to the java class, this
> would allow the classloader to be garbage collected, but may introduce some
> issues if the same-named class is loaded by two different classloaders,
> since they will be indistinguishable in the map. Also, this doesn't fully
> solve the problem because I have found that some of the "values" in the map
> also contain references to the java class.
> (2) Provide a method to explicitly clear out the class_to_type map: I would
> be happy to write code that calls this each time a classloader unload/reload
> occurs. It would be simple to write a function that clears out this map, but
> I am wondering what the ramifications of this would be (beyond the
> efficiency hit, which I am willing to take.) Can someone familiar with
> PyType help fill in some details here? While perusing through the code I
> noticed a lot of classes cache static instances of types returned from
> PyType.fromClass() (e.g. about 90 instances in the org.python.antlr package)
> - if we clear out and repopulate this map then there could be cases where a
> given python type has two PyType instances representing it in the heap. This
> could be a problem if anyone uses == to compare PyTypes (which does appear
> to happen in __builtin__.isinstance()) ... am I reading this right? Would
> this be a problem or not? Would there be any other issues?
> (3) Change class_to_type from a static map to a map associated with a
> specific PythonInterpreter (or PySystemState) object. This is my preferred
> approach, since I can just make sure I destroy all my interpreters any time
> classloaders are reloaded. This also seems like the best long-term solution,
> since static variables cause a *lot* of problems, especially with dynamic
> (4) Anyone have any other ideas?
> P.S.: If you are interested in this thread, please also take a look at bug
> #1522, which I recently commented on. This bug report shows that repeated
> execution of a jython script that includes a python class that implements a
> java interface or subclasses a java class will cause PyType.class_to_type to
> grow without bound, eventually exhausting all permgen space. For some reason
> a new proxy object is created every time the script is run and added to
> class_to_type with a unique name (e.g. MyJavaInterface$0 MyJavaInterface$1
> etc). I would really like to see this fixed as well, but I don't know what
> purpose the unique suffix serves ... maybe someone more knowledgeable with
> this code can explain the rationale for this and suggest a fix.
> ThinkGeek and WIRED's GeekDad team up for the Ultimate
> GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the
> lucky parental unit. See the prize list and enter to win:
> Jython-dev mailing list
ThinkGeek and WIRED's GeekDad team up for the Ultimate
GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the
lucky parental unit. See the prize list and enter to win:
Jython-dev mailing list