Menu
▾
▴
Re: [Springframework-developer] Further Investigations into OOM Exceptions on Redeploy
|
From: <jue...@we...> - 2004-11-21 11:31:28
|
Indeed, it's simply the class loader not the getting garbage-collected; = usually, something still keeps a reference to the class loader. I was = initially confused about the reasons for such effects shown in = profilers, as indicated by some of my mails from earlier this year. = However, it turned out to be all caused by the class loader problem. =20 The main suspect is the JavaBeans Introspector, which keeps references = to all introspected classes and thus also to their class loader: If that = Introspector in the JVM class loader keeps a reference to a class from a = web app class loader, that web app class loader will never become = garbage-collected. =20 A further reason is a ThreadLocal that keeps a reference to an object = loaded in the web app class loader. If such a object is attached to a = thread from the server's thread pool, the thread will keep a reference = to the web app class loader beyond web app shutdown and thus prevent the = class loader from garbage collection. =20 We've cleaned everything we can in Spring, both on the JavaBeans = Introspector side (we immediately clear the JVM-wide cache there and = cache in the web app itself) and the ThreadLocal side (we always reset = ThreadLocals after processing). To the best of my knowledge, there's no = such leak left. =20 For Introspector-related problems in third-party libraries, Spring ships = an IntrospectorCleanupListener for web applications, to be registered as = listener in web.xml. This simply clears the entire JVM-level cache of = the JavaBeans Introspector on web app shutdown. =20 For ThreadLocals, there's nothing we can do to clean up after = third-party libraries, I'm afraid. So the only way to move forward there = is to make the developers aware of those issues in their products. =20 Juergen =20 ________________________________ Von: spr...@li... im Auftrag = von Guillaume Poirier Gesendet: So 21.11.2004 05:54 An: spr...@li... Betreff: Re: [Springframework-developer] Further Investigations into OOM = Exceptions on Redeploy It has nothing to do with what you call full objects contants. There's nothing different from what you call a full object contants and a String contants, except that your profiler might not show you the String contants... Such objects sticking around is a symptom of a ClassLoader leak, not a cause. The issue DOM4J has is its use of ThreadLocal for caching, it never clean it up. I just created an issue in DOM4J's bugtracking tool at sourceforge. (https://sourceforge.net/tracker/index.php?func=3Ddetail&aid=3D1070309&gr= oup_id=3D16035&atid=3D116035) I'm starting to write a tool to detect that kind of memory leak. So far = I never saw any profiler or heap analyser that is able to help at all for = that kind of leak, except may be finding that there's a leak. The problem = with those tools is that they only allow to create object graphs that exclude ClassLoaders and Class objects, and that particular leak comes direct or indirect from reference to thoses. I had some code to detect those from = a while ago, but it is far from user friendly, so I'm gonna try to have something that doesn't involve too much setup. Guillaume ----- Original Message ----- From: "Alef Arendsen" <al...@jt...> To: <spr...@li...> Sent: Saturday, November 20, 2004 9:03 PM Subject: RE: [Springframework-developer] Further Investigations into OOM Exceptions on Redeploy Didn't we come to conclusion some time ago that this is mainly a dom4j issue, with those full object constants? Alef p.s. mail from a couple of months ago: Op de een of andere manier worden full object constanten (private static final BlaType TRUE =3D new BLaType("true")) worden op de een of andere = manier niet ge-GC'ed als je de classloader discard. Dit geeft problemen bij re-deployment in Tomcat en eigenlijk bij elke willekeurige andere = servlet container (re-deployment is het discarding van een classloader die de = webapp heeft ingeladen). Uiteraard is re-deployment tijdens productie niet echt = ok, dus daar geeft het geen issues. Anyway, er is gekeken naar Spring en er is het een en ander veranderd = wat resource leaks in Spring voorkomt (zo goed als mogelijk), maar full = object constanten zijn nou eenmaal een common programming style (hibernate = gebruikt ze ook) dus het out-of-memory-probleem zal niet helemaal weg zijn zodra = we naar Spring 1.0.2 (morgen uit) upgraden. Hieronder concluderend verhaaltje van J=FCrgen. Later, Alef I've just spent about 10 hours profiling Spring, using the Image = Database and Petclinic samples. (BTW, I've used an evaluation version of = JProfiler from ej-technologies - nice product!) Although I still don't completely understand the garbage collection behavior, I've figured out the following issues. Each of them simply prevents the respective classes from getting garbage collected on destruction of the class loader (e.g. on Tomcat web app shutdown). - A classic singleton with a class variable holding the object. I've reworked GlobalAdvisorAdapterRegistry and SQLErrorCodesFactory to hold = the respective singleton as a WeakReference. - A static cache. I've reworked CachedIntrospectionResults to use a WeakHashMap with WeakReferences as values. - A ThreadLocal with a default other than null. I've reworked TransactionSynchronizationManager to use null as default for the = resource map, setting a HashMap there on demand, removing the entire HashMap when unbinding the last resource. - Constants that define a full object. We have a number of those, for example ClassFilters.TRUE and AdvisedSupport.EMPTY_TARGET_SOURCE. I've = tried for quite a while, but I haven't been able to figure out a way to define such constants such that they will be garbage collected. The latter programming style is not uncommon, so I really don't = understand why it causes trouble with garbage collection. Hibernate uses a similar style for its FlushMode, for example. In general, other frameworks like CGLIB, Hibernate, Velocity have huge resource leaks on web app shutdown, while just the constants issue = remains with Spring now. As long as those huge third-party leaks are not = addressed, I'm not worried at all by the single remaining Spring issue. As I initially said, we shouldn't exaggerate the problem, as it = basically just affects hot reloading of web apps - mainly a development feature anyway. We need to make that clear to users too, to avoid comments a la "Spring is not usable for real apps because it leaks on hot = redeployment". Juergen =3D=3D JTeam B.V. Donker Curtiusstraat 7-412 1051 JL Amsterdam T: +31 20 486 20 36 M: +31 6 24 11 1996 F: +31 84 837 00 00 E: al...@jt... <mailto:al...@jt...> W: http://www.jteam.nl <http://www.jteam.nl> ________________________________ From: spr...@li... on behalf of Paul Buying Sent: Sun 11/21/2004 2:45 AM To: spr...@li... Subject: Re: [Springframework-developer] Further Investigations into OOM Exceptions on Redeploy Hi Seth, I'm experiencing exactly the same problem in a Hibernate/Spring = situation in our project. I'm able to redeploy somewhere around 25 times, then we = need to start all over. The problem occurs both with deployment on Tomcat and = JBoss. Next to Hibernate we also indeed use cglib and dom4j. Let me know if I can help by describing our situation here, or something else. Rgds, Paul. Paul Buying On 20 nov 2004, at 1:06, Seth Ladd wrote: I wanted to send out this quick FYI. We have a new Spring/Hibernate webapp starting up. I used this opportunity to look into OOM Exceptions on redeploy (again). Without Hibernate SessionFactory in my applicationContext, I seem to be able to redeploy all day long. Once I add in the SessionFactory, I can redeploy only 10-15 times before OutOfMemoryException. The problem definitely seems to be from Hibernate, or cglib, or dom4j. I'm going to try Guillaume Poirier's listener that cleans up Dom4J. But, the good news is that it appears that Spring is good at not leaving ClassLoaders around. I'll post anything if I found out more. Thanks, Seth ------------------------------------------------------- This SF.Net email is sponsored by: InterSystems CACHE FREE OODBMS DOWNLOAD - A multidimensional database that combines robust object and relational technologies, making it a perfect match for Java, C++,COM, XML, ODBC and JDBC. www.intersystems.com/match8 _______________________________________________ Springframework-developer mailing list Spr...@li... https://lists.sourceforge.net/lists/listinfo/springframework-developer ------------------------------------------------------- This SF.Net email is sponsored by: InterSystems CACHE FREE OODBMS DOWNLOAD - A multidimensional database that combines robust object and relational technologies, making it a perfect match for Java, C++,COM, XML, ODBC and JDBC. www.intersystems.com/match8 _______________________________________________ Springframework-developer mailing list Spr...@li... https://lists.sourceforge.net/lists/listinfo/springframework-developer |
