Re: [Springframework-developer] Further Investigations into OOM Exceptions on Redeploy

[Guillaume P. <gpo...@gl...>]

It has nothing to do with what you call full objects contants.  There's 
nothing different from what you call a full object contants and a String 
contants, except that your profiler might not show you the String 
contants...  Such objects sticking around is a symptom of a ClassLoader 
leak, not a cause.  The issue DOM4J has is its use of ThreadLocal for 
caching, it never clean it up.  I just created an issue in DOM4J's 
bugtracking tool at sourceforge. 
(https://sourceforge.net/tracker/index.php?func=detail&aid=1070309&group_id=16035&atid=116035)

I'm starting to write a tool to detect that kind of memory leak.  So far I 
never saw any profiler or heap analyser that is able to help at all for that 
kind of leak, except may be finding that there's a leak.  The problem with 
those tools is that they only allow to create object graphs that exclude 
ClassLoaders and Class objects, and that particular leak comes direct or 
indirect from reference to thoses.  I had some code to detect those from a 
while ago, but it is far from user friendly, so I'm gonna try to have 
something that doesn't involve too much setup.

Guillaume

----- Original Message ----- 
From: "Alef Arendsen" <al...@jt...>
To: <spr...@li...>
Sent: Saturday, November 20, 2004 9:03 PM
Subject: RE: [Springframework-developer] Further Investigations into OOM 
Exceptions on Redeploy


Didn't we come to conclusion some time ago that this is mainly a dom4j 
issue, with those full object constants?

Alef

p.s. mail from a couple of months ago:

Op de een of andere manier worden full object constanten (private static 
final BlaType TRUE = new BLaType("true")) worden op de een of andere manier 
niet ge-GC'ed als je de classloader discard. Dit geeft problemen bij 
re-deployment in Tomcat en eigenlijk bij elke willekeurige andere servlet 
container (re-deployment is het discarding van een classloader die de webapp 
heeft ingeladen). Uiteraard is re-deployment tijdens productie niet echt ok, 
dus daar geeft het geen issues.



Anyway, er is gekeken naar Spring en er is het een en ander veranderd wat 
resource leaks in Spring voorkomt (zo goed als mogelijk), maar full object 
constanten zijn nou eenmaal een common programming style (hibernate gebruikt 
ze ook) dus het out-of-memory-probleem zal niet helemaal weg zijn zodra we 
naar Spring 1.0.2 (morgen uit) upgraden.



Hieronder concluderend verhaaltje van Jürgen.



Later,



Alef





I've just spent about 10 hours profiling Spring, using the Image Database 
and Petclinic samples. (BTW, I've used an evaluation version of JProfiler 
from ej-technologies - nice product!)



Although I still don't completely understand the garbage collection 
behavior, I've figured out the following issues. Each of them simply 
prevents the respective classes from getting garbage collected on 
destruction of the class loader (e.g. on Tomcat web app shutdown).



- A classic singleton with a class variable holding the object. I've 
reworked GlobalAdvisorAdapterRegistry and SQLErrorCodesFactory to hold the 
respective singleton as a WeakReference.



- A static cache. I've reworked CachedIntrospectionResults to use a 
WeakHashMap with WeakReferences as values.



- A ThreadLocal with a default other than null. I've reworked 
TransactionSynchronizationManager to use null as default for the resource 
map, setting a HashMap there on demand, removing the entire HashMap when 
unbinding the last resource.



- Constants that define a full object. We have a number of those, for 
example ClassFilters.TRUE and AdvisedSupport.EMPTY_TARGET_SOURCE. I've tried 
for quite a while, but I haven't been able to figure out a way to define 
such constants such that they will be garbage collected.



The latter programming style is not uncommon, so I really don't understand 
why it causes trouble with garbage collection. Hibernate uses a similar 
style for its FlushMode, for example.



In general, other frameworks like CGLIB, Hibernate, Velocity have huge 
resource leaks on web app shutdown, while just the constants issue remains 
with Spring now. As long as those huge third-party leaks are not addressed, 
I'm not worried at all by the single remaining Spring issue.



As I initially said, we shouldn't exaggerate the problem, as it basically 
just affects hot reloading of web apps - mainly a development feature 
anyway. We need to make that clear to users too, to avoid comments a la 
"Spring is not usable for real apps because it leaks on hot redeployment".



Juergen





==
JTeam B.V.
Donker Curtiusstraat 7-412
1051 JL Amsterdam
T: +31 20 486 20 36
M: +31 6 24 11 1996
F: +31 84 837 00 00
E: al...@jt... <mailto:al...@jt...>
W: http://www.jteam.nl <http://www.jteam.nl>

________________________________

From: spr...@li... on behalf of 
Paul Buying
Sent: Sun 11/21/2004 2:45 AM
To: spr...@li...
Subject: Re: [Springframework-developer] Further Investigations into OOM 
Exceptions on Redeploy




Hi Seth,

I'm experiencing exactly the same problem in a Hibernate/Spring situation in 
our project. I'm able to redeploy somewhere around 25 times, then we need to 
start all over. The problem occurs both with deployment on Tomcat and JBoss.

Next to Hibernate we also indeed use cglib and dom4j.

Let me know if I can help by describing our situation here, or something 
else.

Rgds,
Paul.

Paul Buying

On 20 nov 2004, at 1:06, Seth Ladd wrote:


I wanted to send out this quick FYI. We have a new Spring/Hibernate
webapp starting up. I used this opportunity to look into OOM
Exceptions on redeploy (again).

Without Hibernate SessionFactory in my applicationContext, I seem to
be able to redeploy all day long. Once I add in the SessionFactory, I
can redeploy only 10-15 times before OutOfMemoryException.

The problem definitely seems to be from Hibernate, or cglib, or dom4j.
I'm going to try Guillaume Poirier's listener that cleans up Dom4J.

But, the good news is that it appears that Spring is good at not
leaving ClassLoaders around. I'll post anything if I found out more.

Thanks,
Seth


------------------------------------------------------- 
This SF.Net email is sponsored by: InterSystems CACHE
FREE OODBMS DOWNLOAD - A multidimensional database that combines
robust object and relational technologies, making it a perfect match
for Java, C++,COM, XML, ODBC and JDBC. www.intersystems.com/match8
_______________________________________________
Springframework-developer mailing list
Spr...@li...
https://lists.sourceforge.net/lists/listinfo/springframework-developer