Re: [Springframework-developer] Cleanup of context resources on webapp reload

SourceForge Headquarters 1320 Columbia Street Suite 310 San Diego, CA 92101 +1 (858) 422-6466

Here is a good read about Reference API: http://www.javaworld.com/javaworld/jw-01-2002/jw-0104-java101.html

Dmitriy.

jürgen höller [werk3AT] wrote:

> Just finished tests on Resin 2.1.11 - exactly same behavior as with Tomcat. This *is* a general class loader respectively garbage collection issue, rather than a server-specific leak.
>  
> Consequently, with the changes I've committed yesterday, there is the same significant benefit as with Tomcat. The only remaining issue are the "full object" constants like ClassFilter.TRUE.
>  
> Juergen
>  
> 
> ________________________________
> 
> Von: spr...@li... im Auftrag von jürgen höller [werk3AT]
> Gesendet: Mo 24.05.2004 09:20
> An: spr...@li...
> Betreff: Re: [Springframework-developer] Cleanup of context resources on webapp reload
> 
> 
> 
> FYI, I've tested both Tomcat 5.0.18 and 4.1.27 - same behavior with both.
> 
> BTW, a couple of related posts from the Resin mailing list:
> 
> http://www.caucho.com/support/resin-interest/0404/0117.html
> http://www.caucho.com/support/resin-interest/0111/0164.html <http://www.caucho.com/support/resin-interest/0111/0164.html>
> 
> Juergen
> 
> 
> ________________________________
> 
> Von: spr...@li... im Auftrag von jürgen höller [werk3AT]
> Gesendet: Mo 24.05.2004 08:42
> An: spr...@li...
> Betreff: Re: [Springframework-developer] Cleanup of context resources on webapp reload
> 
> 
> 
> I've told JProfiler to explicitly run garbage collection - a number of times, actually - before I've had a look at the heap. So I'm sure that those remaining objects were not garbage-collected, and probably would have stayed around in the VM forever...
> 
> I'm aware that it seems odd, but this issue just seems to affect specific static fields: BeanWrapperImpl's defaultEditors did not cause a leak, but CachedIntrospectionResults' classCache did. Normal constants or static logger fields didn't, but "full object" constants like ClassFilters.TRUE did.
> 
> CachedIntrospectionResults' classCache uses the Class as key; the value, a CachedIntrospectionResult object, also refers to the key Class. Consequently, I had to use a WeakHashMap *with WeakReferences as values* to see proper garbage collection. Note that this static cache contains instances of its containing class as values.
> 
> I've run all my tests x times to make sure that I could trust my eyes. I've even undone the WeakReference changes again, and voila, there were the leaks again. I'd be happy to learn more about how the garbage collector works here... All I can state at this point of time is that the changes did cause an obvious difference in terms of resource leaks.
> 
> Juergen
> 
> 
> ________________________________
> 
> Von: spr...@li... im Auftrag von Guillaume Poirier
> Gesendet: Mo 24.05.2004 05:17
> An: spr...@li...
> Betreff: Re: [Springframework-developer] Cleanup of context resources on webapp reload
> 
> 
> 
> The "resource leak" caused by a singleton when a webapp's classloader is
> thrown away is only temorary, the unused classes and the classloader will be
> eventually garbage collected and the resources will be freed.  The only
> thing that could prevent that is if there was a something in the server's
> classloader that still had a reference on an object or a class of the child
> classloader.
> 
> Unless there's a bug in Tomcat or in the application code, I really can't
> see how the classes won't be eventually garbage collected when the JVM needs
> memory.  And anyway, if there was indeed a leak because SQLErrorCodesFactory
> is a singleton, why wouldn't there be one for each static fields such as
> constants?
> 
> Are you sure that JProfiler does not disable garbage collecting in order to
> make its profiling?  I know that in many of the JVMPI method calls are done
> with garbage collecting off.  I suspect the above to be the cause of the
> "resource leak", rather than any singleton that Spring might be using.
> 
> Guillaume
> 
> ----- Original Message -----
> From: "jürgen höller [werk3AT]" <jue...@we...>
> To: <spr...@li...>
> Sent: Sunday, May 23, 2004 4:45 PM
> Subject: Re: [Springframework-developer] Cleanup of context resources on
> webapp reload
> 
> 
> I've just spent about 10 hours profiling Spring, using the Image Database
> and Petclinic samples. (BTW, I've used an evaluation version of JProfiler
> from ej-technologies - nice product!)
> 
> Although I still don't completely understand the garbage collection
> behavior, I've figured out the following issues. Each of them simply
> prevents the respective classes from getting garbage collected on
> destruction of the class loader (e.g. on Tomcat web app shutdown).
> 
> - A classic singleton with a class variable holding the object. I've
> reworked GlobalAdvisorAdapterRegistry and SQLErrorCodesFactory to hold the
> respective singleton as a WeakReference.
> 
> - A static cache. I've reworked CachedIntrospectionResults to use a
> WeakHashMap with WeakReferences as values.
> 
> - A ThreadLocal with a default other than null. I've reworked
> TransactionSynchronizationManager to use null as default for the resource
> map, setting a HashMap there on demand, removing the entire HashMap when
> unbinding the last resource.
> 
> - Constants that define a full object. We have a number of those, for
> example ClassFilters.TRUE and AdvisedSupport.EMPTY_TARGET_SOURCE. I've tried
> for quite a while, but I haven't been able to figure out a way to define
> such constants such that they will be garbage collected.
> 
> The latter programming style is not uncommon, so I really don't understand
> why it causes trouble with garbage collection. Hibernate uses a similar
> style for its FlushMode, for example.
> 
> In general, other frameworks like CGLIB, Hibernate, Velocity have huge
> resource leaks on web app shutdown, while just the constants issue remains
> with Spring now. As long as those huge third-party leaks are not addressed,
> I'm not worried at all by the single remaining Spring issue.
> 
> As I initially said, we shouldn't exaggerate the problem, as it basically
> just affects hot reloading of web apps - mainly a development feature
> anyway. We need to make that clear to users too, to avoid comments a la
> "Spring is not usable for real apps because it leaks on hot redeployment".
> 
> Please, everybody, give the current CVS head a sanity check tomorrow. There
> shouldn't be any issues: the test suite passes, the sample apps run
> properly. Still, I'd feel more comfortable if we make sure that no subtle
> side effects have been introduced.
> 
> For this reason, I will delay release 1.0.2 till tomorrow night.
> 
> Juergen
> 
> 
> ________________________________
> 
> Von: spr...@li... im Auftrag von
> jürgen höller [werk3AT]
> Gesendet: Sa 22.05.2004 15:52
> An: spr...@li...
> Betreff: Re: [Springframework-developer] Cleanup of context resources on
> webapp reload
> 
> 
> 
> So basically, all static caches cause resource leaks when restarting a
> Tomcat web app? I wonder why this happens... The class loader should
> completely dissolve all classes that it has loaded in its lifetime,
> including static caches. Or have I misunderstood something here? Anyone
> having in-detail experience with handling such a scenario?
> 
> Juergen
> 
> 
> ________________________________
> 
> Von: spr...@li... im Auftrag von
> Dmitriy Kopylenko
> Gesendet: Di 04.05.2004 18:02
> An: spr...@li...
> Betreff: Re: [Springframework-developer] Cleanup of context resources on
> webapp reload
> 
> 
> 
> Well,
> 
> for instance SQLErrorCodesFactory is a singleton(GoF, not Spring) which
> caches SQLErrorCodes internally in the Map with strong references. Again, I
> don't know if trying
> to use WeakHashMap there would do the trick...
> 
> Dmitriy
> 
> Tim Kettering wrote:
> 
> 
>>I looked at it some more this morning, and basically what I did was
>>start up tomcat w/ the webapp in the profiler, then after it was done
>>starting up I used tomcat's manager to stop the context.  This should
>>destroy all resources related to the context.   Here is a list of
>>spring related stuff that still were in memory after the context was
>>closed.  Other stuff was cleaned up just fine.
>>
>>org.springframework.beans.CachedIntrospectionResults
>>org.springframework.jdbc.support.SQLCodes
>>org.springframework.core.Constants
>>org.springframework.aop.framework.AdvisedSupport$1
>>org.springframework.aop.framework.adapter.BeforeAdviceAdapter
>>org.springframework.aop.framework.adapter.AfterReturningAdviceAdapter
>>org.springframework.jdbc.support.SQLErrorCodesFactory
>>org.springframework.transaction.support.TransactionSynchronizationManage
>>r$1
>>org.springframework.transaction.interceptor.RollbackRuleAttribute
>>org.springframework.aop.framework.adapter.ThrosAdviceAdapter
>>org.springframework.aop.Pointcut$1
>>org.springframework.aop.framework.adapter.GlobalAdvisorAdapterRegistry
>>
>>On May 3, 2004, at 3:28 PM, Dmitriy Kopylenko wrote:
>>
>>
>>>I'm just wondering, would the use of WeakHashMap in
>>>CachedIntrospectionResults help?
>>>
>>>Dmitriy.
>>>
>>>Tim Kettering wrote:
>>>
>>>
>>>>I posted this to the users list last week and did not receive any
>>>>reply  on it, so I'm posting it again here on the developer list, in
>>>>hopes i  could get an reply from someone here.   I'm trying to
>>>>determine if its  something I should be doing myself, or if hte
>>>>spring context should be  cleaning up those resources by itself on
>>>>the .close() call.  Further  profiling shows that there are
>>>>duplicate  instances of SQLError and  hibernate proxy classes hanging
>>>>around  afterwards too.  Other objects  do get cleaned up properly.
>>>>--------
>>>>Hi everyone,
>>>>We're (meaning me) looking into some resource leaks that are
>>>>occuring   when our webapp context gets reloaded.  I found that
>>>>context.close()   needs to be called on the destroy() method of
>>>>plugin we're using,  and  it works for a good majority of the objects
>>>>we were seeing  leaked, but  there are some objects that I'm unable
>>>>to make go away.  Object in  question is the:
>>>>org.springframework.beans.CachedIntrospectionResults
>>>>Whenever I reload the context - the profiler I'm using shows that I
>>>>have essentially a duplicate group of those objects (same instance
>>>>count) as the original, and successive reloads will continue to
>>>>duplicate this.
>>>>The profiler also shows the final reference to those objects like  this:
>>>>100% - 1008 bytes - 63 alloc.
>>>>org.springframework.context.support.ClassPathXmlApplicationContext.<in
>>>>it >
>>>>So basically I guess what I'm asking is for ideas or suggestions on
>>>>how  I could get those to clean up.  This bean doesnt show up in the
>>>>Spring  javadocs.  And looking in CVS says its a package level bean,
>>>>not for  application use, so I'm thinking that closing the context
>>>>should (in  theory) clean this up?   Thanks in advance.
>>>>-tim
>>>>-------------------------------------------------------
>>>>This SF.Net email is sponsored by: Oracle 10g
>>>>Get certified on the hottest thing ever to hit the market... Oracle
>>>>10g. Take an Oracle 10g class now, and we'll give you the exam FREE.
>>>>http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click
>>>>_______________________________________________
>>>>Springframework-developer mailing list
>>>>Spr...@li...
>>>>https://lists.sourceforge.net/lists/listinfo/springframework-developer
>>>
>>>
>>>
>>>
>>>-------------------------------------------------------
>>>This SF.Net email is sponsored by: Oracle 10g
>>>Get certified on the hottest thing ever to hit the market... Oracle
>>>10g. Take an Oracle 10g class now, and we'll give you the exam FREE.
>>>http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click
>>>_______________________________________________
>>>Springframework-developer mailing list
>>>Spr...@li...
>>>https://lists.sourceforge.net/lists/listinfo/springframework-developer
>>>
>>
>>
>>
>>-------------------------------------------------------
>>This SF.Net email is sponsored by: Oracle 10g
>>Get certified on the hottest thing ever to hit the market... Oracle 10g.
>>Take an Oracle 10g class now, and we'll give you the exam FREE.
>>http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click
>>_______________________________________________
>>Springframework-developer mailing list
>>Spr...@li...
>>https://lists.sourceforge.net/lists/listinfo/springframework-developer
> 
> 
> 
> 
> -------------------------------------------------------
> This SF.Net email is sponsored by: Oracle 10g
> Get certified on the hottest thing ever to hit the market... Oracle 10g.
> Take an Oracle 10g class now, and we'll give you the exam FREE.
> http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click
> _______________________________________________
> Springframework-developer mailing list
> Spr...@li...
> https://lists.sourceforge.net/lists/listinfo/springframework-developer
> 
> 
> 
> 
> -------------------------------------------------------
> This SF.Net email is sponsored by: Oracle 10g
> Get certified on the hottest thing ever to hit the market... Oracle 10g.
> Take an Oracle 10g class now, and we'll give you the exam FREE.
> http://ads.osdn.com/?ad_id149&alloc_id66&op=ick
> _______________________________________________
> Springframework-developer mailing list
> Spr...@li...
> https://lists.sourceforge.net/lists/listinfo/springframework-developer
> 
> 
> 
> 
> -------------------------------------------------------
> This SF.Net email is sponsored by: Oracle 10g
> Get certified on the hottest thing ever to hit the market... Oracle 10g.
> Take an Oracle 10g class now, and we'll give you the exam FREE.
> http://ads.osdn.com/?ad_id149&alloc_id66&op=ick
> _______________________________________________
> Springframework-developer mailing list
> Spr...@li...
> https://lists.sourceforge.net/lists/listinfo/springframework-developer
> 
> 
> 
> 
> -------------------------------------------------------
> This SF.Net email is sponsored by: Oracle 10g
> Get certified on the hottest thing ever to hit the market... Oracle 10g.
> Take an Oracle 10g class now, and we'll give you the exam FREE.
> http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click
> _______________________________________________
> Springframework-developer mailing list
> Spr...@li...
> https://lists.sourceforge.net/lists/listinfo/springframework-developer
> 
> 
> 
> 
> -------------------------------------------------------
> This SF.Net email is sponsored by: Oracle 10g
> Get certified on the hottest thing ever to hit the market... Oracle 10g.
> Take an Oracle 10g class now, and we'll give you the exam FREE.
> http://ads.osdn.com/?ad_id149&alloc_id66&op=ick
> _______________________________________________
> Springframework-developer mailing list
> Spr...@li...
> https://lists.sourceforge.net/lists/listinfo/springframework-developer
> 
> 
> 
> 
> -------------------------------------------------------
> This SF.Net email is sponsored by: Oracle 10g
> Get certified on the hottest thing ever to hit the market... Oracle 10g.
> Take an Oracle 10g class now, and we'll give you the exam FREE.
> http://ads.osdn.com/?ad_id149&alloc_id66&op=ick
> _______________________________________________
> Springframework-developer mailing list
> Spr...@li...
> https://lists.sourceforge.net/lists/listinfo/springframework-developer
> 
> 
> 
> 
> -------------------------------------------------------
> This SF.Net email is sponsored by: Oracle 10g
> Get certified on the hottest thing ever to hit the market... Oracle 10g. 
> Take an Oracle 10g class now, and we'll give you the exam FREE.
> http://ads.osdn.com/?ad_id149&alloc_id66&op=click
> _______________________________________________
> Springframework-developer mailing list
> Spr...@li...
> https://lists.sourceforge.net/lists/listinfo/springframework-developer