Menu
▾
▴
springframework-developer
|
From: <jue...@we...> - 2004-05-22 13:55:53
|
So basically, all static caches cause resource leaks when restarting a = Tomcat web app? I wonder why this happens... The class loader should = completely dissolve all classes that it has loaded in its lifetime, = including static caches. Or have I misunderstood something here? Anyone = having in-detail experience with handling such a scenario? =20 Juergen =20 ________________________________ Von: spr...@li... im Auftrag = von Dmitriy Kopylenko Gesendet: Di 04.05.2004 18:02 An: spr...@li... Betreff: Re: [Springframework-developer] Cleanup of context resources on = webapp reload Well, for instance SQLErrorCodesFactory is a singleton(GoF, not Spring) which = caches SQLErrorCodes internally in the Map with strong references. = Again, I don't know if trying to use WeakHashMap there would do the trick... Dmitriy Tim Kettering wrote: > > I looked at it some more this morning, and basically what I did was=20 > start up tomcat w/ the webapp in the profiler, then after it was done=20 > starting up I used tomcat's manager to stop the context. This should=20 > destroy all resources related to the context. Here is a list of=20 > spring related stuff that still were in memory after the context was=20 > closed. Other stuff was cleaned up just fine. > > org.springframework.beans.CachedIntrospectionResults > org.springframework.jdbc.support.SQLCodes > org.springframework.core.Constants > org.springframework.aop.framework.AdvisedSupport$1 > org.springframework.aop.framework.adapter.BeforeAdviceAdapter > org.springframework.aop.framework.adapter.AfterReturningAdviceAdapter > org.springframework.jdbc.support.SQLErrorCodesFactory > = org.springframework.transaction.support.TransactionSynchronizationManage > r$1 > org.springframework.transaction.interceptor.RollbackRuleAttribute > org.springframework.aop.framework.adapter.ThrosAdviceAdapter > org.springframework.aop.Pointcut$1 > org.springframework.aop.framework.adapter.GlobalAdvisorAdapterRegistry > > On May 3, 2004, at 3:28 PM, Dmitriy Kopylenko wrote: > >> I'm just wondering, would the use of WeakHashMap in=20 >> CachedIntrospectionResults help? >> >> Dmitriy. >> >> Tim Kettering wrote: >> >>> I posted this to the users list last week and did not receive any=20 >>> reply on it, so I'm posting it again here on the developer list, in = >>> hopes i could get an reply from someone here. I'm trying to=20 >>> determine if its something I should be doing myself, or if hte=20 >>> spring context should be cleaning up those resources by itself on=20 >>> the .close() call. Further profiling shows that there are >>> duplicate instances of SQLError and hibernate proxy classes = hanging >>> around afterwards too. Other objects do get cleaned up properly. >>> -------- >>> Hi everyone, >>> We're (meaning me) looking into some resource leaks that are >>> occuring when our webapp context gets reloaded. I found that >>> context.close() needs to be called on the destroy() method of >>> plugin we're using, and it works for a good majority of the = objects >>> we were seeing leaked, but there are some objects that I'm unable >>> to make go away. Object in question is the: >>> org.springframework.beans.CachedIntrospectionResults >>> Whenever I reload the context - the profiler I'm using shows that I = >>> have essentially a duplicate group of those objects (same instance =20 >>> count) as the original, and successive reloads will continue to =20 >>> duplicate this. >>> The profiler also shows the final reference to those objects like = this: >>> 100% - 1008 bytes - 63 alloc. =20 >>> = org.springframework.context.support.ClassPathXmlApplicationContext.<in >>> it > >>> So basically I guess what I'm asking is for ideas or suggestions on=20 >>> how I could get those to clean up. This bean doesnt show up in the = >>> Spring javadocs. And looking in CVS says its a package level bean, = >>> not for application use, so I'm thinking that closing the context=20 >>> should (in theory) clean this up? Thanks in advance. >>> -tim >>> ------------------------------------------------------- >>> This SF.Net email is sponsored by: Oracle 10g >>> Get certified on the hottest thing ever to hit the market... Oracle=20 >>> 10g. Take an Oracle 10g class now, and we'll give you the exam FREE. = >>> http://ads.osdn.com/?ad_id=3D3149&alloc_id=3D8166&op=3Dclick >>> _______________________________________________ >>> Springframework-developer mailing list >>> Spr...@li... >>> = https://lists.sourceforge.net/lists/listinfo/springframework-developer >> >> >> >> >> ------------------------------------------------------- >> This SF.Net email is sponsored by: Oracle 10g >> Get certified on the hottest thing ever to hit the market... Oracle=20 >> 10g. Take an Oracle 10g class now, and we'll give you the exam FREE.=20 >> http://ads.osdn.com/?ad_id=3D3149&alloc_id=3D8166&op=3Dclick >> _______________________________________________ >> Springframework-developer mailing list >> Spr...@li... >> = https://lists.sourceforge.net/lists/listinfo/springframework-developer >> > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: Oracle 10g > Get certified on the hottest thing ever to hit the market... Oracle = 10g. > Take an Oracle 10g class now, and we'll give you the exam FREE. > http://ads.osdn.com/?ad_id=3D3149&alloc_id=3D8166&op=3Dclick > _______________________________________________ > Springframework-developer mailing list > Spr...@li... > https://lists.sourceforge.net/lists/listinfo/springframework-developer ------------------------------------------------------- This SF.Net email is sponsored by: Oracle 10g Get certified on the hottest thing ever to hit the market... Oracle 10g. Take an Oracle 10g class now, and we'll give you the exam FREE. http://ads.osdn.com/?ad_id=3D3149&alloc_id=3D8166&op=3Dclick _______________________________________________ Springframework-developer mailing list Spr...@li... https://lists.sourceforge.net/lists/listinfo/springframework-developer |
|
From: <jue...@we...> - 2004-05-23 20:46:58
|
I've just spent about 10 hours profiling Spring, using the Image = Database and Petclinic samples. (BTW, I've used an evaluation version of = JProfiler from ej-technologies - nice product!) =20 Although I still don't completely understand the garbage collection = behavior, I've figured out the following issues. Each of them simply = prevents the respective classes from getting garbage collected on = destruction of the class loader (e.g. on Tomcat web app shutdown). =20 - A classic singleton with a class variable holding the object. I've = reworked GlobalAdvisorAdapterRegistry and SQLErrorCodesFactory to hold = the respective singleton as a WeakReference. =20 - A static cache. I've reworked CachedIntrospectionResults to use a = WeakHashMap with WeakReferences as values. =20 - A ThreadLocal with a default other than null. I've reworked = TransactionSynchronizationManager to use null as default for the = resource map, setting a HashMap there on demand, removing the entire = HashMap when unbinding the last resource. =20 - Constants that define a full object. We have a number of those, for = example ClassFilters.TRUE and AdvisedSupport.EMPTY_TARGET_SOURCE. I've = tried for quite a while, but I haven't been able to figure out a way to = define such constants such that they will be garbage collected. =20 The latter programming style is not uncommon, so I really don't = understand why it causes trouble with garbage collection. Hibernate uses = a similar style for its FlushMode, for example. =20 In general, other frameworks like CGLIB, Hibernate, Velocity have huge = resource leaks on web app shutdown, while just the constants issue = remains with Spring now. As long as those huge third-party leaks are not = addressed, I'm not worried at all by the single remaining Spring issue. =20 As I initially said, we shouldn't exaggerate the problem, as it = basically just affects hot reloading of web apps - mainly a development = feature anyway. We need to make that clear to users too, to avoid = comments a la "Spring is not usable for real apps because it leaks on = hot redeployment". =20 Please, everybody, give the current CVS head a sanity check tomorrow. = There shouldn't be any issues: the test suite passes, the sample apps = run properly. Still, I'd feel more comfortable if we make sure that no = subtle side effects have been introduced. =20 For this reason, I will delay release 1.0.2 till tomorrow night. =20 Juergen =20 ________________________________ Von: spr...@li... im Auftrag = von j=FCrgen h=F6ller [werk3AT] Gesendet: Sa 22.05.2004 15:52 An: spr...@li... Betreff: Re: [Springframework-developer] Cleanup of context resources on = webapp reload So basically, all static caches cause resource leaks when restarting a = Tomcat web app? I wonder why this happens... The class loader should = completely dissolve all classes that it has loaded in its lifetime, = including static caches. Or have I misunderstood something here? Anyone = having in-detail experience with handling such a scenario? Juergen ________________________________ Von: spr...@li... im Auftrag = von Dmitriy Kopylenko Gesendet: Di 04.05.2004 18:02 An: spr...@li... Betreff: Re: [Springframework-developer] Cleanup of context resources on = webapp reload Well, for instance SQLErrorCodesFactory is a singleton(GoF, not Spring) which = caches SQLErrorCodes internally in the Map with strong references. = Again, I don't know if trying to use WeakHashMap there would do the trick... Dmitriy Tim Kettering wrote: > > I looked at it some more this morning, and basically what I did was > start up tomcat w/ the webapp in the profiler, then after it was done > starting up I used tomcat's manager to stop the context. This should > destroy all resources related to the context. Here is a list of > spring related stuff that still were in memory after the context was > closed. Other stuff was cleaned up just fine. > > org.springframework.beans.CachedIntrospectionResults > org.springframework.jdbc.support.SQLCodes > org.springframework.core.Constants > org.springframework.aop.framework.AdvisedSupport$1 > org.springframework.aop.framework.adapter.BeforeAdviceAdapter > org.springframework.aop.framework.adapter.AfterReturningAdviceAdapter > org.springframework.jdbc.support.SQLErrorCodesFactory > = org.springframework.transaction.support.TransactionSynchronizationManage > r$1 > org.springframework.transaction.interceptor.RollbackRuleAttribute > org.springframework.aop.framework.adapter.ThrosAdviceAdapter > org.springframework.aop.Pointcut$1 > org.springframework.aop.framework.adapter.GlobalAdvisorAdapterRegistry > > On May 3, 2004, at 3:28 PM, Dmitriy Kopylenko wrote: > >> I'm just wondering, would the use of WeakHashMap in >> CachedIntrospectionResults help? >> >> Dmitriy. >> >> Tim Kettering wrote: >> >>> I posted this to the users list last week and did not receive any >>> reply on it, so I'm posting it again here on the developer list, in >>> hopes i could get an reply from someone here. I'm trying to >>> determine if its something I should be doing myself, or if hte >>> spring context should be cleaning up those resources by itself on >>> the .close() call. Further profiling shows that there are >>> duplicate instances of SQLError and hibernate proxy classes = hanging >>> around afterwards too. Other objects do get cleaned up properly. >>> -------- >>> Hi everyone, >>> We're (meaning me) looking into some resource leaks that are >>> occuring when our webapp context gets reloaded. I found that >>> context.close() needs to be called on the destroy() method of >>> plugin we're using, and it works for a good majority of the = objects >>> we were seeing leaked, but there are some objects that I'm unable >>> to make go away. Object in question is the: >>> org.springframework.beans.CachedIntrospectionResults >>> Whenever I reload the context - the profiler I'm using shows that I=20 >>> have essentially a duplicate group of those objects (same instance=20 >>> count) as the original, and successive reloads will continue to=20 >>> duplicate this. >>> The profiler also shows the final reference to those objects like = this: >>> 100% - 1008 bytes - 63 alloc.=20 >>> = org.springframework.context.support.ClassPathXmlApplicationContext.<in >>> it > >>> So basically I guess what I'm asking is for ideas or suggestions on >>> how I could get those to clean up. This bean doesnt show up in the >>> Spring javadocs. And looking in CVS says its a package level bean, >>> not for application use, so I'm thinking that closing the context >>> should (in theory) clean this up? Thanks in advance. >>> -tim >>> ------------------------------------------------------- >>> This SF.Net email is sponsored by: Oracle 10g >>> Get certified on the hottest thing ever to hit the market... Oracle >>> 10g. Take an Oracle 10g class now, and we'll give you the exam FREE. >>> http://ads.osdn.com/?ad_id=3D3149&alloc_id=3D8166&op=3Dclick >>> _______________________________________________ >>> Springframework-developer mailing list >>> Spr...@li... >>> = https://lists.sourceforge.net/lists/listinfo/springframework-developer >> >> >> >> >> ------------------------------------------------------- >> This SF.Net email is sponsored by: Oracle 10g >> Get certified on the hottest thing ever to hit the market... Oracle >> 10g. Take an Oracle 10g class now, and we'll give you the exam FREE. >> http://ads.osdn.com/?ad_id=3D3149&alloc_id=3D8166&op=3Dclick >> _______________________________________________ >> Springframework-developer mailing list >> Spr...@li... >> = https://lists.sourceforge.net/lists/listinfo/springframework-developer >> > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: Oracle 10g > Get certified on the hottest thing ever to hit the market... Oracle = 10g. > Take an Oracle 10g class now, and we'll give you the exam FREE. > http://ads.osdn.com/?ad_id=3D3149&alloc_id=3D8166&op=3Dclick > _______________________________________________ > Springframework-developer mailing list > Spr...@li... > https://lists.sourceforge.net/lists/listinfo/springframework-developer ------------------------------------------------------- This SF.Net email is sponsored by: Oracle 10g Get certified on the hottest thing ever to hit the market... Oracle 10g. Take an Oracle 10g class now, and we'll give you the exam FREE. http://ads.osdn.com/?ad_id=3D3149&alloc_id=3D8166&op=3Dclick _______________________________________________ Springframework-developer mailing list Spr...@li... https://lists.sourceforge.net/lists/listinfo/springframework-developer ------------------------------------------------------- This SF.Net email is sponsored by: Oracle 10g Get certified on the hottest thing ever to hit the market... Oracle 10g. Take an Oracle 10g class now, and we'll give you the exam FREE. http://ads.osdn.com/?ad_id149&alloc_id=8166&op=3Dick _______________________________________________ Springframework-developer mailing list Spr...@li... https://lists.sourceforge.net/lists/listinfo/springframework-developer |
|
From: Darren D. <da...@da...> - 2004-05-23 22:36:11
|
=2D----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On Sunday 23 May 2004 21:45, j=FCrgen h=F6ller [werk3AT] wrote:
> Please, everybody, give the current CVS head a sanity check tomorrow.
> There shouldn't be any issues: the test suite passes, the sample apps run
> properly. Still, I'd feel more comfortable if we make sure that no subtle
> side effects have been introduced.
XSLT views appear to be failing. Tests fail when running the jpetstore app=
=20
in autobuilds and I also checked against a separate test project using=20
classpath and servlet context resource locations for the xsl template. =20
Beginning of stack trace:
org.springframework.context.ApplicationContextException: Can't load=20
stylesheet from resource [/WEB-INF/xsl/home.xslt] of ServletContext in XSLT=
=20
view 'xsl'; nested exception is=20
javax.xml.transform.TransformerConfigurationException:=20
javax.xml.transform.TransformerException:=20
javax.xml.transform.TransformerException: Illegal value: text/html used for=
=20
QNAME attribute: method
org.springframework.web.servlet.view.xslt.AbstractXsltView.cacheTem=
plates(AbstractXsltView.java:147)
org.springframework.web.servlet.view.xslt.AbstractXsltView.initAppl=
icationContext(AbstractXsltView.java:137)
I've not had time to look into it at all, and won't have during the day=20
tomorrow either. It's possible it's something local on my machine (xslt=20
engine impl. or something) but just in case it's not I posted it. I'll=20
check further tomorrow night.
=2D --=20
Darren Davison
Public Key: http://www.davison.uk.net/pages/key.htm
=2D----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)
iD8DBQFAsSdUKLMLAN01aw0RAo17AJ9DAsQJ0mUjgrtD2cmztKhHDUDq5wCdEoyd
WWrrh1wj44cMJJYFuEhk5Bc=3D
=3DathJ
=2D----END PGP SIGNATURE-----
|
|
From: Guillaume P. <gpo...@gl...> - 2004-05-24 03:17:21
|
The "resource leak" caused by a singleton when a webapp's classloader is thrown away is only temorary, the unused classes and the classloader will be eventually garbage collected and the resources will be freed. The only thing that could prevent that is if there was a something in the server's classloader that still had a reference on an object or a class of the child classloader. Unless there's a bug in Tomcat or in the application code, I really can't see how the classes won't be eventually garbage collected when the JVM needs memory. And anyway, if there was indeed a leak because SQLErrorCodesFactory is a singleton, why wouldn't there be one for each static fields such as constants? Are you sure that JProfiler does not disable garbage collecting in order to make its profiling? I know that in many of the JVMPI method calls are done with garbage collecting off. I suspect the above to be the cause of the "resource leak", rather than any singleton that Spring might be using. Guillaume ----- Original Message ----- From: "jürgen höller [werk3AT]" <jue...@we...> To: <spr...@li...> Sent: Sunday, May 23, 2004 4:45 PM Subject: Re: [Springframework-developer] Cleanup of context resources on webapp reload I've just spent about 10 hours profiling Spring, using the Image Database and Petclinic samples. (BTW, I've used an evaluation version of JProfiler from ej-technologies - nice product!) Although I still don't completely understand the garbage collection behavior, I've figured out the following issues. Each of them simply prevents the respective classes from getting garbage collected on destruction of the class loader (e.g. on Tomcat web app shutdown). - A classic singleton with a class variable holding the object. I've reworked GlobalAdvisorAdapterRegistry and SQLErrorCodesFactory to hold the respective singleton as a WeakReference. - A static cache. I've reworked CachedIntrospectionResults to use a WeakHashMap with WeakReferences as values. - A ThreadLocal with a default other than null. I've reworked TransactionSynchronizationManager to use null as default for the resource map, setting a HashMap there on demand, removing the entire HashMap when unbinding the last resource. - Constants that define a full object. We have a number of those, for example ClassFilters.TRUE and AdvisedSupport.EMPTY_TARGET_SOURCE. I've tried for quite a while, but I haven't been able to figure out a way to define such constants such that they will be garbage collected. The latter programming style is not uncommon, so I really don't understand why it causes trouble with garbage collection. Hibernate uses a similar style for its FlushMode, for example. In general, other frameworks like CGLIB, Hibernate, Velocity have huge resource leaks on web app shutdown, while just the constants issue remains with Spring now. As long as those huge third-party leaks are not addressed, I'm not worried at all by the single remaining Spring issue. As I initially said, we shouldn't exaggerate the problem, as it basically just affects hot reloading of web apps - mainly a development feature anyway. We need to make that clear to users too, to avoid comments a la "Spring is not usable for real apps because it leaks on hot redeployment". Please, everybody, give the current CVS head a sanity check tomorrow. There shouldn't be any issues: the test suite passes, the sample apps run properly. Still, I'd feel more comfortable if we make sure that no subtle side effects have been introduced. For this reason, I will delay release 1.0.2 till tomorrow night. Juergen ________________________________ Von: spr...@li... im Auftrag von jürgen höller [werk3AT] Gesendet: Sa 22.05.2004 15:52 An: spr...@li... Betreff: Re: [Springframework-developer] Cleanup of context resources on webapp reload So basically, all static caches cause resource leaks when restarting a Tomcat web app? I wonder why this happens... The class loader should completely dissolve all classes that it has loaded in its lifetime, including static caches. Or have I misunderstood something here? Anyone having in-detail experience with handling such a scenario? Juergen ________________________________ Von: spr...@li... im Auftrag von Dmitriy Kopylenko Gesendet: Di 04.05.2004 18:02 An: spr...@li... Betreff: Re: [Springframework-developer] Cleanup of context resources on webapp reload Well, for instance SQLErrorCodesFactory is a singleton(GoF, not Spring) which caches SQLErrorCodes internally in the Map with strong references. Again, I don't know if trying to use WeakHashMap there would do the trick... Dmitriy Tim Kettering wrote: > > I looked at it some more this morning, and basically what I did was > start up tomcat w/ the webapp in the profiler, then after it was done > starting up I used tomcat's manager to stop the context. This should > destroy all resources related to the context. Here is a list of > spring related stuff that still were in memory after the context was > closed. Other stuff was cleaned up just fine. > > org.springframework.beans.CachedIntrospectionResults > org.springframework.jdbc.support.SQLCodes > org.springframework.core.Constants > org.springframework.aop.framework.AdvisedSupport$1 > org.springframework.aop.framework.adapter.BeforeAdviceAdapter > org.springframework.aop.framework.adapter.AfterReturningAdviceAdapter > org.springframework.jdbc.support.SQLErrorCodesFactory > org.springframework.transaction.support.TransactionSynchronizationManage > r$1 > org.springframework.transaction.interceptor.RollbackRuleAttribute > org.springframework.aop.framework.adapter.ThrosAdviceAdapter > org.springframework.aop.Pointcut$1 > org.springframework.aop.framework.adapter.GlobalAdvisorAdapterRegistry > > On May 3, 2004, at 3:28 PM, Dmitriy Kopylenko wrote: > >> I'm just wondering, would the use of WeakHashMap in >> CachedIntrospectionResults help? >> >> Dmitriy. >> >> Tim Kettering wrote: >> >>> I posted this to the users list last week and did not receive any >>> reply on it, so I'm posting it again here on the developer list, in >>> hopes i could get an reply from someone here. I'm trying to >>> determine if its something I should be doing myself, or if hte >>> spring context should be cleaning up those resources by itself on >>> the .close() call. Further profiling shows that there are >>> duplicate instances of SQLError and hibernate proxy classes hanging >>> around afterwards too. Other objects do get cleaned up properly. >>> -------- >>> Hi everyone, >>> We're (meaning me) looking into some resource leaks that are >>> occuring when our webapp context gets reloaded. I found that >>> context.close() needs to be called on the destroy() method of >>> plugin we're using, and it works for a good majority of the objects >>> we were seeing leaked, but there are some objects that I'm unable >>> to make go away. Object in question is the: >>> org.springframework.beans.CachedIntrospectionResults >>> Whenever I reload the context - the profiler I'm using shows that I >>> have essentially a duplicate group of those objects (same instance >>> count) as the original, and successive reloads will continue to >>> duplicate this. >>> The profiler also shows the final reference to those objects like this: >>> 100% - 1008 bytes - 63 alloc. >>> org.springframework.context.support.ClassPathXmlApplicationContext.<in >>> it > >>> So basically I guess what I'm asking is for ideas or suggestions on >>> how I could get those to clean up. This bean doesnt show up in the >>> Spring javadocs. And looking in CVS says its a package level bean, >>> not for application use, so I'm thinking that closing the context >>> should (in theory) clean this up? Thanks in advance. >>> -tim >>> ------------------------------------------------------- >>> This SF.Net email is sponsored by: Oracle 10g >>> Get certified on the hottest thing ever to hit the market... Oracle >>> 10g. Take an Oracle 10g class now, and we'll give you the exam FREE. >>> http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click >>> _______________________________________________ >>> Springframework-developer mailing list >>> Spr...@li... >>> https://lists.sourceforge.net/lists/listinfo/springframework-developer >> >> >> >> >> ------------------------------------------------------- >> This SF.Net email is sponsored by: Oracle 10g >> Get certified on the hottest thing ever to hit the market... Oracle >> 10g. Take an Oracle 10g class now, and we'll give you the exam FREE. >> http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click >> _______________________________________________ >> Springframework-developer mailing list >> Spr...@li... >> https://lists.sourceforge.net/lists/listinfo/springframework-developer >> > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: Oracle 10g > Get certified on the hottest thing ever to hit the market... Oracle 10g. > Take an Oracle 10g class now, and we'll give you the exam FREE. > http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click > _______________________________________________ > Springframework-developer mailing list > Spr...@li... > https://lists.sourceforge.net/lists/listinfo/springframework-developer ------------------------------------------------------- This SF.Net email is sponsored by: Oracle 10g Get certified on the hottest thing ever to hit the market... Oracle 10g. Take an Oracle 10g class now, and we'll give you the exam FREE. http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click _______________________________________________ Springframework-developer mailing list Spr...@li... https://lists.sourceforge.net/lists/listinfo/springframework-developer ------------------------------------------------------- This SF.Net email is sponsored by: Oracle 10g Get certified on the hottest thing ever to hit the market... Oracle 10g. Take an Oracle 10g class now, and we'll give you the exam FREE. http://ads.osdn.com/?ad_id149&alloc_id66&op=ick _______________________________________________ Springframework-developer mailing list Spr...@li... https://lists.sourceforge.net/lists/listinfo/springframework-developer ------------------------------------------------------- This SF.Net email is sponsored by: Oracle 10g Get certified on the hottest thing ever to hit the market... Oracle 10g. Take an Oracle 10g class now, and we'll give you the exam FREE. http://ads.osdn.com/?ad_id149&alloc_id66&op=ick _______________________________________________ Springframework-developer mailing list Spr...@li... https://lists.sourceforge.net/lists/listinfo/springframework-developer |
|
From: Colin S. <col...@ex...> - 2004-05-25 01:45:36
|
Just gave CVS Head a go with my main app (was using source about a week old, previously). All seems ok for basic container operations, transactions, and Hibernate. Unit tests all run fine on the couple of machines I build on... jürgen höller [werk3AT] wrote: >I've just spent about 10 hours profiling Spring, using the Image Database and Petclinic samples. (BTW, I've used an evaluation version of JProfiler from ej-technologies - nice product!) > >Although I still don't completely understand the garbage collection behavior, I've figured out the following issues. Each of them simply prevents the respective classes from getting garbage collected on destruction of the class loader (e.g. on Tomcat web app shutdown). > >- A classic singleton with a class variable holding the object. I've reworked GlobalAdvisorAdapterRegistry and SQLErrorCodesFactory to hold the respective singleton as a WeakReference. > >- A static cache. I've reworked CachedIntrospectionResults to use a WeakHashMap with WeakReferences as values. > >- A ThreadLocal with a default other than null. I've reworked TransactionSynchronizationManager to use null as default for the resource map, setting a HashMap there on demand, removing the entire HashMap when unbinding the last resource. > >- Constants that define a full object. We have a number of those, for example ClassFilters.TRUE and AdvisedSupport.EMPTY_TARGET_SOURCE. I've tried for quite a while, but I haven't been able to figure out a way to define such constants such that they will be garbage collected. > >The latter programming style is not uncommon, so I really don't understand why it causes trouble with garbage collection. Hibernate uses a similar style for its FlushMode, for example. > >In general, other frameworks like CGLIB, Hibernate, Velocity have huge resource leaks on web app shutdown, while just the constants issue remains with Spring now. As long as those huge third-party leaks are not addressed, I'm not worried at all by the single remaining Spring issue. > >As I initially said, we shouldn't exaggerate the problem, as it basically just affects hot reloading of web apps - mainly a development feature anyway. We need to make that clear to users too, to avoid comments a la "Spring is not usable for real apps because it leaks on hot redeployment". > >Please, everybody, give the current CVS head a sanity check tomorrow. There shouldn't be any issues: the test suite passes, the sample apps run properly. Still, I'd feel more comfortable if we make sure that no subtle side effects have been introduced. > >For this reason, I will delay release 1.0.2 till tomorrow night. > >Juergen > > >________________________________ > >Von: spr...@li... im Auftrag von jürgen höller [werk3AT] >Gesendet: Sa 22.05.2004 15:52 >An: spr...@li... >Betreff: Re: [Springframework-developer] Cleanup of context resources on webapp reload > > > >So basically, all static caches cause resource leaks when restarting a Tomcat web app? I wonder why this happens... The class loader should completely dissolve all classes that it has loaded in its lifetime, including static caches. Or have I misunderstood something here? Anyone having in-detail experience with handling such a scenario? > >Juergen > > >________________________________ > >Von: spr...@li... im Auftrag von Dmitriy Kopylenko >Gesendet: Di 04.05.2004 18:02 >An: spr...@li... >Betreff: Re: [Springframework-developer] Cleanup of context resources on webapp reload > > > >Well, > >for instance SQLErrorCodesFactory is a singleton(GoF, not Spring) which caches SQLErrorCodes internally in the Map with strong references. Again, I don't know if trying >to use WeakHashMap there would do the trick... > >Dmitriy > >Tim Kettering wrote: > > > >>I looked at it some more this morning, and basically what I did was >>start up tomcat w/ the webapp in the profiler, then after it was done >>starting up I used tomcat's manager to stop the context. This should >>destroy all resources related to the context. Here is a list of >>spring related stuff that still were in memory after the context was >>closed. Other stuff was cleaned up just fine. >> >>org.springframework.beans.CachedIntrospectionResults >>org.springframework.jdbc.support.SQLCodes >>org.springframework.core.Constants >>org.springframework.aop.framework.AdvisedSupport$1 >>org.springframework.aop.framework.adapter.BeforeAdviceAdapter >>org.springframework.aop.framework.adapter.AfterReturningAdviceAdapter >>org.springframework.jdbc.support.SQLErrorCodesFactory >>org.springframework.transaction.support.TransactionSynchronizationManage >>r$1 >>org.springframework.transaction.interceptor.RollbackRuleAttribute >>org.springframework.aop.framework.adapter.ThrosAdviceAdapter >>org.springframework.aop.Pointcut$1 >>org.springframework.aop.framework.adapter.GlobalAdvisorAdapterRegistry >> >>On May 3, 2004, at 3:28 PM, Dmitriy Kopylenko wrote: >> >> >> >>>I'm just wondering, would the use of WeakHashMap in >>>CachedIntrospectionResults help? >>> >>>Dmitriy. >>> >>>Tim Kettering wrote: >>> >>> >>> >>>>I posted this to the users list last week and did not receive any >>>>reply on it, so I'm posting it again here on the developer list, in >>>>hopes i could get an reply from someone here. I'm trying to >>>>determine if its something I should be doing myself, or if hte >>>>spring context should be cleaning up those resources by itself on >>>>the .close() call. Further profiling shows that there are >>>>duplicate instances of SQLError and hibernate proxy classes hanging >>>>around afterwards too. Other objects do get cleaned up properly. >>>>-------- >>>>Hi everyone, >>>>We're (meaning me) looking into some resource leaks that are >>>>occuring when our webapp context gets reloaded. I found that >>>>context.close() needs to be called on the destroy() method of >>>>plugin we're using, and it works for a good majority of the objects >>>>we were seeing leaked, but there are some objects that I'm unable >>>>to make go away. Object in question is the: >>>>org.springframework.beans.CachedIntrospectionResults >>>>Whenever I reload the context - the profiler I'm using shows that I >>>>have essentially a duplicate group of those objects (same instance >>>>count) as the original, and successive reloads will continue to >>>>duplicate this. >>>>The profiler also shows the final reference to those objects like this: >>>>100% - 1008 bytes - 63 alloc. >>>>org.springframework.context.support.ClassPathXmlApplicationContext.<in >>>>it > >>>>So basically I guess what I'm asking is for ideas or suggestions on >>>>how I could get those to clean up. This bean doesnt show up in the >>>>Spring javadocs. And looking in CVS says its a package level bean, >>>>not for application use, so I'm thinking that closing the context >>>>should (in theory) clean this up? Thanks in advance. >>>>-tim >>>>------------------------------------------------------- >>>>This SF.Net email is sponsored by: Oracle 10g >>>>Get certified on the hottest thing ever to hit the market... Oracle >>>>10g. Take an Oracle 10g class now, and we'll give you the exam FREE. >>>>http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click >>>> >>>> |
|
From: Matt R. <li...@ra...> - 2004-05-25 07:03:11
|
I tried CVS Head in 2 Spring apps and all tests pass. Matt > -----Original Message----- > From: spr...@li...=20 > [mailto:spr...@li...] > On Behalf Of Colin Sampaleanu > Sent: Monday, May 24, 2004 7:49 PM > To: spr...@li... > Subject: Re: [Springframework-developer] Cleanup of context=20 > resources on webapp reload >=20 >=20 > Just gave CVS Head a go with my main app (was using source=20 > about a week=20 > old, previously). All seems ok for basic container operations,=20 > transactions, and Hibernate. Unit tests all run fine on the couple of=20 > machines I build on... >=20 >=20 > j=FCrgen h=F6ller [werk3AT] wrote: >=20 > >I've just spent about 10 hours profiling Spring, using the Image=20 > >Database and Petclinic samples. (BTW, I've used an=20 > evaluation version of JProfiler from ej-technologies - nice product!) > >=20 > >Although I still don't completely understand the garbage collection=20 > >behavior, I've figured out the following issues. Each of=20 > them simply prevents the respective classes from getting=20 > garbage collected on destruction of the class loader (e.g. on=20 > Tomcat web app shutdown). > >=20 > >- A classic singleton with a class variable holding the object. I've=20 > >reworked GlobalAdvisorAdapterRegistry and=20 > SQLErrorCodesFactory to hold the respective singleton as a=20 > WeakReference. > >=20 > >- A static cache. I've reworked CachedIntrospectionResults to use a=20 > >WeakHashMap with WeakReferences as values. > >=20 > >- A ThreadLocal with a default other than null. I've reworked=20 > >TransactionSynchronizationManager to use null as default for=20 > the resource map, setting a HashMap there on demand, removing=20 > the entire HashMap when unbinding the last resource. > >=20 > >- Constants that define a full object. We have a number of=20 > those, for=20 > >example ClassFilters.TRUE and=20 > AdvisedSupport.EMPTY_TARGET_SOURCE. I've tried for quite a=20 > while, but I haven't been able to figure out a way to define=20 > such constants such that they will be garbage collected. > >=20 > >The latter programming style is not uncommon, so I really don't=20 > >understand why it causes trouble with garbage collection.=20 > Hibernate uses a similar style for its FlushMode, for example. > >=20 > >In general, other frameworks like CGLIB, Hibernate, Velocity=20 > have huge=20 > >resource leaks on web app shutdown, while just the constants=20 > issue remains with Spring now. As long as those huge=20 > third-party leaks are not addressed, I'm not worried at all=20 > by the single remaining Spring issue. > >=20 > >As I initially said, we shouldn't exaggerate the problem, as it=20 > >basically just affects hot reloading of web apps - mainly a=20 > development feature anyway. We need to make that clear to=20 > users too, to avoid comments a la "Spring is not usable for=20 > real apps because it leaks on hot redeployment". > >=20 > >Please, everybody, give the current CVS head a sanity check=20 > tomorrow.=20 > >There shouldn't be any issues: the test suite passes, the=20 > sample apps run properly. Still, I'd feel more comfortable if=20 > we make sure that no subtle side effects have been introduced. > >=20 > >For this reason, I will delay release 1.0.2 till tomorrow night. > >=20 > >Juergen > >=20 > > > >________________________________ > > > >Von: spr...@li...=20 > im Auftrag=20 > >von j=FCrgen h=F6ller [werk3AT] > >Gesendet: Sa 22.05.2004 15:52 > >An: spr...@li... > >Betreff: Re: [Springframework-developer] Cleanup of context=20 > resources on webapp reload > > > > > > > >So basically, all static caches cause resource leaks when=20 > restarting a=20 > >Tomcat web app? I wonder why this happens... The class loader should=20 > >completely dissolve all classes that it has loaded in its lifetime,=20 > >including static caches. Or have I misunderstood something=20 > here? Anyone=20 > >having in-detail experience with handling such a scenario? > > > >Juergen > > > > > >________________________________ > > > >Von: spr...@li...=20 > im Auftrag=20 > >von Dmitriy Kopylenko > >Gesendet: Di 04.05.2004 18:02 > >An: spr...@li... > >Betreff: Re: [Springframework-developer] Cleanup of context=20 > resources on webapp reload > > > > > > > >Well, > > > >for instance SQLErrorCodesFactory is a singleton(GoF, not=20 > Spring) which=20 > >caches SQLErrorCodes internally in the Map with strong references.=20 > >Again, I don't know if trying to use WeakHashMap there would do the=20 > >trick... > > > >Dmitriy > > > >Tim Kettering wrote: > > > > =20 > > > >>I looked at it some more this morning, and basically what I did was=20 > >>start up tomcat w/ the webapp in the profiler, then after=20 > it was done=20 > >>starting up I used tomcat's manager to stop the context. =20 > This should > >>destroy all resources related to the context. Here is a list of > >>spring related stuff that still were in memory after the=20 > context was=20 > >>closed. Other stuff was cleaned up just fine. > >> > >>org.springframework.beans.CachedIntrospectionResults > >>org.springframework.jdbc.support.SQLCodes > >>org.springframework.core.Constants > >>org.springframework.aop.framework.AdvisedSupport$1 > >>org.springframework.aop.framework.adapter.BeforeAdviceAdapter > >>org.springframework.aop.framework.adapter.AfterReturningAdvi > ceAdapter > >>org.springframework.jdbc.support.SQLErrorCodesFactory > >>org.springframework.transaction.support.TransactionSynchroni > zationMana > >>ge > >>r$1 > >>org.springframework.transaction.interceptor.RollbackRuleAttribute > >>org.springframework.aop.framework.adapter.ThrosAdviceAdapter > >>org.springframework.aop.Pointcut$1 > >>org.springframework.aop.framework.adapter.GlobalAdvisorAdapt > erRegistry > >> > >>On May 3, 2004, at 3:28 PM, Dmitriy Kopylenko wrote: > >> > >> =20 > >> > >>>I'm just wondering, would the use of WeakHashMap in=20 > >>>CachedIntrospectionResults help? > >>> > >>>Dmitriy. > >>> > >>>Tim Kettering wrote: > >>> > >>> =20 > >>> > >>>>I posted this to the users list last week and did not receive any=20 > >>>>reply on it, so I'm posting it again here on the=20 > developer list, in > >>>>hopes i could get an reply from someone here. I'm trying to > >>>>determine if its something I should be doing myself, or if hte=20 > >>>>spring context should be cleaning up those resources by=20 > itself on=20 > >>>>the .close() call. Further profiling shows that there are=20 > >>>>duplicate instances of SQLError and hibernate proxy classes=20 > >>>>hanging around afterwards too. Other objects do get cleaned up=20 > >>>>properly. > >>>>-------- > >>>>Hi everyone, > >>>>We're (meaning me) looking into some resource leaks that are > >>>>occuring when our webapp context gets reloaded. I found that > >>>>context.close() needs to be called on the destroy() method of > >>>>plugin we're using, and it works for a good majority of=20 > the objects > >>>>we were seeing leaked, but there are some objects that=20 > I'm unable > >>>>to make go away. Object in question is the: > >>>>org.springframework.beans.CachedIntrospectionResults > >>>>Whenever I reload the context - the profiler I'm using=20 > shows that I=20 > >>>>have essentially a duplicate group of those objects (same=20 > instance=20 > >>>>count) as the original, and successive reloads will continue to=20 > >>>>duplicate this. > >>>>The profiler also shows the final reference to those=20 > objects like this: > >>>>100% - 1008 bytes - 63 alloc.=20 > >>>>org.springframework.context.support.ClassPathXmlApplicatio > nContext.<in > >>>>it > > >>>>So basically I guess what I'm asking is for ideas or=20 > suggestions on > >>>>how I could get those to clean up. This bean doesnt=20 > show up in the > >>>>Spring javadocs. And looking in CVS says its a package=20 > level bean, > >>>>not for application use, so I'm thinking that closing the context > >>>>should (in theory) clean this up? Thanks in advance. > >>>>-tim > >>>>------------------------------------------------------- > >>>>This SF.Net email is sponsored by: Oracle 10g > >>>>Get certified on the hottest thing ever to hit the=20 > market... Oracle > >>>>10g. Take an Oracle 10g class now, and we'll give you the=20 > exam FREE. > >>>>http://ads.osdn.com/?ad_id=3D3149&alloc_id=3D8166&op=3Dclick > >>>> =20 > >>>> >=20 >=20 >=20 >=20 > ------------------------------------------------------- > This SF.Net email is sponsored by: Oracle 10g > Get certified on the hottest thing ever to hit the market...=20 > Oracle 10g.=20 > Take an Oracle 10g class now, and we'll give you the exam=20 > FREE. http://ads.osdn.com/?ad_id=3D3149&alloc_id=3D8166&op=3Dclick > _______________________________________________ > Springframework-developer mailing list=20 > Spr...@li... > https://lists.sourceforge.net/lists/listinfo/springframework-developer >=20 |
|
From: Rod J. <rod...@in...> - 2004-05-25 07:31:41
|
> >- Constants that define a full object. We have a number of those, for example ClassFilters.TRUE and AdvisedSupport.EMPTY_TARGET_SOURCE. I've tried for quite a while, but I haven't been able to figure out a way to define such constants such that they will be garbage collected. Juergen Good detective work! As you say, I don't think there's any real issue now, especially when you consider other libraries. Surely the amount of leakage in hot deployment scenarios with the constants is going to be tiny... And Hibernate, CGLIB et al will account for way more issues. Is it correct to assume that problem scenarios in development are now unlikely? Maintaining canonical instances is good style IMHO. However, if you do have a list of those you detected, please send it to me. (I expect they're mainly in my code.) Some of them could be changed fairly easily: for example, a null pointcut could have the same effect as Pointcut.TRUE. I'd like to take a look when I have time (2-3 weeks) and see if it's appropriate to change some or all of them. HOWEVER, such a change would be effectively a public API change in some cases. I don't imagine too many people use the canonical pointcut and ClassFilter instances (because it's not really necessary in those cases), but they _are_ public. EMPTY_TARGET_SOURCE doesn't really have such a problem. That can be changed easily, and I've thought of doing it once or twice. It can just be promoted to a package-level class, and a new instance could be created for each AdvisedSupport, as the class itself is trivial. Rgds Rod |
|
From: Guillaume P. <gpo...@gl...> - 2004-05-29 04:12:44
|
I experimented some more about this cleanup issue, and I was able to narrow down the problem to the caching done by the java.beans.Introspector. It stores the BeanInfo instances in a WeakHashMap, but in that Map implementation, only the keys uses WeakReference, the values are stored with hard references since BeanInfo has an hard reference on the class it gives info about (indirectly through BeanDescriptor and others), any time Introspector.getBeanInfo(Class) is used, that class and it's static members will not ever be able to be garbage collected. I kind of remember someone mentioning something related to this in the mailling list, but I cannot find the mail. I wonder if there's other case where the java[x] classes might have an hard reference on a class or its instances. A fix for this particular problem is to have a ServletContextListener call Introspector.flushCaches() when the context is destroyed. It seems like a known issue at Sun : http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4291376 http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4730581 http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4809008 So, unless I'm missing something here, that means fixes like using synchronization and WeakReference on singleton probably won't help much if at all. The only way that I can see for a class not to be gargage collected when no more active thread use it, is if another ClassLoader has an hard reference to the class instance, or an instance of that class. Having the class itself have an hard reference on a its own singleton has no effect, it's a circular reference that will not prevent the class or the instance to be gargabe collected when neither is being refered to by something else. Guillaume |
|
From: <jue...@we...> - 2004-05-24 06:43:28
|
I've told JProfiler to explicitly run garbage collection - a number of = times, actually - before I've had a look at the heap. So I'm sure that = those remaining objects were not garbage-collected, and probably would = have stayed around in the VM forever... =20 I'm aware that it seems odd, but this issue just seems to affect = specific static fields: BeanWrapperImpl's defaultEditors did not cause a = leak, but CachedIntrospectionResults' classCache did. Normal constants = or static logger fields didn't, but "full object" constants like = ClassFilters.TRUE did. =20 CachedIntrospectionResults' classCache uses the Class as key; the value, = a CachedIntrospectionResult object, also refers to the key Class. = Consequently, I had to use a WeakHashMap *with WeakReferences as values* = to see proper garbage collection. Note that this static cache contains = instances of its containing class as values. =20 I've run all my tests x times to make sure that I could trust my eyes. = I've even undone the WeakReference changes again, and voila, there were = the leaks again. I'd be happy to learn more about how the garbage = collector works here... All I can state at this point of time is that = the changes did cause an obvious difference in terms of resource leaks. =20 Juergen =20 ________________________________ Von: spr...@li... im Auftrag = von Guillaume Poirier Gesendet: Mo 24.05.2004 05:17 An: spr...@li... Betreff: Re: [Springframework-developer] Cleanup of context resources on = webapp reload The "resource leak" caused by a singleton when a webapp's classloader is thrown away is only temorary, the unused classes and the classloader = will be eventually garbage collected and the resources will be freed. The only thing that could prevent that is if there was a something in the = server's classloader that still had a reference on an object or a class of the = child classloader. Unless there's a bug in Tomcat or in the application code, I really = can't see how the classes won't be eventually garbage collected when the JVM = needs memory. And anyway, if there was indeed a leak because = SQLErrorCodesFactory is a singleton, why wouldn't there be one for each static fields such as constants? Are you sure that JProfiler does not disable garbage collecting in order = to make its profiling? I know that in many of the JVMPI method calls are = done with garbage collecting off. I suspect the above to be the cause of the "resource leak", rather than any singleton that Spring might be using. Guillaume ----- Original Message ----- From: "j=FCrgen h=F6ller [werk3AT]" <jue...@we...> To: <spr...@li...> Sent: Sunday, May 23, 2004 4:45 PM Subject: Re: [Springframework-developer] Cleanup of context resources on webapp reload I've just spent about 10 hours profiling Spring, using the Image = Database and Petclinic samples. (BTW, I've used an evaluation version of = JProfiler from ej-technologies - nice product!) Although I still don't completely understand the garbage collection behavior, I've figured out the following issues. Each of them simply prevents the respective classes from getting garbage collected on destruction of the class loader (e.g. on Tomcat web app shutdown). - A classic singleton with a class variable holding the object. I've reworked GlobalAdvisorAdapterRegistry and SQLErrorCodesFactory to hold = the respective singleton as a WeakReference. - A static cache. I've reworked CachedIntrospectionResults to use a WeakHashMap with WeakReferences as values. - A ThreadLocal with a default other than null. I've reworked TransactionSynchronizationManager to use null as default for the = resource map, setting a HashMap there on demand, removing the entire HashMap when unbinding the last resource. - Constants that define a full object. We have a number of those, for example ClassFilters.TRUE and AdvisedSupport.EMPTY_TARGET_SOURCE. I've = tried for quite a while, but I haven't been able to figure out a way to define such constants such that they will be garbage collected. The latter programming style is not uncommon, so I really don't = understand why it causes trouble with garbage collection. Hibernate uses a similar style for its FlushMode, for example. In general, other frameworks like CGLIB, Hibernate, Velocity have huge resource leaks on web app shutdown, while just the constants issue = remains with Spring now. As long as those huge third-party leaks are not = addressed, I'm not worried at all by the single remaining Spring issue. As I initially said, we shouldn't exaggerate the problem, as it = basically just affects hot reloading of web apps - mainly a development feature anyway. We need to make that clear to users too, to avoid comments a la "Spring is not usable for real apps because it leaks on hot = redeployment". Please, everybody, give the current CVS head a sanity check tomorrow. = There shouldn't be any issues: the test suite passes, the sample apps run properly. Still, I'd feel more comfortable if we make sure that no = subtle side effects have been introduced. For this reason, I will delay release 1.0.2 till tomorrow night. Juergen ________________________________ Von: spr...@li... im Auftrag = von j=FCrgen h=F6ller [werk3AT] Gesendet: Sa 22.05.2004 15:52 An: spr...@li... Betreff: Re: [Springframework-developer] Cleanup of context resources on webapp reload So basically, all static caches cause resource leaks when restarting a Tomcat web app? I wonder why this happens... The class loader should completely dissolve all classes that it has loaded in its lifetime, including static caches. Or have I misunderstood something here? Anyone having in-detail experience with handling such a scenario? Juergen ________________________________ Von: spr...@li... im Auftrag = von Dmitriy Kopylenko Gesendet: Di 04.05.2004 18:02 An: spr...@li... Betreff: Re: [Springframework-developer] Cleanup of context resources on webapp reload Well, for instance SQLErrorCodesFactory is a singleton(GoF, not Spring) which caches SQLErrorCodes internally in the Map with strong references. = Again, I don't know if trying to use WeakHashMap there would do the trick... Dmitriy Tim Kettering wrote: > > I looked at it some more this morning, and basically what I did was > start up tomcat w/ the webapp in the profiler, then after it was done > starting up I used tomcat's manager to stop the context. This should > destroy all resources related to the context. Here is a list of > spring related stuff that still were in memory after the context was > closed. Other stuff was cleaned up just fine. > > org.springframework.beans.CachedIntrospectionResults > org.springframework.jdbc.support.SQLCodes > org.springframework.core.Constants > org.springframework.aop.framework.AdvisedSupport$1 > org.springframework.aop.framework.adapter.BeforeAdviceAdapter > org.springframework.aop.framework.adapter.AfterReturningAdviceAdapter > org.springframework.jdbc.support.SQLErrorCodesFactory > = org.springframework.transaction.support.TransactionSynchronizationManage > r$1 > org.springframework.transaction.interceptor.RollbackRuleAttribute > org.springframework.aop.framework.adapter.ThrosAdviceAdapter > org.springframework.aop.Pointcut$1 > org.springframework.aop.framework.adapter.GlobalAdvisorAdapterRegistry > > On May 3, 2004, at 3:28 PM, Dmitriy Kopylenko wrote: > >> I'm just wondering, would the use of WeakHashMap in >> CachedIntrospectionResults help? >> >> Dmitriy. >> >> Tim Kettering wrote: >> >>> I posted this to the users list last week and did not receive any >>> reply on it, so I'm posting it again here on the developer list, in >>> hopes i could get an reply from someone here. I'm trying to >>> determine if its something I should be doing myself, or if hte >>> spring context should be cleaning up those resources by itself on >>> the .close() call. Further profiling shows that there are >>> duplicate instances of SQLError and hibernate proxy classes = hanging >>> around afterwards too. Other objects do get cleaned up properly. >>> -------- >>> Hi everyone, >>> We're (meaning me) looking into some resource leaks that are >>> occuring when our webapp context gets reloaded. I found that >>> context.close() needs to be called on the destroy() method of >>> plugin we're using, and it works for a good majority of the = objects >>> we were seeing leaked, but there are some objects that I'm unable >>> to make go away. Object in question is the: >>> org.springframework.beans.CachedIntrospectionResults >>> Whenever I reload the context - the profiler I'm using shows that I >>> have essentially a duplicate group of those objects (same instance >>> count) as the original, and successive reloads will continue to >>> duplicate this. >>> The profiler also shows the final reference to those objects like = this: >>> 100% - 1008 bytes - 63 alloc. >>> = org.springframework.context.support.ClassPathXmlApplicationContext.<in >>> it > >>> So basically I guess what I'm asking is for ideas or suggestions on >>> how I could get those to clean up. This bean doesnt show up in the >>> Spring javadocs. And looking in CVS says its a package level bean, >>> not for application use, so I'm thinking that closing the context >>> should (in theory) clean this up? Thanks in advance. >>> -tim >>> ------------------------------------------------------- >>> This SF.Net email is sponsored by: Oracle 10g >>> Get certified on the hottest thing ever to hit the market... Oracle >>> 10g. Take an Oracle 10g class now, and we'll give you the exam FREE. >>> http://ads.osdn.com/?ad_id=3D3149&alloc_id=3D8166&op=3Dclick >>> _______________________________________________ >>> Springframework-developer mailing list >>> Spr...@li... >>> = https://lists.sourceforge.net/lists/listinfo/springframework-developer >> >> >> >> >> ------------------------------------------------------- >> This SF.Net email is sponsored by: Oracle 10g >> Get certified on the hottest thing ever to hit the market... Oracle >> 10g. Take an Oracle 10g class now, and we'll give you the exam FREE. >> http://ads.osdn.com/?ad_id=3D3149&alloc_id=3D8166&op=3Dclick >> _______________________________________________ >> Springframework-developer mailing list >> Spr...@li... >> = https://lists.sourceforge.net/lists/listinfo/springframework-developer >> > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: Oracle 10g > Get certified on the hottest thing ever to hit the market... Oracle = 10g. > Take an Oracle 10g class now, and we'll give you the exam FREE. > http://ads.osdn.com/?ad_id=3D3149&alloc_id=3D8166&op=3Dclick > _______________________________________________ > Springframework-developer mailing list > Spr...@li... > https://lists.sourceforge.net/lists/listinfo/springframework-developer ------------------------------------------------------- This SF.Net email is sponsored by: Oracle 10g Get certified on the hottest thing ever to hit the market... Oracle 10g. Take an Oracle 10g class now, and we'll give you the exam FREE. http://ads.osdn.com/?ad_id=3D3149&alloc_id=3D8166&op=3Dclick _______________________________________________ Springframework-developer mailing list Spr...@li... https://lists.sourceforge.net/lists/listinfo/springframework-developer ------------------------------------------------------- This SF.Net email is sponsored by: Oracle 10g Get certified on the hottest thing ever to hit the market... Oracle 10g. Take an Oracle 10g class now, and we'll give you the exam FREE. http://ads.osdn.com/?ad_id149&alloc_id=8166&op=3Dick _______________________________________________ Springframework-developer mailing list Spr...@li... https://lists.sourceforge.net/lists/listinfo/springframework-developer ------------------------------------------------------- This SF.Net email is sponsored by: Oracle 10g Get certified on the hottest thing ever to hit the market... Oracle 10g. Take an Oracle 10g class now, and we'll give you the exam FREE. http://ads.osdn.com/?ad_id149&alloc_id=8166&op=3Dick _______________________________________________ Springframework-developer mailing list Spr...@li... https://lists.sourceforge.net/lists/listinfo/springframework-developer ------------------------------------------------------- This SF.Net email is sponsored by: Oracle 10g Get certified on the hottest thing ever to hit the market... Oracle 10g. Take an Oracle 10g class now, and we'll give you the exam FREE. http://ads.osdn.com/?ad_id=3D3149&alloc_id=3D8166&op=3Dclick _______________________________________________ Springframework-developer mailing list Spr...@li... https://lists.sourceforge.net/lists/listinfo/springframework-developer |
|
From: <jue...@we...> - 2004-05-24 07:22:13
|
FYI, I've tested both Tomcat 5.0.18 and 4.1.27 - same behavior with = both. =20 BTW, a couple of related posts from the Resin mailing list: =20 http://www.caucho.com/support/resin-interest/0404/0117.html http://www.caucho.com/support/resin-interest/0111/0164.html = <http://www.caucho.com/support/resin-interest/0111/0164.html>=20 =20 Juergen =20 ________________________________ Von: spr...@li... im Auftrag = von j=FCrgen h=F6ller [werk3AT] Gesendet: Mo 24.05.2004 08:42 An: spr...@li... Betreff: Re: [Springframework-developer] Cleanup of context resources on = webapp reload I've told JProfiler to explicitly run garbage collection - a number of = times, actually - before I've had a look at the heap. So I'm sure that = those remaining objects were not garbage-collected, and probably would = have stayed around in the VM forever... I'm aware that it seems odd, but this issue just seems to affect = specific static fields: BeanWrapperImpl's defaultEditors did not cause a = leak, but CachedIntrospectionResults' classCache did. Normal constants = or static logger fields didn't, but "full object" constants like = ClassFilters.TRUE did. CachedIntrospectionResults' classCache uses the Class as key; the value, = a CachedIntrospectionResult object, also refers to the key Class. = Consequently, I had to use a WeakHashMap *with WeakReferences as values* = to see proper garbage collection. Note that this static cache contains = instances of its containing class as values. I've run all my tests x times to make sure that I could trust my eyes. = I've even undone the WeakReference changes again, and voila, there were = the leaks again. I'd be happy to learn more about how the garbage = collector works here... All I can state at this point of time is that = the changes did cause an obvious difference in terms of resource leaks. Juergen ________________________________ Von: spr...@li... im Auftrag = von Guillaume Poirier Gesendet: Mo 24.05.2004 05:17 An: spr...@li... Betreff: Re: [Springframework-developer] Cleanup of context resources on = webapp reload The "resource leak" caused by a singleton when a webapp's classloader is thrown away is only temorary, the unused classes and the classloader = will be eventually garbage collected and the resources will be freed. The only thing that could prevent that is if there was a something in the = server's classloader that still had a reference on an object or a class of the = child classloader. Unless there's a bug in Tomcat or in the application code, I really = can't see how the classes won't be eventually garbage collected when the JVM = needs memory. And anyway, if there was indeed a leak because = SQLErrorCodesFactory is a singleton, why wouldn't there be one for each static fields such as constants? Are you sure that JProfiler does not disable garbage collecting in order = to make its profiling? I know that in many of the JVMPI method calls are = done with garbage collecting off. I suspect the above to be the cause of the "resource leak", rather than any singleton that Spring might be using. Guillaume ----- Original Message ----- From: "j=FCrgen h=F6ller [werk3AT]" <jue...@we...> To: <spr...@li...> Sent: Sunday, May 23, 2004 4:45 PM Subject: Re: [Springframework-developer] Cleanup of context resources on webapp reload I've just spent about 10 hours profiling Spring, using the Image = Database and Petclinic samples. (BTW, I've used an evaluation version of = JProfiler from ej-technologies - nice product!) Although I still don't completely understand the garbage collection behavior, I've figured out the following issues. Each of them simply prevents the respective classes from getting garbage collected on destruction of the class loader (e.g. on Tomcat web app shutdown). - A classic singleton with a class variable holding the object. I've reworked GlobalAdvisorAdapterRegistry and SQLErrorCodesFactory to hold = the respective singleton as a WeakReference. - A static cache. I've reworked CachedIntrospectionResults to use a WeakHashMap with WeakReferences as values. - A ThreadLocal with a default other than null. I've reworked TransactionSynchronizationManager to use null as default for the = resource map, setting a HashMap there on demand, removing the entire HashMap when unbinding the last resource. - Constants that define a full object. We have a number of those, for example ClassFilters.TRUE and AdvisedSupport.EMPTY_TARGET_SOURCE. I've = tried for quite a while, but I haven't been able to figure out a way to define such constants such that they will be garbage collected. The latter programming style is not uncommon, so I really don't = understand why it causes trouble with garbage collection. Hibernate uses a similar style for its FlushMode, for example. In general, other frameworks like CGLIB, Hibernate, Velocity have huge resource leaks on web app shutdown, while just the constants issue = remains with Spring now. As long as those huge third-party leaks are not = addressed, I'm not worried at all by the single remaining Spring issue. As I initially said, we shouldn't exaggerate the problem, as it = basically just affects hot reloading of web apps - mainly a development feature anyway. We need to make that clear to users too, to avoid comments a la "Spring is not usable for real apps because it leaks on hot = redeployment". Please, everybody, give the current CVS head a sanity check tomorrow. = There shouldn't be any issues: the test suite passes, the sample apps run properly. Still, I'd feel more comfortable if we make sure that no = subtle side effects have been introduced. For this reason, I will delay release 1.0.2 till tomorrow night. Juergen ________________________________ Von: spr...@li... im Auftrag = von j=FCrgen h=F6ller [werk3AT] Gesendet: Sa 22.05.2004 15:52 An: spr...@li... Betreff: Re: [Springframework-developer] Cleanup of context resources on webapp reload So basically, all static caches cause resource leaks when restarting a Tomcat web app? I wonder why this happens... The class loader should completely dissolve all classes that it has loaded in its lifetime, including static caches. Or have I misunderstood something here? Anyone having in-detail experience with handling such a scenario? Juergen ________________________________ Von: spr...@li... im Auftrag = von Dmitriy Kopylenko Gesendet: Di 04.05.2004 18:02 An: spr...@li... Betreff: Re: [Springframework-developer] Cleanup of context resources on webapp reload Well, for instance SQLErrorCodesFactory is a singleton(GoF, not Spring) which caches SQLErrorCodes internally in the Map with strong references. = Again, I don't know if trying to use WeakHashMap there would do the trick... Dmitriy Tim Kettering wrote: > > I looked at it some more this morning, and basically what I did was > start up tomcat w/ the webapp in the profiler, then after it was done > starting up I used tomcat's manager to stop the context. This should > destroy all resources related to the context. Here is a list of > spring related stuff that still were in memory after the context was > closed. Other stuff was cleaned up just fine. > > org.springframework.beans.CachedIntrospectionResults > org.springframework.jdbc.support.SQLCodes > org.springframework.core.Constants > org.springframework.aop.framework.AdvisedSupport$1 > org.springframework.aop.framework.adapter.BeforeAdviceAdapter > org.springframework.aop.framework.adapter.AfterReturningAdviceAdapter > org.springframework.jdbc.support.SQLErrorCodesFactory > = org.springframework.transaction.support.TransactionSynchronizationManage > r$1 > org.springframework.transaction.interceptor.RollbackRuleAttribute > org.springframework.aop.framework.adapter.ThrosAdviceAdapter > org.springframework.aop.Pointcut$1 > org.springframework.aop.framework.adapter.GlobalAdvisorAdapterRegistry > > On May 3, 2004, at 3:28 PM, Dmitriy Kopylenko wrote: > >> I'm just wondering, would the use of WeakHashMap in >> CachedIntrospectionResults help? >> >> Dmitriy. >> >> Tim Kettering wrote: >> >>> I posted this to the users list last week and did not receive any >>> reply on it, so I'm posting it again here on the developer list, in >>> hopes i could get an reply from someone here. I'm trying to >>> determine if its something I should be doing myself, or if hte >>> spring context should be cleaning up those resources by itself on >>> the .close() call. Further profiling shows that there are >>> duplicate instances of SQLError and hibernate proxy classes = hanging >>> around afterwards too. Other objects do get cleaned up properly. >>> -------- >>> Hi everyone, >>> We're (meaning me) looking into some resource leaks that are >>> occuring when our webapp context gets reloaded. I found that >>> context.close() needs to be called on the destroy() method of >>> plugin we're using, and it works for a good majority of the = objects >>> we were seeing leaked, but there are some objects that I'm unable >>> to make go away. Object in question is the: >>> org.springframework.beans.CachedIntrospectionResults >>> Whenever I reload the context - the profiler I'm using shows that I >>> have essentially a duplicate group of those objects (same instance >>> count) as the original, and successive reloads will continue to >>> duplicate this. >>> The profiler also shows the final reference to those objects like = this: >>> 100% - 1008 bytes - 63 alloc. >>> = org.springframework.context.support.ClassPathXmlApplicationContext.<in >>> it > >>> So basically I guess what I'm asking is for ideas or suggestions on >>> how I could get those to clean up. This bean doesnt show up in the >>> Spring javadocs. And looking in CVS says its a package level bean, >>> not for application use, so I'm thinking that closing the context >>> should (in theory) clean this up? Thanks in advance. >>> -tim >>> ------------------------------------------------------- >>> This SF.Net email is sponsored by: Oracle 10g >>> Get certified on the hottest thing ever to hit the market... Oracle >>> 10g. Take an Oracle 10g class now, and we'll give you the exam FREE. >>> http://ads.osdn.com/?ad_id=3D3149&alloc_id=3D8166&op=3Dclick >>> _______________________________________________ >>> Springframework-developer mailing list >>> Spr...@li... >>> = https://lists.sourceforge.net/lists/listinfo/springframework-developer >> >> >> >> >> ------------------------------------------------------- >> This SF.Net email is sponsored by: Oracle 10g >> Get certified on the hottest thing ever to hit the market... Oracle >> 10g. Take an Oracle 10g class now, and we'll give you the exam FREE. >> http://ads.osdn.com/?ad_id=3D3149&alloc_id=3D8166&op=3Dclick >> _______________________________________________ >> Springframework-developer mailing list >> Spr...@li... >> = https://lists.sourceforge.net/lists/listinfo/springframework-developer >> > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: Oracle 10g > Get certified on the hottest thing ever to hit the market... Oracle = 10g. > Take an Oracle 10g class now, and we'll give you the exam FREE. > http://ads.osdn.com/?ad_id=3D3149&alloc_id=3D8166&op=3Dclick > _______________________________________________ > Springframework-developer mailing list > Spr...@li... > https://lists.sourceforge.net/lists/listinfo/springframework-developer ------------------------------------------------------- This SF.Net email is sponsored by: Oracle 10g Get certified on the hottest thing ever to hit the market... Oracle 10g. Take an Oracle 10g class now, and we'll give you the exam FREE. http://ads.osdn.com/?ad_id=3D3149&alloc_id=3D8166&op=3Dclick _______________________________________________ Springframework-developer mailing list Spr...@li... https://lists.sourceforge.net/lists/listinfo/springframework-developer ------------------------------------------------------- This SF.Net email is sponsored by: Oracle 10g Get certified on the hottest thing ever to hit the market... Oracle 10g. Take an Oracle 10g class now, and we'll give you the exam FREE. http://ads.osdn.com/?ad_id149&alloc_id=8166&op=3Dick _______________________________________________ Springframework-developer mailing list Spr...@li... https://lists.sourceforge.net/lists/listinfo/springframework-developer ------------------------------------------------------- This SF.Net email is sponsored by: Oracle 10g Get certified on the hottest thing ever to hit the market... Oracle 10g. Take an Oracle 10g class now, and we'll give you the exam FREE. http://ads.osdn.com/?ad_id149&alloc_id=8166&op=3Dick _______________________________________________ Springframework-developer mailing list Spr...@li... https://lists.sourceforge.net/lists/listinfo/springframework-developer ------------------------------------------------------- This SF.Net email is sponsored by: Oracle 10g Get certified on the hottest thing ever to hit the market... Oracle 10g. Take an Oracle 10g class now, and we'll give you the exam FREE. http://ads.osdn.com/?ad_id=3D3149&alloc_id=3D8166&op=3Dclick _______________________________________________ Springframework-developer mailing list Spr...@li... https://lists.sourceforge.net/lists/listinfo/springframework-developer ------------------------------------------------------- This SF.Net email is sponsored by: Oracle 10g Get certified on the hottest thing ever to hit the market... Oracle 10g. Take an Oracle 10g class now, and we'll give you the exam FREE. http://ads.osdn.com/?ad_id149&alloc_id=8166&op=3Dick _______________________________________________ Springframework-developer mailing list Spr...@li... https://lists.sourceforge.net/lists/listinfo/springframework-developer |
|
From: Arto P. <art...@of...> - 2004-05-24 08:09:22
|
Hi! I have developed couple web application to tomcat, and i have used those Juergen's mentioned versions of tomcat, and i'm positive sure that there is memory leak in tomcat, which appear when you install web application by copying war to webapps directory. I think that leak appears to tomcat, when it moved to use catalina source, earlier (maby version 3.x) there wasn't that leak. I have tested it with couple JVM version's and windows and linux enviroments, but only with sun's virtual machine, and leak appears every time, so i'm sure that problem is in tomcat. I have used Hibernate&Template in every application, and Springframework some application's and leak appears. It's easy to notice it by looking how much JVM reserves memory, and copy that war again and again to webapps directory. Artsi. On Mon, 2004-05-24 at 10:20, jürgen höller [werk3AT] wrote: > FYI, I've tested both Tomcat 5.0.18 and 4.1.27 - same behavior with both. > > BTW, a couple of related posts from the Resin mailing list: > > http://www.caucho.com/support/resin-interest/0404/0117.html > http://www.caucho.com/support/resin-interest/0111/0164.html <http://www.caucho.com/support/resin-interest/0111/0164.html> > > Juergen > > > ________________________________ > > Von: spr...@li... im Auftrag von jürgen höller [werk3AT] > Gesendet: Mo 24.05.2004 08:42 > An: spr...@li... > Betreff: Re: [Springframework-developer] Cleanup of context resources on webapp reload > > > > I've told JProfiler to explicitly run garbage collection - a number of times, actually - before I've had a look at the heap. So I'm sure that those remaining objects were not garbage-collected, and probably would have stayed around in the VM forever... > > I'm aware that it seems odd, but this issue just seems to affect specific static fields: BeanWrapperImpl's defaultEditors did not cause a leak, but CachedIntrospectionResults' classCache did. Normal constants or static logger fields didn't, but "full object" constants like ClassFilters.TRUE did. > > CachedIntrospectionResults' classCache uses the Class as key; the value, a CachedIntrospectionResult object, also refers to the key Class. Consequently, I had to use a WeakHashMap *with WeakReferences as values* to see proper garbage collection. Note that this static cache contains instances of its containing class as values. > > I've run all my tests x times to make sure that I could trust my eyes. I've even undone the WeakReference changes again, and voila, there were the leaks again. I'd be happy to learn more about how the garbage collector works here... All I can state at this point of time is that the changes did cause an obvious difference in terms of resource leaks. > > Juergen > > > ________________________________ > > Von: spr...@li... im Auftrag von Guillaume Poirier > Gesendet: Mo 24.05.2004 05:17 > An: spr...@li... > Betreff: Re: [Springframework-developer] Cleanup of context resources on webapp reload > > > > The "resource leak" caused by a singleton when a webapp's classloader is > thrown away is only temorary, the unused classes and the classloader will be > eventually garbage collected and the resources will be freed. The only > thing that could prevent that is if there was a something in the server's > classloader that still had a reference on an object or a class of the child > classloader. > > Unless there's a bug in Tomcat or in the application code, I really can't > see how the classes won't be eventually garbage collected when the JVM needs > memory. And anyway, if there was indeed a leak because SQLErrorCodesFactory > is a singleton, why wouldn't there be one for each static fields such as > constants? > > Are you sure that JProfiler does not disable garbage collecting in order to > make its profiling? I know that in many of the JVMPI method calls are done > with garbage collecting off. I suspect the above to be the cause of the > "resource leak", rather than any singleton that Spring might be using. > > Guillaume > > ----- Original Message ----- > From: "jürgen höller [werk3AT]" <jue...@we...> > To: <spr...@li...> > Sent: Sunday, May 23, 2004 4:45 PM > Subject: Re: [Springframework-developer] Cleanup of context resources on > webapp reload > > > I've just spent about 10 hours profiling Spring, using the Image Database > and Petclinic samples. (BTW, I've used an evaluation version of JProfiler > from ej-technologies - nice product!) > > Although I still don't completely understand the garbage collection > behavior, I've figured out the following issues. Each of them simply > prevents the respective classes from getting garbage collected on > destruction of the class loader (e.g. on Tomcat web app shutdown). > > - A classic singleton with a class variable holding the object. I've > reworked GlobalAdvisorAdapterRegistry and SQLErrorCodesFactory to hold the > respective singleton as a WeakReference. > > - A static cache. I've reworked CachedIntrospectionResults to use a > WeakHashMap with WeakReferences as values. > > - A ThreadLocal with a default other than null. I've reworked > TransactionSynchronizationManager to use null as default for the resource > map, setting a HashMap there on demand, removing the entire HashMap when > unbinding the last resource. > > - Constants that define a full object. We have a number of those, for > example ClassFilters.TRUE and AdvisedSupport.EMPTY_TARGET_SOURCE. I've tried > for quite a while, but I haven't been able to figure out a way to define > such constants such that they will be garbage collected. > > The latter programming style is not uncommon, so I really don't understand > why it causes trouble with garbage collection. Hibernate uses a similar > style for its FlushMode, for example. > > In general, other frameworks like CGLIB, Hibernate, Velocity have huge > resource leaks on web app shutdown, while just the constants issue remains > with Spring now. As long as those huge third-party leaks are not addressed, > I'm not worried at all by the single remaining Spring issue. > > As I initially said, we shouldn't exaggerate the problem, as it basically > just affects hot reloading of web apps - mainly a development feature > anyway. We need to make that clear to users too, to avoid comments a la > "Spring is not usable for real apps because it leaks on hot redeployment". > > Please, everybody, give the current CVS head a sanity check tomorrow. There > shouldn't be any issues: the test suite passes, the sample apps run > properly. Still, I'd feel more comfortable if we make sure that no subtle > side effects have been introduced. > > For this reason, I will delay release 1.0.2 till tomorrow night. > > Juergen > > > ________________________________ > > Von: spr...@li... im Auftrag von > jürgen höller [werk3AT] > Gesendet: Sa 22.05.2004 15:52 > An: spr...@li... > Betreff: Re: [Springframework-developer] Cleanup of context resources on > webapp reload > > > > So basically, all static caches cause resource leaks when restarting a > Tomcat web app? I wonder why this happens... The class loader should > completely dissolve all classes that it has loaded in its lifetime, > including static caches. Or have I misunderstood something here? Anyone > having in-detail experience with handling such a scenario? > > Juergen > > > ________________________________ > > Von: spr...@li... im Auftrag von > Dmitriy Kopylenko > Gesendet: Di 04.05.2004 18:02 > An: spr...@li... > Betreff: Re: [Springframework-developer] Cleanup of context resources on > webapp reload > > > > Well, > > for instance SQLErrorCodesFactory is a singleton(GoF, not Spring) which > caches SQLErrorCodes internally in the Map with strong references. Again, I > don't know if trying > to use WeakHashMap there would do the trick... > > Dmitriy > > Tim Kettering wrote: > > > > > I looked at it some more this morning, and basically what I did was > > start up tomcat w/ the webapp in the profiler, then after it was done > > starting up I used tomcat's manager to stop the context. This should > > destroy all resources related to the context. Here is a list of > > spring related stuff that still were in memory after the context was > > closed. Other stuff was cleaned up just fine. > > > > org.springframework.beans.CachedIntrospectionResults > > org.springframework.jdbc.support.SQLCodes > > org.springframework.core.Constants > > org.springframework.aop.framework.AdvisedSupport$1 > > org.springframework.aop.framework.adapter.BeforeAdviceAdapter > > org.springframework.aop.framework.adapter.AfterReturningAdviceAdapter > > org.springframework.jdbc.support.SQLErrorCodesFactory > > org.springframework.transaction.support.TransactionSynchronizationManage > > r$1 > > org.springframework.transaction.interceptor.RollbackRuleAttribute > > org.springframework.aop.framework.adapter.ThrosAdviceAdapter > > org.springframework.aop.Pointcut$1 > > org.springframework.aop.framework.adapter.GlobalAdvisorAdapterRegistry > > > > On May 3, 2004, at 3:28 PM, Dmitriy Kopylenko wrote: > > > >> I'm just wondering, would the use of WeakHashMap in > >> CachedIntrospectionResults help? > >> > >> Dmitriy. > >> > >> Tim Kettering wrote: > >> > >>> I posted this to the users list last week and did not receive any > >>> reply on it, so I'm posting it again here on the developer list, in > >>> hopes i could get an reply from someone here. I'm trying to > >>> determine if its something I should be doing myself, or if hte > >>> spring context should be cleaning up those resources by itself on > >>> the .close() call. Further profiling shows that there are > >>> duplicate instances of SQLError and hibernate proxy classes hanging > >>> around afterwards too. Other objects do get cleaned up properly. > >>> -------- > >>> Hi everyone, > >>> We're (meaning me) looking into some resource leaks that are > >>> occuring when our webapp context gets reloaded. I found that > >>> context.close() needs to be called on the destroy() method of > >>> plugin we're using, and it works for a good majority of the objects > >>> we were seeing leaked, but there are some objects that I'm unable > >>> to make go away. Object in question is the: > >>> org.springframework.beans.CachedIntrospectionResults > >>> Whenever I reload the context - the profiler I'm using shows that I > >>> have essentially a duplicate group of those objects (same instance > >>> count) as the original, and successive reloads will continue to > >>> duplicate this. > >>> The profiler also shows the final reference to those objects like this: > >>> 100% - 1008 bytes - 63 alloc. > >>> org.springframework.context.support.ClassPathXmlApplicationContext.<in > >>> it > > >>> So basically I guess what I'm asking is for ideas or suggestions on > >>> how I could get those to clean up. This bean doesnt show up in the > >>> Spring javadocs. And looking in CVS says its a package level bean, > >>> not for application use, so I'm thinking that closing the context > >>> should (in theory) clean this up? Thanks in advance. > >>> -tim > >>> ------------------------------------------------------- > >>> This SF.Net email is sponsored by: Oracle 10g > >>> Get certified on the hottest thing ever to hit the market... Oracle > >>> 10g. Take an Oracle 10g class now, and we'll give you the exam FREE. > >>> http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click > >>> _______________________________________________ > >>> Springframework-developer mailing list > >>> Spr...@li... > >>> https://lists.sourceforge.net/lists/listinfo/springframework-developer > >> > >> > >> > >> > >> ------------------------------------------------------- > >> This SF.Net email is sponsored by: Oracle 10g > >> Get certified on the hottest thing ever to hit the market... Oracle > >> 10g. Take an Oracle 10g class now, and we'll give you the exam FREE. > >> http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click > >> _______________________________________________ > >> Springframework-developer mailing list > >> Spr...@li... > >> https://lists.sourceforge.net/lists/listinfo/springframework-developer > >> > > > > > > > > ------------------------------------------------------- > > This SF.Net email is sponsored by: Oracle 10g > > Get certified on the hottest thing ever to hit the market... Oracle 10g. > > Take an Oracle 10g class now, and we'll give you the exam FREE. > > http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click > > _______________________________________________ > > Springframework-developer mailing list > > Spr...@li... > > https://lists.sourceforge.net/lists/listinfo/springframework-developer > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: Oracle 10g > Get certified on the hottest thing ever to hit the market... Oracle 10g. > Take an Oracle 10g class now, and we'll give you the exam FREE. > http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click > _______________________________________________ > Springframework-developer mailing list > Spr...@li... > https://lists.sourceforge.net/lists/listinfo/springframework-developer > > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: Oracle 10g > Get certified on the hottest thing ever to hit the market... Oracle 10g. > Take an Oracle 10g class now, and we'll give you the exam FREE. > http://ads.osdn.com/?ad_id149&alloc_id66&op=ick > _______________________________________________ > Springframework-developer mailing list > Spr...@li... > https://lists.sourceforge.net/lists/listinfo/springframework-developer > > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: Oracle 10g > Get certified on the hottest thing ever to hit the market... Oracle 10g. > Take an Oracle 10g class now, and we'll give you the exam FREE. > http://ads.osdn.com/?ad_id149&alloc_id66&op=ick > _______________________________________________ > Springframework-developer mailing list > Spr...@li... > https://lists.sourceforge.net/lists/listinfo/springframework-developer > > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: Oracle 10g > Get certified on the hottest thing ever to hit the market... Oracle 10g. > Take an Oracle 10g class now, and we'll give you the exam FREE. > http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click > _______________________________________________ > Springframework-developer mailing list > Spr...@li... > https://lists.sourceforge.net/lists/listinfo/springframework-developer > > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: Oracle 10g > Get certified on the hottest thing ever to hit the market... Oracle 10g. > Take an Oracle 10g class now, and we'll give you the exam FREE. > http://ads.osdn.com/?ad_id149&alloc_id66&op=ick > _______________________________________________ > Springframework-developer mailing list > Spr...@li... > https://lists.sourceforge.net/lists/listinfo/springframework-developer > > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: Oracle 10g > Get certified on the hottest thing ever to hit the market... Oracle 10g. > Take an Oracle 10g class now, and we'll give you the exam FREE. > http://ads.osdn.com/?ad_id149&alloc_id66&op=click > _______________________________________________ > Springframework-developer mailing list > Spr...@li... > https://lists.sourceforge.net/lists/listinfo/springframework-developer |
|
From: Alef A. <al...@jt...> - 2004-05-24 08:17:06
|
There's a thread on the tapestry user list as well about this, they seem = to conclude that OC4J has the same issue... http://www.caddr.com/macho/archives/tapestry-users/2004-3/4939.html Also, there was a discussion going on at the forums, I'll update the = people involved there... Alef > -----Original Message----- > From: spr...@li... > [mailto:spr...@li...] On = Behalf > Of j=FCrgen h=F6ller [werk3AT] > Sent: Monday, May 24, 2004 9:21 AM > To: spr...@li... > Subject: Re: [Springframework-developer] Cleanup of context resources = on > webapp reload >=20 > FYI, I've tested both Tomcat 5.0.18 and 4.1.27 - same behavior with = both. >=20 > BTW, a couple of related posts from the Resin mailing list: >=20 > http://www.caucho.com/support/resin-interest/0404/0117.html > http://www.caucho.com/support/resin-interest/0111/0164.html > <http://www.caucho.com/support/resin-interest/0111/0164.html> >=20 > Juergen >=20 >=20 > ________________________________ >=20 > Von: spr...@li... im Auftrag = von > j=FCrgen h=F6ller [werk3AT] > Gesendet: Mo 24.05.2004 08:42 > An: spr...@li... > Betreff: Re: [Springframework-developer] Cleanup of context resources = on > webapp reload >=20 >=20 >=20 > I've told JProfiler to explicitly run garbage collection - a number of > times, actually - before I've had a look at the heap. So I'm sure that > those remaining objects were not garbage-collected, and probably would > have stayed around in the VM forever... >=20 > I'm aware that it seems odd, but this issue just seems to affect = specific > static fields: BeanWrapperImpl's defaultEditors did not cause a leak, = but > CachedIntrospectionResults' classCache did. Normal constants or static > logger fields didn't, but "full object" constants like = ClassFilters.TRUE > did. >=20 > CachedIntrospectionResults' classCache uses the Class as key; the = value, a > CachedIntrospectionResult object, also refers to the key Class. > Consequently, I had to use a WeakHashMap *with WeakReferences as = values* > to see proper garbage collection. Note that this static cache contains > instances of its containing class as values. >=20 > I've run all my tests x times to make sure that I could trust my eyes. > I've even undone the WeakReference changes again, and voila, there = were > the leaks again. I'd be happy to learn more about how the garbage > collector works here... All I can state at this point of time is that = the > changes did cause an obvious difference in terms of resource leaks. >=20 > Juergen >=20 >=20 > ________________________________ >=20 > Von: spr...@li... im Auftrag = von > Guillaume Poirier > Gesendet: Mo 24.05.2004 05:17 > An: spr...@li... > Betreff: Re: [Springframework-developer] Cleanup of context resources = on > webapp reload >=20 >=20 >=20 > The "resource leak" caused by a singleton when a webapp's classloader = is > thrown away is only temorary, the unused classes and the classloader = will > be > eventually garbage collected and the resources will be freed. The = only > thing that could prevent that is if there was a something in the = server's > classloader that still had a reference on an object or a class of the > child > classloader. >=20 > Unless there's a bug in Tomcat or in the application code, I really = can't > see how the classes won't be eventually garbage collected when the JVM > needs > memory. And anyway, if there was indeed a leak because > SQLErrorCodesFactory > is a singleton, why wouldn't there be one for each static fields such = as > constants? >=20 > Are you sure that JProfiler does not disable garbage collecting in = order > to > make its profiling? I know that in many of the JVMPI method calls are > done > with garbage collecting off. I suspect the above to be the cause of = the > "resource leak", rather than any singleton that Spring might be using. >=20 > Guillaume >=20 > ----- Original Message ----- > From: "j=FCrgen h=F6ller [werk3AT]" <jue...@we...> > To: <spr...@li...> > Sent: Sunday, May 23, 2004 4:45 PM > Subject: Re: [Springframework-developer] Cleanup of context resources = on > webapp reload >=20 >=20 > I've just spent about 10 hours profiling Spring, using the Image = Database > and Petclinic samples. (BTW, I've used an evaluation version of = JProfiler > from ej-technologies - nice product!) >=20 > Although I still don't completely understand the garbage collection > behavior, I've figured out the following issues. Each of them simply > prevents the respective classes from getting garbage collected on > destruction of the class loader (e.g. on Tomcat web app shutdown). >=20 > - A classic singleton with a class variable holding the object. I've > reworked GlobalAdvisorAdapterRegistry and SQLErrorCodesFactory to hold = the > respective singleton as a WeakReference. >=20 > - A static cache. I've reworked CachedIntrospectionResults to use a > WeakHashMap with WeakReferences as values. >=20 > - A ThreadLocal with a default other than null. I've reworked > TransactionSynchronizationManager to use null as default for the = resource > map, setting a HashMap there on demand, removing the entire HashMap = when > unbinding the last resource. >=20 > - Constants that define a full object. We have a number of those, for > example ClassFilters.TRUE and AdvisedSupport.EMPTY_TARGET_SOURCE. I've > tried > for quite a while, but I haven't been able to figure out a way to = define > such constants such that they will be garbage collected. >=20 > The latter programming style is not uncommon, so I really don't = understand > why it causes trouble with garbage collection. Hibernate uses a = similar > style for its FlushMode, for example. >=20 > In general, other frameworks like CGLIB, Hibernate, Velocity have huge > resource leaks on web app shutdown, while just the constants issue = remains > with Spring now. As long as those huge third-party leaks are not > addressed, > I'm not worried at all by the single remaining Spring issue. >=20 > As I initially said, we shouldn't exaggerate the problem, as it = basically > just affects hot reloading of web apps - mainly a development feature > anyway. We need to make that clear to users too, to avoid comments a = la > "Spring is not usable for real apps because it leaks on hot = redeployment". >=20 > Please, everybody, give the current CVS head a sanity check tomorrow. > There > shouldn't be any issues: the test suite passes, the sample apps run > properly. Still, I'd feel more comfortable if we make sure that no = subtle > side effects have been introduced. >=20 > For this reason, I will delay release 1.0.2 till tomorrow night. >=20 > Juergen >=20 >=20 > ________________________________ >=20 > Von: spr...@li... im Auftrag = von > j=FCrgen h=F6ller [werk3AT] > Gesendet: Sa 22.05.2004 15:52 > An: spr...@li... > Betreff: Re: [Springframework-developer] Cleanup of context resources = on > webapp reload >=20 >=20 >=20 > So basically, all static caches cause resource leaks when restarting a > Tomcat web app? I wonder why this happens... The class loader should > completely dissolve all classes that it has loaded in its lifetime, > including static caches. Or have I misunderstood something here? = Anyone > having in-detail experience with handling such a scenario? >=20 > Juergen >=20 >=20 > ________________________________ >=20 > Von: spr...@li... im Auftrag = von > Dmitriy Kopylenko > Gesendet: Di 04.05.2004 18:02 > An: spr...@li... > Betreff: Re: [Springframework-developer] Cleanup of context resources = on > webapp reload >=20 >=20 >=20 > Well, >=20 > for instance SQLErrorCodesFactory is a singleton(GoF, not Spring) = which > caches SQLErrorCodes internally in the Map with strong references. = Again, > I > don't know if trying > to use WeakHashMap there would do the trick... >=20 > Dmitriy >=20 > Tim Kettering wrote: >=20 > > > > I looked at it some more this morning, and basically what I did was > > start up tomcat w/ the webapp in the profiler, then after it was = done > > starting up I used tomcat's manager to stop the context. This = should > > destroy all resources related to the context. Here is a list of > > spring related stuff that still were in memory after the context was > > closed. Other stuff was cleaned up just fine. > > > > org.springframework.beans.CachedIntrospectionResults > > org.springframework.jdbc.support.SQLCodes > > org.springframework.core.Constants > > org.springframework.aop.framework.AdvisedSupport$1 > > org.springframework.aop.framework.adapter.BeforeAdviceAdapter > > = org.springframework.aop.framework.adapter.AfterReturningAdviceAdapter > > org.springframework.jdbc.support.SQLErrorCodesFactory > > = org.springframework.transaction.support.TransactionSynchronizationManage > > r$1 > > org.springframework.transaction.interceptor.RollbackRuleAttribute > > org.springframework.aop.framework.adapter.ThrosAdviceAdapter > > org.springframework.aop.Pointcut$1 > > = org.springframework.aop.framework.adapter.GlobalAdvisorAdapterRegistry > > > > On May 3, 2004, at 3:28 PM, Dmitriy Kopylenko wrote: > > > >> I'm just wondering, would the use of WeakHashMap in > >> CachedIntrospectionResults help? > >> > >> Dmitriy. > >> > >> Tim Kettering wrote: > >> > >>> I posted this to the users list last week and did not receive any > >>> reply on it, so I'm posting it again here on the developer list, = in > >>> hopes i could get an reply from someone here. I'm trying to > >>> determine if its something I should be doing myself, or if hte > >>> spring context should be cleaning up those resources by itself on > >>> the .close() call. Further profiling shows that there are > >>> duplicate instances of SQLError and hibernate proxy classes = hanging > >>> around afterwards too. Other objects do get cleaned up = properly. > >>> -------- > >>> Hi everyone, > >>> We're (meaning me) looking into some resource leaks that are > >>> occuring when our webapp context gets reloaded. I found that > >>> context.close() needs to be called on the destroy() method of > >>> plugin we're using, and it works for a good majority of the = objects > >>> we were seeing leaked, but there are some objects that I'm = unable > >>> to make go away. Object in question is the: > >>> org.springframework.beans.CachedIntrospectionResults > >>> Whenever I reload the context - the profiler I'm using shows that = I > >>> have essentially a duplicate group of those objects (same instance > >>> count) as the original, and successive reloads will continue to > >>> duplicate this. > >>> The profiler also shows the final reference to those objects like > this: > >>> 100% - 1008 bytes - 63 alloc. > >>> = org.springframework.context.support.ClassPathXmlApplicationContext.<in > >>> it > > >>> So basically I guess what I'm asking is for ideas or suggestions = on > >>> how I could get those to clean up. This bean doesnt show up in = the > >>> Spring javadocs. And looking in CVS says its a package level = bean, > >>> not for application use, so I'm thinking that closing the context > >>> should (in theory) clean this up? Thanks in advance. > >>> -tim > >>> ------------------------------------------------------- > >>> This SF.Net email is sponsored by: Oracle 10g > >>> Get certified on the hottest thing ever to hit the market... = Oracle > >>> 10g. Take an Oracle 10g class now, and we'll give you the exam = FREE. > >>> http://ads.osdn.com/?ad_id149&alloc_id=8166&op=3Dclick > >>> _______________________________________________ > >>> Springframework-developer mailing list > >>> Spr...@li... > >>> = https://lists.sourceforge.net/lists/listinfo/springframework-developer > >> > >> > >> > >> > >> ------------------------------------------------------- > >> This SF.Net email is sponsored by: Oracle 10g > >> Get certified on the hottest thing ever to hit the market... Oracle > >> 10g. Take an Oracle 10g class now, and we'll give you the exam = FREE. > >> http://ads.osdn.com/?ad_id149&alloc_id=8166&op=3Dclick > >> _______________________________________________ > >> Springframework-developer mailing list > >> Spr...@li... > >> = https://lists.sourceforge.net/lists/listinfo/springframework-developer > >> > > > > > > > > ------------------------------------------------------- > > This SF.Net email is sponsored by: Oracle 10g > > Get certified on the hottest thing ever to hit the market... Oracle = 10g. > > Take an Oracle 10g class now, and we'll give you the exam FREE. > > http://ads.osdn.com/?ad_id149&alloc_id=8166&op=3Dclick > > _______________________________________________ > > Springframework-developer mailing list > > Spr...@li... > > = https://lists.sourceforge.net/lists/listinfo/springframework-developer >=20 >=20 >=20 > ------------------------------------------------------- > This SF.Net email is sponsored by: Oracle 10g > Get certified on the hottest thing ever to hit the market... Oracle = 10g. > Take an Oracle 10g class now, and we'll give you the exam FREE. > http://ads.osdn.com/?ad_id149&alloc_id=8166&op=3Dclick > _______________________________________________ > Springframework-developer mailing list > Spr...@li... > https://lists.sourceforge.net/lists/listinfo/springframework-developer >=20 >=20 >=20 >=20 > ------------------------------------------------------- > This SF.Net email is sponsored by: Oracle 10g > Get certified on the hottest thing ever to hit the market... Oracle = 10g. > Take an Oracle 10g class now, and we'll give you the exam FREE. > http://ads.osdn.com/?ad_id149&alloc_id=8166&op=3Dick > _______________________________________________ > Springframework-developer mailing list > Spr...@li... > https://lists.sourceforge.net/lists/listinfo/springframework-developer >=20 >=20 >=20 >=20 > ------------------------------------------------------- > This SF.Net email is sponsored by: Oracle 10g > Get certified on the hottest thing ever to hit the market... Oracle = 10g. > Take an Oracle 10g class now, and we'll give you the exam FREE. > http://ads.osdn.com/?ad_id149&alloc_id=8166&op=3Dick > _______________________________________________ > Springframework-developer mailing list > Spr...@li... > https://lists.sourceforge.net/lists/listinfo/springframework-developer >=20 >=20 >=20 >=20 > ------------------------------------------------------- > This SF.Net email is sponsored by: Oracle 10g > Get certified on the hottest thing ever to hit the market... Oracle = 10g. > Take an Oracle 10g class now, and we'll give you the exam FREE. > http://ads.osdn.com/?ad_id149&alloc_id=8166&op=3Dclick > _______________________________________________ > Springframework-developer mailing list > Spr...@li... > https://lists.sourceforge.net/lists/listinfo/springframework-developer >=20 >=20 >=20 >=20 > ------------------------------------------------------- > This SF.Net email is sponsored by: Oracle 10g > Get certified on the hottest thing ever to hit the market... Oracle = 10g. > Take an Oracle 10g class now, and we'll give you the exam FREE. > http://ads.osdn.com/?ad_id149&alloc_id=8166&op=3Dick > _______________________________________________ > Springframework-developer mailing list > Spr...@li... > https://lists.sourceforge.net/lists/listinfo/springframework-developer >=20 >=20 >=20 >=20 > ------------------------------------------------------- > This SF.Net email is sponsored by: Oracle 10g > Get certified on the hottest thing ever to hit the market... Oracle = 10g. > Take an Oracle 10g class now, and we'll give you the exam FREE. > http://ads.osdn.com/?ad_id149&alloc_id=8166&op=3Dick > _______________________________________________ > Springframework-developer mailing list > Spr...@li... > https://lists.sourceforge.net/lists/listinfo/springframework-developer |
|
From: <jue...@we...> - 2004-05-24 08:46:36
|
Just finished tests on Resin 2.1.11 - exactly same behavior as with = Tomcat. This *is* a general class loader respectively garbage collection = issue, rather than a server-specific leak. =20 Consequently, with the changes I've committed yesterday, there is the = same significant benefit as with Tomcat. The only remaining issue are = the "full object" constants like ClassFilter.TRUE. =20 Juergen =20 ________________________________ Von: spr...@li... im Auftrag = von j=FCrgen h=F6ller [werk3AT] Gesendet: Mo 24.05.2004 09:20 An: spr...@li... Betreff: Re: [Springframework-developer] Cleanup of context resources on = webapp reload FYI, I've tested both Tomcat 5.0.18 and 4.1.27 - same behavior with = both. BTW, a couple of related posts from the Resin mailing list: http://www.caucho.com/support/resin-interest/0404/0117.html http://www.caucho.com/support/resin-interest/0111/0164.html = <http://www.caucho.com/support/resin-interest/0111/0164.html> Juergen ________________________________ Von: spr...@li... im Auftrag = von j=FCrgen h=F6ller [werk3AT] Gesendet: Mo 24.05.2004 08:42 An: spr...@li... Betreff: Re: [Springframework-developer] Cleanup of context resources on = webapp reload I've told JProfiler to explicitly run garbage collection - a number of = times, actually - before I've had a look at the heap. So I'm sure that = those remaining objects were not garbage-collected, and probably would = have stayed around in the VM forever... I'm aware that it seems odd, but this issue just seems to affect = specific static fields: BeanWrapperImpl's defaultEditors did not cause a = leak, but CachedIntrospectionResults' classCache did. Normal constants = or static logger fields didn't, but "full object" constants like = ClassFilters.TRUE did. CachedIntrospectionResults' classCache uses the Class as key; the value, = a CachedIntrospectionResult object, also refers to the key Class. = Consequently, I had to use a WeakHashMap *with WeakReferences as values* = to see proper garbage collection. Note that this static cache contains = instances of its containing class as values. I've run all my tests x times to make sure that I could trust my eyes. = I've even undone the WeakReference changes again, and voila, there were = the leaks again. I'd be happy to learn more about how the garbage = collector works here... All I can state at this point of time is that = the changes did cause an obvious difference in terms of resource leaks. Juergen ________________________________ Von: spr...@li... im Auftrag = von Guillaume Poirier Gesendet: Mo 24.05.2004 05:17 An: spr...@li... Betreff: Re: [Springframework-developer] Cleanup of context resources on = webapp reload The "resource leak" caused by a singleton when a webapp's classloader is thrown away is only temorary, the unused classes and the classloader = will be eventually garbage collected and the resources will be freed. The only thing that could prevent that is if there was a something in the = server's classloader that still had a reference on an object or a class of the = child classloader. Unless there's a bug in Tomcat or in the application code, I really = can't see how the classes won't be eventually garbage collected when the JVM = needs memory. And anyway, if there was indeed a leak because = SQLErrorCodesFactory is a singleton, why wouldn't there be one for each static fields such as constants? Are you sure that JProfiler does not disable garbage collecting in order = to make its profiling? I know that in many of the JVMPI method calls are = done with garbage collecting off. I suspect the above to be the cause of the "resource leak", rather than any singleton that Spring might be using. Guillaume ----- Original Message ----- From: "j=FCrgen h=F6ller [werk3AT]" <jue...@we...> To: <spr...@li...> Sent: Sunday, May 23, 2004 4:45 PM Subject: Re: [Springframework-developer] Cleanup of context resources on webapp reload I've just spent about 10 hours profiling Spring, using the Image = Database and Petclinic samples. (BTW, I've used an evaluation version of = JProfiler from ej-technologies - nice product!) Although I still don't completely understand the garbage collection behavior, I've figured out the following issues. Each of them simply prevents the respective classes from getting garbage collected on destruction of the class loader (e.g. on Tomcat web app shutdown). - A classic singleton with a class variable holding the object. I've reworked GlobalAdvisorAdapterRegistry and SQLErrorCodesFactory to hold = the respective singleton as a WeakReference. - A static cache. I've reworked CachedIntrospectionResults to use a WeakHashMap with WeakReferences as values. - A ThreadLocal with a default other than null. I've reworked TransactionSynchronizationManager to use null as default for the = resource map, setting a HashMap there on demand, removing the entire HashMap when unbinding the last resource. - Constants that define a full object. We have a number of those, for example ClassFilters.TRUE and AdvisedSupport.EMPTY_TARGET_SOURCE. I've = tried for quite a while, but I haven't been able to figure out a way to define such constants such that they will be garbage collected. The latter programming style is not uncommon, so I really don't = understand why it causes trouble with garbage collection. Hibernate uses a similar style for its FlushMode, for example. In general, other frameworks like CGLIB, Hibernate, Velocity have huge resource leaks on web app shutdown, while just the constants issue = remains with Spring now. As long as those huge third-party leaks are not = addressed, I'm not worried at all by the single remaining Spring issue. As I initially said, we shouldn't exaggerate the problem, as it = basically just affects hot reloading of web apps - mainly a development feature anyway. We need to make that clear to users too, to avoid comments a la "Spring is not usable for real apps because it leaks on hot = redeployment". Please, everybody, give the current CVS head a sanity check tomorrow. = There shouldn't be any issues: the test suite passes, the sample apps run properly. Still, I'd feel more comfortable if we make sure that no = subtle side effects have been introduced. For this reason, I will delay release 1.0.2 till tomorrow night. Juergen ________________________________ Von: spr...@li... im Auftrag = von j=FCrgen h=F6ller [werk3AT] Gesendet: Sa 22.05.2004 15:52 An: spr...@li... Betreff: Re: [Springframework-developer] Cleanup of context resources on webapp reload So basically, all static caches cause resource leaks when restarting a Tomcat web app? I wonder why this happens... The class loader should completely dissolve all classes that it has loaded in its lifetime, including static caches. Or have I misunderstood something here? Anyone having in-detail experience with handling such a scenario? Juergen ________________________________ Von: spr...@li... im Auftrag = von Dmitriy Kopylenko Gesendet: Di 04.05.2004 18:02 An: spr...@li... Betreff: Re: [Springframework-developer] Cleanup of context resources on webapp reload Well, for instance SQLErrorCodesFactory is a singleton(GoF, not Spring) which caches SQLErrorCodes internally in the Map with strong references. = Again, I don't know if trying to use WeakHashMap there would do the trick... Dmitriy Tim Kettering wrote: > > I looked at it some more this morning, and basically what I did was > start up tomcat w/ the webapp in the profiler, then after it was done > starting up I used tomcat's manager to stop the context. This should > destroy all resources related to the context. Here is a list of > spring related stuff that still were in memory after the context was > closed. Other stuff was cleaned up just fine. > > org.springframework.beans.CachedIntrospectionResults > org.springframework.jdbc.support.SQLCodes > org.springframework.core.Constants > org.springframework.aop.framework.AdvisedSupport$1 > org.springframework.aop.framework.adapter.BeforeAdviceAdapter > org.springframework.aop.framework.adapter.AfterReturningAdviceAdapter > org.springframework.jdbc.support.SQLErrorCodesFactory > = org.springframework.transaction.support.TransactionSynchronizationManage > r$1 > org.springframework.transaction.interceptor.RollbackRuleAttribute > org.springframework.aop.framework.adapter.ThrosAdviceAdapter > org.springframework.aop.Pointcut$1 > org.springframework.aop.framework.adapter.GlobalAdvisorAdapterRegistry > > On May 3, 2004, at 3:28 PM, Dmitriy Kopylenko wrote: > >> I'm just wondering, would the use of WeakHashMap in >> CachedIntrospectionResults help? >> >> Dmitriy. >> >> Tim Kettering wrote: >> >>> I posted this to the users list last week and did not receive any >>> reply on it, so I'm posting it again here on the developer list, in >>> hopes i could get an reply from someone here. I'm trying to >>> determine if its something I should be doing myself, or if hte >>> spring context should be cleaning up those resources by itself on >>> the .close() call. Further profiling shows that there are >>> duplicate instances of SQLError and hibernate proxy classes = hanging >>> around afterwards too. Other objects do get cleaned up properly. >>> -------- >>> Hi everyone, >>> We're (meaning me) looking into some resource leaks that are >>> occuring when our webapp context gets reloaded. I found that >>> context.close() needs to be called on the destroy() method of >>> plugin we're using, and it works for a good majority of the = objects >>> we were seeing leaked, but there are some objects that I'm unable >>> to make go away. Object in question is the: >>> org.springframework.beans.CachedIntrospectionResults >>> Whenever I reload the context - the profiler I'm using shows that I >>> have essentially a duplicate group of those objects (same instance >>> count) as the original, and successive reloads will continue to >>> duplicate this. >>> The profiler also shows the final reference to those objects like = this: >>> 100% - 1008 bytes - 63 alloc. >>> = org.springframework.context.support.ClassPathXmlApplicationContext.<in >>> it > >>> So basically I guess what I'm asking is for ideas or suggestions on >>> how I could get those to clean up. This bean doesnt show up in the >>> Spring javadocs. And looking in CVS says its a package level bean, >>> not for application use, so I'm thinking that closing the context >>> should (in theory) clean this up? Thanks in advance. >>> -tim >>> ------------------------------------------------------- >>> This SF.Net email is sponsored by: Oracle 10g >>> Get certified on the hottest thing ever to hit the market... Oracle >>> 10g. Take an Oracle 10g class now, and we'll give you the exam FREE. >>> http://ads.osdn.com/?ad_id=3D3149&alloc_id=3D8166&op=3Dclick >>> _______________________________________________ >>> Springframework-developer mailing list >>> Spr...@li... >>> = https://lists.sourceforge.net/lists/listinfo/springframework-developer >> >> >> >> >> ------------------------------------------------------- >> This SF.Net email is sponsored by: Oracle 10g >> Get certified on the hottest thing ever to hit the market... Oracle >> 10g. Take an Oracle 10g class now, and we'll give you the exam FREE. >> http://ads.osdn.com/?ad_id=3D3149&alloc_id=3D8166&op=3Dclick >> _______________________________________________ >> Springframework-developer mailing list >> Spr...@li... >> = https://lists.sourceforge.net/lists/listinfo/springframework-developer >> > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: Oracle 10g > Get certified on the hottest thing ever to hit the market... Oracle = 10g. > Take an Oracle 10g class now, and we'll give you the exam FREE. > http://ads.osdn.com/?ad_id=3D3149&alloc_id=3D8166&op=3Dclick > _______________________________________________ > Springframework-developer mailing list > Spr...@li... > https://lists.sourceforge.net/lists/listinfo/springframework-developer ------------------------------------------------------- This SF.Net email is sponsored by: Oracle 10g Get certified on the hottest thing ever to hit the market... Oracle 10g. Take an Oracle 10g class now, and we'll give you the exam FREE. http://ads.osdn.com/?ad_id=3D3149&alloc_id=3D8166&op=3Dclick _______________________________________________ Springframework-developer mailing list Spr...@li... https://lists.sourceforge.net/lists/listinfo/springframework-developer ------------------------------------------------------- This SF.Net email is sponsored by: Oracle 10g Get certified on the hottest thing ever to hit the market... Oracle 10g. Take an Oracle 10g class now, and we'll give you the exam FREE. http://ads.osdn.com/?ad_id149&alloc_id=8166&op=3Dick _______________________________________________ Springframework-developer mailing list Spr...@li... https://lists.sourceforge.net/lists/listinfo/springframework-developer ------------------------------------------------------- This SF.Net email is sponsored by: Oracle 10g Get certified on the hottest thing ever to hit the market... Oracle 10g. Take an Oracle 10g class now, and we'll give you the exam FREE. http://ads.osdn.com/?ad_id149&alloc_id=8166&op=3Dick _______________________________________________ Springframework-developer mailing list Spr...@li... https://lists.sourceforge.net/lists/listinfo/springframework-developer ------------------------------------------------------- This SF.Net email is sponsored by: Oracle 10g Get certified on the hottest thing ever to hit the market... Oracle 10g. Take an Oracle 10g class now, and we'll give you the exam FREE. http://ads.osdn.com/?ad_id=3D3149&alloc_id=3D8166&op=3Dclick _______________________________________________ Springframework-developer mailing list Spr...@li... https://lists.sourceforge.net/lists/listinfo/springframework-developer ------------------------------------------------------- This SF.Net email is sponsored by: Oracle 10g Get certified on the hottest thing ever to hit the market... Oracle 10g. Take an Oracle 10g class now, and we'll give you the exam FREE. http://ads.osdn.com/?ad_id149&alloc_id=8166&op=3Dick _______________________________________________ Springframework-developer mailing list Spr...@li... https://lists.sourceforge.net/lists/listinfo/springframework-developer ------------------------------------------------------- This SF.Net email is sponsored by: Oracle 10g Get certified on the hottest thing ever to hit the market... Oracle 10g. Take an Oracle 10g class now, and we'll give you the exam FREE. http://ads.osdn.com/?ad_id149&alloc_id=8166&op=3Dick _______________________________________________ Springframework-developer mailing list Spr...@li... https://lists.sourceforge.net/lists/listinfo/springframework-developer |
|
From: Guillaume P. <gpo...@gl...> - 2004-05-24 12:48:22
Attachments:
classloader.tar.gz
|
Well, I find it really weird, because I've tested such behavior before on Resin 2.1.10, and while it took a while before the resources were garbage collected, they were always eventually collected. And if I added a System.gc() when the ServletContext was loaded, all my singletons were being collected right away. During all your tests, did you notice higher memory consumption or did you actually get a OutOfMemoryError ? May be it's the profiler that keeps a reference on some resource(s) for some reasons? But if it is really a ClassLoader issue, shouldn't you be able to simulate the situation outside a Servlet Container? I've run the attached test, and while I create a memory leak quickly if I keep a reference on the ClassLoader, I am not able to produce one without holding a ref on it. Or is my test flawed? Guillaume ----- Original Message ----- From: "jürgen höller [werk3AT]" <jue...@we...> To: <spr...@li...> Sent: Monday, May 24, 2004 4:41 AM Subject: Re: [Springframework-developer] Cleanup of context resources on webapp reload Just finished tests on Resin 2.1.11 - exactly same behavior as with Tomcat. This *is* a general class loader respectively garbage collection issue, rather than a server-specific leak. Consequently, with the changes I've committed yesterday, there is the same significant benefit as with Tomcat. The only remaining issue are the "full object" constants like ClassFilter.TRUE. Juergen ________________________________ Von: spr...@li... im Auftrag von jürgen höller [werk3AT] Gesendet: Mo 24.05.2004 09:20 An: spr...@li... Betreff: Re: [Springframework-developer] Cleanup of context resources on webapp reload FYI, I've tested both Tomcat 5.0.18 and 4.1.27 - same behavior with both. BTW, a couple of related posts from the Resin mailing list: http://www.caucho.com/support/resin-interest/0404/0117.html http://www.caucho.com/support/resin-interest/0111/0164.html <http://www.caucho.com/support/resin-interest/0111/0164.html> Juergen ________________________________ Von: spr...@li... im Auftrag von jürgen höller [werk3AT] Gesendet: Mo 24.05.2004 08:42 An: spr...@li... Betreff: Re: [Springframework-developer] Cleanup of context resources on webapp reload I've told JProfiler to explicitly run garbage collection - a number of times, actually - before I've had a look at the heap. So I'm sure that those remaining objects were not garbage-collected, and probably would have stayed around in the VM forever... I'm aware that it seems odd, but this issue just seems to affect specific static fields: BeanWrapperImpl's defaultEditors did not cause a leak, but CachedIntrospectionResults' classCache did. Normal constants or static logger fields didn't, but "full object" constants like ClassFilters.TRUE did. CachedIntrospectionResults' classCache uses the Class as key; the value, a CachedIntrospectionResult object, also refers to the key Class. Consequently, I had to use a WeakHashMap *with WeakReferences as values* to see proper garbage collection. Note that this static cache contains instances of its containing class as values. I've run all my tests x times to make sure that I could trust my eyes. I've even undone the WeakReference changes again, and voila, there were the leaks again. I'd be happy to learn more about how the garbage collector works here... All I can state at this point of time is that the changes did cause an obvious difference in terms of resource leaks. Juergen ________________________________ Von: spr...@li... im Auftrag von Guillaume Poirier Gesendet: Mo 24.05.2004 05:17 An: spr...@li... Betreff: Re: [Springframework-developer] Cleanup of context resources on webapp reload The "resource leak" caused by a singleton when a webapp's classloader is thrown away is only temorary, the unused classes and the classloader will be eventually garbage collected and the resources will be freed. The only thing that could prevent that is if there was a something in the server's classloader that still had a reference on an object or a class of the child classloader. Unless there's a bug in Tomcat or in the application code, I really can't see how the classes won't be eventually garbage collected when the JVM needs memory. And anyway, if there was indeed a leak because SQLErrorCodesFactory is a singleton, why wouldn't there be one for each static fields such as constants? Are you sure that JProfiler does not disable garbage collecting in order to make its profiling? I know that in many of the JVMPI method calls are done with garbage collecting off. I suspect the above to be the cause of the "resource leak", rather than any singleton that Spring might be using. Guillaume ----- Original Message ----- From: "jürgen höller [werk3AT]" <jue...@we...> To: <spr...@li...> Sent: Sunday, May 23, 2004 4:45 PM Subject: Re: [Springframework-developer] Cleanup of context resources on webapp reload I've just spent about 10 hours profiling Spring, using the Image Database and Petclinic samples. (BTW, I've used an evaluation version of JProfiler from ej-technologies - nice product!) Although I still don't completely understand the garbage collection behavior, I've figured out the following issues. Each of them simply prevents the respective classes from getting garbage collected on destruction of the class loader (e.g. on Tomcat web app shutdown). - A classic singleton with a class variable holding the object. I've reworked GlobalAdvisorAdapterRegistry and SQLErrorCodesFactory to hold the respective singleton as a WeakReference. - A static cache. I've reworked CachedIntrospectionResults to use a WeakHashMap with WeakReferences as values. - A ThreadLocal with a default other than null. I've reworked TransactionSynchronizationManager to use null as default for the resource map, setting a HashMap there on demand, removing the entire HashMap when unbinding the last resource. - Constants that define a full object. We have a number of those, for example ClassFilters.TRUE and AdvisedSupport.EMPTY_TARGET_SOURCE. I've tried for quite a while, but I haven't been able to figure out a way to define such constants such that they will be garbage collected. The latter programming style is not uncommon, so I really don't understand why it causes trouble with garbage collection. Hibernate uses a similar style for its FlushMode, for example. In general, other frameworks like CGLIB, Hibernate, Velocity have huge resource leaks on web app shutdown, while just the constants issue remains with Spring now. As long as those huge third-party leaks are not addressed, I'm not worried at all by the single remaining Spring issue. As I initially said, we shouldn't exaggerate the problem, as it basically just affects hot reloading of web apps - mainly a development feature anyway. We need to make that clear to users too, to avoid comments a la "Spring is not usable for real apps because it leaks on hot redeployment". Please, everybody, give the current CVS head a sanity check tomorrow. There shouldn't be any issues: the test suite passes, the sample apps run properly. Still, I'd feel more comfortable if we make sure that no subtle side effects have been introduced. For this reason, I will delay release 1.0.2 till tomorrow night. Juergen ________________________________ Von: spr...@li... im Auftrag von jürgen höller [werk3AT] Gesendet: Sa 22.05.2004 15:52 An: spr...@li... Betreff: Re: [Springframework-developer] Cleanup of context resources on webapp reload So basically, all static caches cause resource leaks when restarting a Tomcat web app? I wonder why this happens... The class loader should completely dissolve all classes that it has loaded in its lifetime, including static caches. Or have I misunderstood something here? Anyone having in-detail experience with handling such a scenario? Juergen ________________________________ Von: spr...@li... im Auftrag von Dmitriy Kopylenko Gesendet: Di 04.05.2004 18:02 An: spr...@li... Betreff: Re: [Springframework-developer] Cleanup of context resources on webapp reload Well, for instance SQLErrorCodesFactory is a singleton(GoF, not Spring) which caches SQLErrorCodes internally in the Map with strong references. Again, I don't know if trying to use WeakHashMap there would do the trick... Dmitriy Tim Kettering wrote: > > I looked at it some more this morning, and basically what I did was > start up tomcat w/ the webapp in the profiler, then after it was done > starting up I used tomcat's manager to stop the context. This should > destroy all resources related to the context. Here is a list of > spring related stuff that still were in memory after the context was > closed. Other stuff was cleaned up just fine. > > org.springframework.beans.CachedIntrospectionResults > org.springframework.jdbc.support.SQLCodes > org.springframework.core.Constants > org.springframework.aop.framework.AdvisedSupport$1 > org.springframework.aop.framework.adapter.BeforeAdviceAdapter > org.springframework.aop.framework.adapter.AfterReturningAdviceAdapter > org.springframework.jdbc.support.SQLErrorCodesFactory > org.springframework.transaction.support.TransactionSynchronizationManage > r$1 > org.springframework.transaction.interceptor.RollbackRuleAttribute > org.springframework.aop.framework.adapter.ThrosAdviceAdapter > org.springframework.aop.Pointcut$1 > org.springframework.aop.framework.adapter.GlobalAdvisorAdapterRegistry > > On May 3, 2004, at 3:28 PM, Dmitriy Kopylenko wrote: > >> I'm just wondering, would the use of WeakHashMap in >> CachedIntrospectionResults help? >> >> Dmitriy. >> >> Tim Kettering wrote: >> >>> I posted this to the users list last week and did not receive any >>> reply on it, so I'm posting it again here on the developer list, in >>> hopes i could get an reply from someone here. I'm trying to >>> determine if its something I should be doing myself, or if hte >>> spring context should be cleaning up those resources by itself on >>> the .close() call. Further profiling shows that there are >>> duplicate instances of SQLError and hibernate proxy classes hanging >>> around afterwards too. Other objects do get cleaned up properly. >>> -------- >>> Hi everyone, >>> We're (meaning me) looking into some resource leaks that are >>> occuring when our webapp context gets reloaded. I found that >>> context.close() needs to be called on the destroy() method of >>> plugin we're using, and it works for a good majority of the objects >>> we were seeing leaked, but there are some objects that I'm unable >>> to make go away. Object in question is the: >>> org.springframework.beans.CachedIntrospectionResults >>> Whenever I reload the context - the profiler I'm using shows that I >>> have essentially a duplicate group of those objects (same instance >>> count) as the original, and successive reloads will continue to >>> duplicate this. >>> The profiler also shows the final reference to those objects like this: >>> 100% - 1008 bytes - 63 alloc. >>> org.springframework.context.support.ClassPathXmlApplicationContext.<in >>> it > >>> So basically I guess what I'm asking is for ideas or suggestions on >>> how I could get those to clean up. This bean doesnt show up in the >>> Spring javadocs. And looking in CVS says its a package level bean, >>> not for application use, so I'm thinking that closing the context >>> should (in theory) clean this up? Thanks in advance. >>> -tim >>> ------------------------------------------------------- >>> This SF.Net email is sponsored by: Oracle 10g >>> Get certified on the hottest thing ever to hit the market... Oracle >>> 10g. Take an Oracle 10g class now, and we'll give you the exam FREE. >>> http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click >>> _______________________________________________ >>> Springframework-developer mailing list >>> Spr...@li... >>> https://lists.sourceforge.net/lists/listinfo/springframework-developer >> >> >> >> >> ------------------------------------------------------- >> This SF.Net email is sponsored by: Oracle 10g >> Get certified on the hottest thing ever to hit the market... Oracle >> 10g. Take an Oracle 10g class now, and we'll give you the exam FREE. >> http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click >> _______________________________________________ >> Springframework-developer mailing list >> Spr...@li... >> https://lists.sourceforge.net/lists/listinfo/springframework-developer >> > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: Oracle 10g > Get certified on the hottest thing ever to hit the market... Oracle 10g. > Take an Oracle 10g class now, and we'll give you the exam FREE. > http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click > _______________________________________________ > Springframework-developer mailing list > Spr...@li... > https://lists.sourceforge.net/lists/listinfo/springframework-developer ------------------------------------------------------- This SF.Net email is sponsored by: Oracle 10g Get certified on the hottest thing ever to hit the market... Oracle 10g. Take an Oracle 10g class now, and we'll give you the exam FREE. http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click _______________________________________________ Springframework-developer mailing list Spr...@li... https://lists.sourceforge.net/lists/listinfo/springframework-developer ------------------------------------------------------- This SF.Net email is sponsored by: Oracle 10g Get certified on the hottest thing ever to hit the market... Oracle 10g. Take an Oracle 10g class now, and we'll give you the exam FREE. http://ads.osdn.com/?ad_id149&alloc_id66&op=ick _______________________________________________ Springframework-developer mailing list Spr...@li... https://lists.sourceforge.net/lists/listinfo/springframework-developer ------------------------------------------------------- This SF.Net email is sponsored by: Oracle 10g Get certified on the hottest thing ever to hit the market... Oracle 10g. Take an Oracle 10g class now, and we'll give you the exam FREE. http://ads.osdn.com/?ad_id149&alloc_id66&op=ick _______________________________________________ Springframework-developer mailing list Spr...@li... https://lists.sourceforge.net/lists/listinfo/springframework-developer ------------------------------------------------------- This SF.Net email is sponsored by: Oracle 10g Get certified on the hottest thing ever to hit the market... Oracle 10g. Take an Oracle 10g class now, and we'll give you the exam FREE. http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click _______________________________________________ Springframework-developer mailing list Spr...@li... https://lists.sourceforge.net/lists/listinfo/springframework-developer ------------------------------------------------------- This SF.Net email is sponsored by: Oracle 10g Get certified on the hottest thing ever to hit the market... Oracle 10g. Take an Oracle 10g class now, and we'll give you the exam FREE. http://ads.osdn.com/?ad_id149&alloc_id66&op=ick _______________________________________________ Springframework-developer mailing list Spr...@li... https://lists.sourceforge.net/lists/listinfo/springframework-developer ------------------------------------------------------- This SF.Net email is sponsored by: Oracle 10g Get certified on the hottest thing ever to hit the market... Oracle 10g. Take an Oracle 10g class now, and we'll give you the exam FREE. http://ads.osdn.com/?ad_id149&alloc_id66&op=ick _______________________________________________ Springframework-developer mailing list Spr...@li... https://lists.sourceforge.net/lists/listinfo/springframework-developer ------------------------------------------------------- This SF.Net email is sponsored by: Oracle 10g Get certified on the hottest thing ever to hit the market... Oracle 10g. Take an Oracle 10g class now, and we'll give you the exam FREE. http://ads.osdn.com/?ad_id149&alloc_id66&op=ick _______________________________________________ Springframework-developer mailing list Spr...@li... https://lists.sourceforge.net/lists/listinfo/springframework-developer |
|
From: Tim K. <tim...@vi...> - 2004-05-24 14:47:24
|
Jurgen, Thanks for looking into this issue. When I originally reported on it, =20= and didnt hear much discussion on it, I had started to think that i was =20= the only person experiencing the memory leak and that I was coding =20 something wrong. Your experiences w/ JProfiler are very similar to what I was seeing =20 when I was running JProfiler against our code, although you've =20 obviously delved deeper into the exact reasons for those leaks. I will =20= give the latest build of spring a try on our code and hopefully it =20 should help alleviate the situation some. I also wanted to add a comment in respect to what Guillaume said below =20= that we've consistently seen OOM errors coming from Tomcat after X =20 amount of reloads done by the test server. And this can happen over =20 the span of several days, so I do not believe that those resources are =20= going to be GC'ed at all, unless the definition of "took a while" means =20= a week or so. :) Resin may be doing something different however. -tim On May 24, 2004, at 8:48 AM, Guillaume Poirier wrote: > Well, I find it really weird, because I've tested such behavior before = =20 > on > Resin 2.1.10, and while it took a while before the resources were =20 > garbage > collected, they were always eventually collected. And if I added a > System.gc() when the ServletContext was loaded, all my singletons were = =20 > being > collected right away. During all your tests, did you notice higher =20= > memory > consumption or did you actually get a OutOfMemoryError ? May be it's =20= > the > profiler that keeps a reference on some resource(s) for some reasons? > > But if it is really a ClassLoader issue, shouldn't you be able to =20 > simulate > the situation outside a Servlet Container? I've run the attached =20 > test, and > while I create a memory leak quickly if I keep a reference on the > ClassLoader, I am not able to produce one without holding a ref on it. > > Or is my test flawed? > > Guillaume > > ----- Original Message ----- > From: "j=FCrgen h=F6ller [werk3AT]" <jue...@we...> > To: <spr...@li...> > Sent: Monday, May 24, 2004 4:41 AM > Subject: Re: [Springframework-developer] Cleanup of context resources =20= > on > webapp reload > > > Just finished tests on Resin 2.1.11 - exactly same behavior as with =20= > Tomcat. > This *is* a general class loader respectively garbage collection = issue, > rather than a server-specific leak. > > Consequently, with the changes I've committed yesterday, there is the =20= > same > significant benefit as with Tomcat. The only remaining issue are the =20= > "full > object" constants like ClassFilter.TRUE. > > Juergen > > > ________________________________ > > Von: spr...@li... im Auftrag =20= > von > j=FCrgen h=F6ller [werk3AT] > Gesendet: Mo 24.05.2004 09:20 > An: spr...@li... > Betreff: Re: [Springframework-developer] Cleanup of context resources =20= > on > webapp reload > > > > FYI, I've tested both Tomcat 5.0.18 and 4.1.27 - same behavior with =20= > both. > > BTW, a couple of related posts from the Resin mailing list: > > http://www.caucho.com/support/resin-interest/0404/0117.html > http://www.caucho.com/support/resin-interest/0111/0164.html > <http://www.caucho.com/support/resin-interest/0111/0164.html> > > Juergen > > > ________________________________ > > Von: spr...@li... im Auftrag =20= > von > j=FCrgen h=F6ller [werk3AT] > Gesendet: Mo 24.05.2004 08:42 > An: spr...@li... > Betreff: Re: [Springframework-developer] Cleanup of context resources =20= > on > webapp reload > > > > I've told JProfiler to explicitly run garbage collection - a number of > times, actually - before I've had a look at the heap. So I'm sure that = =20 > those > remaining objects were not garbage-collected, and probably would have =20= > stayed > around in the VM forever... > > I'm aware that it seems odd, but this issue just seems to affect =20 > specific > static fields: BeanWrapperImpl's defaultEditors did not cause a leak, =20= > but > CachedIntrospectionResults' classCache did. Normal constants or static > logger fields didn't, but "full object" constants like =20 > ClassFilters.TRUE > did. > > CachedIntrospectionResults' classCache uses the Class as key; the =20 > value, a > CachedIntrospectionResult object, also refers to the key Class. > Consequently, I had to use a WeakHashMap *with WeakReferences as =20 > values* to > see proper garbage collection. Note that this static cache contains > instances of its containing class as values. > > I've run all my tests x times to make sure that I could trust my eyes. = =20 > I've > even undone the WeakReference changes again, and voila, there were the = =20 > leaks > again. I'd be happy to learn more about how the garbage collector = works > here... All I can state at this point of time is that the changes did =20= > cause > an obvious difference in terms of resource leaks. > > Juergen > > > ________________________________ > > Von: spr...@li... im Auftrag =20= > von > Guillaume Poirier > Gesendet: Mo 24.05.2004 05:17 > An: spr...@li... > Betreff: Re: [Springframework-developer] Cleanup of context resources =20= > on > webapp reload > > > > The "resource leak" caused by a singleton when a webapp's classloader =20= > is > thrown away is only temorary, the unused classes and the classloader =20= > will be > eventually garbage collected and the resources will be freed. The = only > thing that could prevent that is if there was a something in the =20 > server's > classloader that still had a reference on an object or a class of the =20= > child > classloader. > > Unless there's a bug in Tomcat or in the application code, I really =20= > can't > see how the classes won't be eventually garbage collected when the JVM = =20 > needs > memory. And anyway, if there was indeed a leak because =20 > SQLErrorCodesFactory > is a singleton, why wouldn't there be one for each static fields such =20= > as > constants? > > Are you sure that JProfiler does not disable garbage collecting in =20 > order to > make its profiling? I know that in many of the JVMPI method calls are = =20 > done > with garbage collecting off. I suspect the above to be the cause of =20= > the > "resource leak", rather than any singleton that Spring might be using. > > Guillaume > > ----- Original Message ----- > From: "j=FCrgen h=F6ller [werk3AT]" <jue...@we...> > To: <spr...@li...> > Sent: Sunday, May 23, 2004 4:45 PM > Subject: Re: [Springframework-developer] Cleanup of context resources =20= > on > webapp reload > > > I've just spent about 10 hours profiling Spring, using the Image =20 > Database > and Petclinic samples. (BTW, I've used an evaluation version of =20 > JProfiler > from ej-technologies - nice product!) > > Although I still don't completely understand the garbage collection > behavior, I've figured out the following issues. Each of them simply > prevents the respective classes from getting garbage collected on > destruction of the class loader (e.g. on Tomcat web app shutdown). > > - A classic singleton with a class variable holding the object. I've > reworked GlobalAdvisorAdapterRegistry and SQLErrorCodesFactory to hold = =20 > the > respective singleton as a WeakReference. > > - A static cache. I've reworked CachedIntrospectionResults to use a > WeakHashMap with WeakReferences as values. > > - A ThreadLocal with a default other than null. I've reworked > TransactionSynchronizationManager to use null as default for the =20 > resource > map, setting a HashMap there on demand, removing the entire HashMap =20= > when > unbinding the last resource. > > - Constants that define a full object. We have a number of those, for > example ClassFilters.TRUE and AdvisedSupport.EMPTY_TARGET_SOURCE. I've = =20 > tried > for quite a while, but I haven't been able to figure out a way to =20 > define > such constants such that they will be garbage collected. > > The latter programming style is not uncommon, so I really don't =20 > understand > why it causes trouble with garbage collection. Hibernate uses a = similar > style for its FlushMode, for example. > > In general, other frameworks like CGLIB, Hibernate, Velocity have huge > resource leaks on web app shutdown, while just the constants issue =20 > remains > with Spring now. As long as those huge third-party leaks are not =20 > addressed, > I'm not worried at all by the single remaining Spring issue. > > As I initially said, we shouldn't exaggerate the problem, as it =20 > basically > just affects hot reloading of web apps - mainly a development feature > anyway. We need to make that clear to users too, to avoid comments a = la > "Spring is not usable for real apps because it leaks on hot =20 > redeployment". > > Please, everybody, give the current CVS head a sanity check tomorrow. =20= > There > shouldn't be any issues: the test suite passes, the sample apps run > properly. Still, I'd feel more comfortable if we make sure that no =20 > subtle > side effects have been introduced. > > For this reason, I will delay release 1.0.2 till tomorrow night. > > Juergen > > > ________________________________ > > Von: spr...@li... im Auftrag =20= > von > j=FCrgen h=F6ller [werk3AT] > Gesendet: Sa 22.05.2004 15:52 > An: spr...@li... > Betreff: Re: [Springframework-developer] Cleanup of context resources =20= > on > webapp reload > > > > So basically, all static caches cause resource leaks when restarting a > Tomcat web app? I wonder why this happens... The class loader should > completely dissolve all classes that it has loaded in its lifetime, > including static caches. Or have I misunderstood something here? = Anyone > having in-detail experience with handling such a scenario? > > Juergen > > > ________________________________ > > Von: spr...@li... im Auftrag =20= > von > Dmitriy Kopylenko > Gesendet: Di 04.05.2004 18:02 > An: spr...@li... > Betreff: Re: [Springframework-developer] Cleanup of context resources =20= > on > webapp reload > > > > Well, > > for instance SQLErrorCodesFactory is a singleton(GoF, not Spring) = which > caches SQLErrorCodes internally in the Map with strong references. =20 > Again, I > don't know if trying > to use WeakHashMap there would do the trick... > > Dmitriy > > Tim Kettering wrote: > >> >> I looked at it some more this morning, and basically what I did was >> start up tomcat w/ the webapp in the profiler, then after it was done >> starting up I used tomcat's manager to stop the context. This should >> destroy all resources related to the context. Here is a list of >> spring related stuff that still were in memory after the context was >> closed. Other stuff was cleaned up just fine. >> >> org.springframework.beans.CachedIntrospectionResults >> org.springframework.jdbc.support.SQLCodes >> org.springframework.core.Constants >> org.springframework.aop.framework.AdvisedSupport$1 >> org.springframework.aop.framework.adapter.BeforeAdviceAdapter >> org.springframework.aop.framework.adapter.AfterReturningAdviceAdapter >> org.springframework.jdbc.support.SQLErrorCodesFactory >> = org.springframework.transaction.support.TransactionSynchronizationMana=20= >> ge >> r$1 >> org.springframework.transaction.interceptor.RollbackRuleAttribute >> org.springframework.aop.framework.adapter.ThrosAdviceAdapter >> org.springframework.aop.Pointcut$1 >> = org.springframework.aop.framework.adapter.GlobalAdvisorAdapterRegistry >> >> On May 3, 2004, at 3:28 PM, Dmitriy Kopylenko wrote: >> >>> I'm just wondering, would the use of WeakHashMap in >>> CachedIntrospectionResults help? >>> >>> Dmitriy. >>> >>> Tim Kettering wrote: >>> >>>> I posted this to the users list last week and did not receive any >>>> reply on it, so I'm posting it again here on the developer list, = in >>>> hopes i could get an reply from someone here. I'm trying to >>>> determine if its something I should be doing myself, or if hte >>>> spring context should be cleaning up those resources by itself on >>>> the .close() call. Further profiling shows that there are >>>> duplicate instances of SQLError and hibernate proxy classes =20 >>>> hanging >>>> around afterwards too. Other objects do get cleaned up properly. >>>> -------- >>>> Hi everyone, >>>> We're (meaning me) looking into some resource leaks that are >>>> occuring when our webapp context gets reloaded. I found that >>>> context.close() needs to be called on the destroy() method of >>>> plugin we're using, and it works for a good majority of the =20 >>>> objects >>>> we were seeing leaked, but there are some objects that I'm unable >>>> to make go away. Object in question is the: >>>> org.springframework.beans.CachedIntrospectionResults >>>> Whenever I reload the context - the profiler I'm using shows that I >>>> have essentially a duplicate group of those objects (same instance >>>> count) as the original, and successive reloads will continue to >>>> duplicate this. >>>> The profiler also shows the final reference to those objects like =20= >>>> this: >>>> 100% - 1008 bytes - 63 alloc. >>>> = org.springframework.context.support.ClassPathXmlApplicationContext.<=20 >>>> in >>>> it > >>>> So basically I guess what I'm asking is for ideas or suggestions on >>>> how I could get those to clean up. This bean doesnt show up in = the >>>> Spring javadocs. And looking in CVS says its a package level = bean, >>>> not for application use, so I'm thinking that closing the context >>>> should (in theory) clean this up? Thanks in advance. >>>> -tim >>>> ------------------------------------------------------- >>>> This SF.Net email is sponsored by: Oracle 10g >>>> Get certified on the hottest thing ever to hit the market... Oracle >>>> 10g. Take an Oracle 10g class now, and we'll give you the exam = FREE. >>>> http://ads.osdn.com/?ad_id=3D3149&alloc_id=3D8166&op=3Dclick >>>> _______________________________________________ >>>> Springframework-developer mailing list >>>> Spr...@li... >>>> https://lists.sourceforge.net/lists/listinfo/springframework-=20 >>>> developer >>> >>> >>> >>> >>> ------------------------------------------------------- >>> This SF.Net email is sponsored by: Oracle 10g >>> Get certified on the hottest thing ever to hit the market... Oracle >>> 10g. Take an Oracle 10g class now, and we'll give you the exam FREE. >>> http://ads.osdn.com/?ad_id=3D3149&alloc_id=3D8166&op=3Dclick >>> _______________________________________________ >>> Springframework-developer mailing list >>> Spr...@li... >>> https://lists.sourceforge.net/lists/listinfo/springframework-=20 >>> developer >>> >> >> >> >> ------------------------------------------------------- >> This SF.Net email is sponsored by: Oracle 10g >> Get certified on the hottest thing ever to hit the market... Oracle =20= >> 10g. >> Take an Oracle 10g class now, and we'll give you the exam FREE. >> http://ads.osdn.com/?ad_id=3D3149&alloc_id=3D8166&op=3Dclick >> _______________________________________________ >> Springframework-developer mailing list >> Spr...@li... >> = https://lists.sourceforge.net/lists/listinfo/springframework-developer > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: Oracle 10g > Get certified on the hottest thing ever to hit the market... Oracle =20= > 10g. > Take an Oracle 10g class now, and we'll give you the exam FREE. > http://ads.osdn.com/?ad_id=3D3149&alloc_id=3D8166&op=3Dclick > _______________________________________________ > Springframework-developer mailing list > Spr...@li... > https://lists.sourceforge.net/lists/listinfo/springframework-developer > > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: Oracle 10g > Get certified on the hottest thing ever to hit the market... Oracle =20= > 10g. > Take an Oracle 10g class now, and we'll give you the exam FREE. > http://ads.osdn.com/?ad_id149&alloc_id=9966&op=3Dick > _______________________________________________ > Springframework-developer mailing list > Spr...@li... > https://lists.sourceforge.net/lists/listinfo/springframework-developer > > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: Oracle 10g > Get certified on the hottest thing ever to hit the market... Oracle =20= > 10g. > Take an Oracle 10g class now, and we'll give you the exam FREE. > http://ads.osdn.com/?ad_id149&alloc_id=9966&op=3Dick > _______________________________________________ > Springframework-developer mailing list > Spr...@li... > https://lists.sourceforge.net/lists/listinfo/springframework-developer > > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: Oracle 10g > Get certified on the hottest thing ever to hit the market... Oracle =20= > 10g. > Take an Oracle 10g class now, and we'll give you the exam FREE. > http://ads.osdn.com/?ad_id=3D3149&alloc_id=3D8166&op=3Dclick > _______________________________________________ > Springframework-developer mailing list > Spr...@li... > https://lists.sourceforge.net/lists/listinfo/springframework-developer > > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: Oracle 10g > Get certified on the hottest thing ever to hit the market... Oracle =20= > 10g. > Take an Oracle 10g class now, and we'll give you the exam FREE. > http://ads.osdn.com/?ad_id149&alloc_id=9966&op=3Dick > _______________________________________________ > Springframework-developer mailing list > Spr...@li... > https://lists.sourceforge.net/lists/listinfo/springframework-developer > > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: Oracle 10g > Get certified on the hottest thing ever to hit the market... Oracle =20= > 10g. > Take an Oracle 10g class now, and we'll give you the exam FREE. > http://ads.osdn.com/?ad_id149&alloc_id=9966&op=3Dick > _______________________________________________ > Springframework-developer mailing list > Spr...@li... > https://lists.sourceforge.net/lists/listinfo/springframework-developer > > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: Oracle 10g > Get certified on the hottest thing ever to hit the market... Oracle =20= > 10g. > Take an Oracle 10g class now, and we'll give you the exam FREE. > http://ads.osdn.com/?ad_id149&alloc_id=9966&op=3Dick > _______________________________________________ > Springframework-developer mailing list > Spr...@li... > https://lists.sourceforge.net/lists/listinfo/springframework-developer > <classloader.tar.gz>= |
|
From: Dmitriy K. <dko...@ru...> - 2004-05-24 13:12:59
|
Juergen, I'm just wondering - did you try to play with different GC algorithms (through command line switches in the container's bootstrap script, I guess)while doing the profiling, i.e. -XX:+UseParNewGC, -XX:+UseParallelGC, -Xincgc, -XX:+UseConMarkSweepGC ? Dmitriy. jürgen höller [werk3AT] wrote: > Just finished tests on Resin 2.1.11 - exactly same behavior as with Tomcat. This *is* a general class loader respectively garbage collection issue, rather than a server-specific leak. > > Consequently, with the changes I've committed yesterday, there is the same significant benefit as with Tomcat. The only remaining issue are the "full object" constants like ClassFilter.TRUE. > > Juergen > > > ________________________________ > > Von: spr...@li... im Auftrag von jürgen höller [werk3AT] > Gesendet: Mo 24.05.2004 09:20 > An: spr...@li... > Betreff: Re: [Springframework-developer] Cleanup of context resources on webapp reload > > > > FYI, I've tested both Tomcat 5.0.18 and 4.1.27 - same behavior with both. > > BTW, a couple of related posts from the Resin mailing list: > > http://www.caucho.com/support/resin-interest/0404/0117.html > http://www.caucho.com/support/resin-interest/0111/0164.html <http://www.caucho.com/support/resin-interest/0111/0164.html> > > Juergen > > > ________________________________ > > Von: spr...@li... im Auftrag von jürgen höller [werk3AT] > Gesendet: Mo 24.05.2004 08:42 > An: spr...@li... > Betreff: Re: [Springframework-developer] Cleanup of context resources on webapp reload > > > > I've told JProfiler to explicitly run garbage collection - a number of times, actually - before I've had a look at the heap. So I'm sure that those remaining objects were not garbage-collected, and probably would have stayed around in the VM forever... > > I'm aware that it seems odd, but this issue just seems to affect specific static fields: BeanWrapperImpl's defaultEditors did not cause a leak, but CachedIntrospectionResults' classCache did. Normal constants or static logger fields didn't, but "full object" constants like ClassFilters.TRUE did. > > CachedIntrospectionResults' classCache uses the Class as key; the value, a CachedIntrospectionResult object, also refers to the key Class. Consequently, I had to use a WeakHashMap *with WeakReferences as values* to see proper garbage collection. Note that this static cache contains instances of its containing class as values. > > I've run all my tests x times to make sure that I could trust my eyes. I've even undone the WeakReference changes again, and voila, there were the leaks again. I'd be happy to learn more about how the garbage collector works here... All I can state at this point of time is that the changes did cause an obvious difference in terms of resource leaks. > > Juergen > > > ________________________________ > > Von: spr...@li... im Auftrag von Guillaume Poirier > Gesendet: Mo 24.05.2004 05:17 > An: spr...@li... > Betreff: Re: [Springframework-developer] Cleanup of context resources on webapp reload > > > > The "resource leak" caused by a singleton when a webapp's classloader is > thrown away is only temorary, the unused classes and the classloader will be > eventually garbage collected and the resources will be freed. The only > thing that could prevent that is if there was a something in the server's > classloader that still had a reference on an object or a class of the child > classloader. > > Unless there's a bug in Tomcat or in the application code, I really can't > see how the classes won't be eventually garbage collected when the JVM needs > memory. And anyway, if there was indeed a leak because SQLErrorCodesFactory > is a singleton, why wouldn't there be one for each static fields such as > constants? > > Are you sure that JProfiler does not disable garbage collecting in order to > make its profiling? I know that in many of the JVMPI method calls are done > with garbage collecting off. I suspect the above to be the cause of the > "resource leak", rather than any singleton that Spring might be using. > > Guillaume > > ----- Original Message ----- > From: "jürgen höller [werk3AT]" <jue...@we...> > To: <spr...@li...> > Sent: Sunday, May 23, 2004 4:45 PM > Subject: Re: [Springframework-developer] Cleanup of context resources on > webapp reload > > > I've just spent about 10 hours profiling Spring, using the Image Database > and Petclinic samples. (BTW, I've used an evaluation version of JProfiler > from ej-technologies - nice product!) > > Although I still don't completely understand the garbage collection > behavior, I've figured out the following issues. Each of them simply > prevents the respective classes from getting garbage collected on > destruction of the class loader (e.g. on Tomcat web app shutdown). > > - A classic singleton with a class variable holding the object. I've > reworked GlobalAdvisorAdapterRegistry and SQLErrorCodesFactory to hold the > respective singleton as a WeakReference. > > - A static cache. I've reworked CachedIntrospectionResults to use a > WeakHashMap with WeakReferences as values. > > - A ThreadLocal with a default other than null. I've reworked > TransactionSynchronizationManager to use null as default for the resource > map, setting a HashMap there on demand, removing the entire HashMap when > unbinding the last resource. > > - Constants that define a full object. We have a number of those, for > example ClassFilters.TRUE and AdvisedSupport.EMPTY_TARGET_SOURCE. I've tried > for quite a while, but I haven't been able to figure out a way to define > such constants such that they will be garbage collected. > > The latter programming style is not uncommon, so I really don't understand > why it causes trouble with garbage collection. Hibernate uses a similar > style for its FlushMode, for example. > > In general, other frameworks like CGLIB, Hibernate, Velocity have huge > resource leaks on web app shutdown, while just the constants issue remains > with Spring now. As long as those huge third-party leaks are not addressed, > I'm not worried at all by the single remaining Spring issue. > > As I initially said, we shouldn't exaggerate the problem, as it basically > just affects hot reloading of web apps - mainly a development feature > anyway. We need to make that clear to users too, to avoid comments a la > "Spring is not usable for real apps because it leaks on hot redeployment". > > Please, everybody, give the current CVS head a sanity check tomorrow. There > shouldn't be any issues: the test suite passes, the sample apps run > properly. Still, I'd feel more comfortable if we make sure that no subtle > side effects have been introduced. > > For this reason, I will delay release 1.0.2 till tomorrow night. > > Juergen > > > ________________________________ > > Von: spr...@li... im Auftrag von > jürgen höller [werk3AT] > Gesendet: Sa 22.05.2004 15:52 > An: spr...@li... > Betreff: Re: [Springframework-developer] Cleanup of context resources on > webapp reload > > > > So basically, all static caches cause resource leaks when restarting a > Tomcat web app? I wonder why this happens... The class loader should > completely dissolve all classes that it has loaded in its lifetime, > including static caches. Or have I misunderstood something here? Anyone > having in-detail experience with handling such a scenario? > > Juergen > > > ________________________________ > > Von: spr...@li... im Auftrag von > Dmitriy Kopylenko > Gesendet: Di 04.05.2004 18:02 > An: spr...@li... > Betreff: Re: [Springframework-developer] Cleanup of context resources on > webapp reload > > > > Well, > > for instance SQLErrorCodesFactory is a singleton(GoF, not Spring) which > caches SQLErrorCodes internally in the Map with strong references. Again, I > don't know if trying > to use WeakHashMap there would do the trick... > > Dmitriy > > Tim Kettering wrote: > > >>I looked at it some more this morning, and basically what I did was >>start up tomcat w/ the webapp in the profiler, then after it was done >>starting up I used tomcat's manager to stop the context. This should >>destroy all resources related to the context. Here is a list of >>spring related stuff that still were in memory after the context was >>closed. Other stuff was cleaned up just fine. >> >>org.springframework.beans.CachedIntrospectionResults >>org.springframework.jdbc.support.SQLCodes >>org.springframework.core.Constants >>org.springframework.aop.framework.AdvisedSupport$1 >>org.springframework.aop.framework.adapter.BeforeAdviceAdapter >>org.springframework.aop.framework.adapter.AfterReturningAdviceAdapter >>org.springframework.jdbc.support.SQLErrorCodesFactory >>org.springframework.transaction.support.TransactionSynchronizationManage >>r$1 >>org.springframework.transaction.interceptor.RollbackRuleAttribute >>org.springframework.aop.framework.adapter.ThrosAdviceAdapter >>org.springframework.aop.Pointcut$1 >>org.springframework.aop.framework.adapter.GlobalAdvisorAdapterRegistry >> >>On May 3, 2004, at 3:28 PM, Dmitriy Kopylenko wrote: >> >> >>>I'm just wondering, would the use of WeakHashMap in >>>CachedIntrospectionResults help? >>> >>>Dmitriy. >>> >>>Tim Kettering wrote: >>> >>> >>>>I posted this to the users list last week and did not receive any >>>>reply on it, so I'm posting it again here on the developer list, in >>>>hopes i could get an reply from someone here. I'm trying to >>>>determine if its something I should be doing myself, or if hte >>>>spring context should be cleaning up those resources by itself on >>>>the .close() call. Further profiling shows that there are >>>>duplicate instances of SQLError and hibernate proxy classes hanging >>>>around afterwards too. Other objects do get cleaned up properly. >>>>-------- >>>>Hi everyone, >>>>We're (meaning me) looking into some resource leaks that are >>>>occuring when our webapp context gets reloaded. I found that >>>>context.close() needs to be called on the destroy() method of >>>>plugin we're using, and it works for a good majority of the objects >>>>we were seeing leaked, but there are some objects that I'm unable >>>>to make go away. Object in question is the: >>>>org.springframework.beans.CachedIntrospectionResults >>>>Whenever I reload the context - the profiler I'm using shows that I >>>>have essentially a duplicate group of those objects (same instance >>>>count) as the original, and successive reloads will continue to >>>>duplicate this. >>>>The profiler also shows the final reference to those objects like this: >>>>100% - 1008 bytes - 63 alloc. >>>>org.springframework.context.support.ClassPathXmlApplicationContext.<in >>>>it > >>>>So basically I guess what I'm asking is for ideas or suggestions on >>>>how I could get those to clean up. This bean doesnt show up in the >>>>Spring javadocs. And looking in CVS says its a package level bean, >>>>not for application use, so I'm thinking that closing the context >>>>should (in theory) clean this up? Thanks in advance. >>>>-tim >>>>------------------------------------------------------- >>>>This SF.Net email is sponsored by: Oracle 10g >>>>Get certified on the hottest thing ever to hit the market... Oracle >>>>10g. Take an Oracle 10g class now, and we'll give you the exam FREE. >>>>http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click >>>>_______________________________________________ >>>>Springframework-developer mailing list >>>>Spr...@li... >>>>https://lists.sourceforge.net/lists/listinfo/springframework-developer >>> >>> >>> >>> >>>------------------------------------------------------- >>>This SF.Net email is sponsored by: Oracle 10g >>>Get certified on the hottest thing ever to hit the market... Oracle >>>10g. Take an Oracle 10g class now, and we'll give you the exam FREE. >>>http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click >>>_______________________________________________ >>>Springframework-developer mailing list >>>Spr...@li... >>>https://lists.sourceforge.net/lists/listinfo/springframework-developer >>> >> >> >> >>------------------------------------------------------- >>This SF.Net email is sponsored by: Oracle 10g >>Get certified on the hottest thing ever to hit the market... Oracle 10g. >>Take an Oracle 10g class now, and we'll give you the exam FREE. >>http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click >>_______________________________________________ >>Springframework-developer mailing list >>Spr...@li... >>https://lists.sourceforge.net/lists/listinfo/springframework-developer > > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: Oracle 10g > Get certified on the hottest thing ever to hit the market... Oracle 10g. > Take an Oracle 10g class now, and we'll give you the exam FREE. > http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click > _______________________________________________ > Springframework-developer mailing list > Spr...@li... > https://lists.sourceforge.net/lists/listinfo/springframework-developer > > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: Oracle 10g > Get certified on the hottest thing ever to hit the market... Oracle 10g. > Take an Oracle 10g class now, and we'll give you the exam FREE. > http://ads.osdn.com/?ad_id149&alloc_id66&op=ick > _______________________________________________ > Springframework-developer mailing list > Spr...@li... > https://lists.sourceforge.net/lists/listinfo/springframework-developer > > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: Oracle 10g > Get certified on the hottest thing ever to hit the market... Oracle 10g. > Take an Oracle 10g class now, and we'll give you the exam FREE. > http://ads.osdn.com/?ad_id149&alloc_id66&op=ick > _______________________________________________ > Springframework-developer mailing list > Spr...@li... > https://lists.sourceforge.net/lists/listinfo/springframework-developer > > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: Oracle 10g > Get certified on the hottest thing ever to hit the market... Oracle 10g. > Take an Oracle 10g class now, and we'll give you the exam FREE. > http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click > _______________________________________________ > Springframework-developer mailing list > Spr...@li... > https://lists.sourceforge.net/lists/listinfo/springframework-developer > > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: Oracle 10g > Get certified on the hottest thing ever to hit the market... Oracle 10g. > Take an Oracle 10g class now, and we'll give you the exam FREE. > http://ads.osdn.com/?ad_id149&alloc_id66&op=ick > _______________________________________________ > Springframework-developer mailing list > Spr...@li... > https://lists.sourceforge.net/lists/listinfo/springframework-developer > > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: Oracle 10g > Get certified on the hottest thing ever to hit the market... Oracle 10g. > Take an Oracle 10g class now, and we'll give you the exam FREE. > http://ads.osdn.com/?ad_id149&alloc_id66&op=ick > _______________________________________________ > Springframework-developer mailing list > Spr...@li... > https://lists.sourceforge.net/lists/listinfo/springframework-developer > > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: Oracle 10g > Get certified on the hottest thing ever to hit the market... Oracle 10g. > Take an Oracle 10g class now, and we'll give you the exam FREE. > http://ads.osdn.com/?ad_id149&alloc_id66&op=click > _______________________________________________ > Springframework-developer mailing list > Spr...@li... > https://lists.sourceforge.net/lists/listinfo/springframework-developer |
|
From: Dmitriy K. <dko...@ru...> - 2004-05-24 13:46:02
|
Here is a good read about Reference API: http://www.javaworld.com/javaworld/jw-01-2002/jw-0104-java101.html Dmitriy. jürgen höller [werk3AT] wrote: > Just finished tests on Resin 2.1.11 - exactly same behavior as with Tomcat. This *is* a general class loader respectively garbage collection issue, rather than a server-specific leak. > > Consequently, with the changes I've committed yesterday, there is the same significant benefit as with Tomcat. The only remaining issue are the "full object" constants like ClassFilter.TRUE. > > Juergen > > > ________________________________ > > Von: spr...@li... im Auftrag von jürgen höller [werk3AT] > Gesendet: Mo 24.05.2004 09:20 > An: spr...@li... > Betreff: Re: [Springframework-developer] Cleanup of context resources on webapp reload > > > > FYI, I've tested both Tomcat 5.0.18 and 4.1.27 - same behavior with both. > > BTW, a couple of related posts from the Resin mailing list: > > http://www.caucho.com/support/resin-interest/0404/0117.html > http://www.caucho.com/support/resin-interest/0111/0164.html <http://www.caucho.com/support/resin-interest/0111/0164.html> > > Juergen > > > ________________________________ > > Von: spr...@li... im Auftrag von jürgen höller [werk3AT] > Gesendet: Mo 24.05.2004 08:42 > An: spr...@li... > Betreff: Re: [Springframework-developer] Cleanup of context resources on webapp reload > > > > I've told JProfiler to explicitly run garbage collection - a number of times, actually - before I've had a look at the heap. So I'm sure that those remaining objects were not garbage-collected, and probably would have stayed around in the VM forever... > > I'm aware that it seems odd, but this issue just seems to affect specific static fields: BeanWrapperImpl's defaultEditors did not cause a leak, but CachedIntrospectionResults' classCache did. Normal constants or static logger fields didn't, but "full object" constants like ClassFilters.TRUE did. > > CachedIntrospectionResults' classCache uses the Class as key; the value, a CachedIntrospectionResult object, also refers to the key Class. Consequently, I had to use a WeakHashMap *with WeakReferences as values* to see proper garbage collection. Note that this static cache contains instances of its containing class as values. > > I've run all my tests x times to make sure that I could trust my eyes. I've even undone the WeakReference changes again, and voila, there were the leaks again. I'd be happy to learn more about how the garbage collector works here... All I can state at this point of time is that the changes did cause an obvious difference in terms of resource leaks. > > Juergen > > > ________________________________ > > Von: spr...@li... im Auftrag von Guillaume Poirier > Gesendet: Mo 24.05.2004 05:17 > An: spr...@li... > Betreff: Re: [Springframework-developer] Cleanup of context resources on webapp reload > > > > The "resource leak" caused by a singleton when a webapp's classloader is > thrown away is only temorary, the unused classes and the classloader will be > eventually garbage collected and the resources will be freed. The only > thing that could prevent that is if there was a something in the server's > classloader that still had a reference on an object or a class of the child > classloader. > > Unless there's a bug in Tomcat or in the application code, I really can't > see how the classes won't be eventually garbage collected when the JVM needs > memory. And anyway, if there was indeed a leak because SQLErrorCodesFactory > is a singleton, why wouldn't there be one for each static fields such as > constants? > > Are you sure that JProfiler does not disable garbage collecting in order to > make its profiling? I know that in many of the JVMPI method calls are done > with garbage collecting off. I suspect the above to be the cause of the > "resource leak", rather than any singleton that Spring might be using. > > Guillaume > > ----- Original Message ----- > From: "jürgen höller [werk3AT]" <jue...@we...> > To: <spr...@li...> > Sent: Sunday, May 23, 2004 4:45 PM > Subject: Re: [Springframework-developer] Cleanup of context resources on > webapp reload > > > I've just spent about 10 hours profiling Spring, using the Image Database > and Petclinic samples. (BTW, I've used an evaluation version of JProfiler > from ej-technologies - nice product!) > > Although I still don't completely understand the garbage collection > behavior, I've figured out the following issues. Each of them simply > prevents the respective classes from getting garbage collected on > destruction of the class loader (e.g. on Tomcat web app shutdown). > > - A classic singleton with a class variable holding the object. I've > reworked GlobalAdvisorAdapterRegistry and SQLErrorCodesFactory to hold the > respective singleton as a WeakReference. > > - A static cache. I've reworked CachedIntrospectionResults to use a > WeakHashMap with WeakReferences as values. > > - A ThreadLocal with a default other than null. I've reworked > TransactionSynchronizationManager to use null as default for the resource > map, setting a HashMap there on demand, removing the entire HashMap when > unbinding the last resource. > > - Constants that define a full object. We have a number of those, for > example ClassFilters.TRUE and AdvisedSupport.EMPTY_TARGET_SOURCE. I've tried > for quite a while, but I haven't been able to figure out a way to define > such constants such that they will be garbage collected. > > The latter programming style is not uncommon, so I really don't understand > why it causes trouble with garbage collection. Hibernate uses a similar > style for its FlushMode, for example. > > In general, other frameworks like CGLIB, Hibernate, Velocity have huge > resource leaks on web app shutdown, while just the constants issue remains > with Spring now. As long as those huge third-party leaks are not addressed, > I'm not worried at all by the single remaining Spring issue. > > As I initially said, we shouldn't exaggerate the problem, as it basically > just affects hot reloading of web apps - mainly a development feature > anyway. We need to make that clear to users too, to avoid comments a la > "Spring is not usable for real apps because it leaks on hot redeployment". > > Please, everybody, give the current CVS head a sanity check tomorrow. There > shouldn't be any issues: the test suite passes, the sample apps run > properly. Still, I'd feel more comfortable if we make sure that no subtle > side effects have been introduced. > > For this reason, I will delay release 1.0.2 till tomorrow night. > > Juergen > > > ________________________________ > > Von: spr...@li... im Auftrag von > jürgen höller [werk3AT] > Gesendet: Sa 22.05.2004 15:52 > An: spr...@li... > Betreff: Re: [Springframework-developer] Cleanup of context resources on > webapp reload > > > > So basically, all static caches cause resource leaks when restarting a > Tomcat web app? I wonder why this happens... The class loader should > completely dissolve all classes that it has loaded in its lifetime, > including static caches. Or have I misunderstood something here? Anyone > having in-detail experience with handling such a scenario? > > Juergen > > > ________________________________ > > Von: spr...@li... im Auftrag von > Dmitriy Kopylenko > Gesendet: Di 04.05.2004 18:02 > An: spr...@li... > Betreff: Re: [Springframework-developer] Cleanup of context resources on > webapp reload > > > > Well, > > for instance SQLErrorCodesFactory is a singleton(GoF, not Spring) which > caches SQLErrorCodes internally in the Map with strong references. Again, I > don't know if trying > to use WeakHashMap there would do the trick... > > Dmitriy > > Tim Kettering wrote: > > >>I looked at it some more this morning, and basically what I did was >>start up tomcat w/ the webapp in the profiler, then after it was done >>starting up I used tomcat's manager to stop the context. This should >>destroy all resources related to the context. Here is a list of >>spring related stuff that still were in memory after the context was >>closed. Other stuff was cleaned up just fine. >> >>org.springframework.beans.CachedIntrospectionResults >>org.springframework.jdbc.support.SQLCodes >>org.springframework.core.Constants >>org.springframework.aop.framework.AdvisedSupport$1 >>org.springframework.aop.framework.adapter.BeforeAdviceAdapter >>org.springframework.aop.framework.adapter.AfterReturningAdviceAdapter >>org.springframework.jdbc.support.SQLErrorCodesFactory >>org.springframework.transaction.support.TransactionSynchronizationManage >>r$1 >>org.springframework.transaction.interceptor.RollbackRuleAttribute >>org.springframework.aop.framework.adapter.ThrosAdviceAdapter >>org.springframework.aop.Pointcut$1 >>org.springframework.aop.framework.adapter.GlobalAdvisorAdapterRegistry >> >>On May 3, 2004, at 3:28 PM, Dmitriy Kopylenko wrote: >> >> >>>I'm just wondering, would the use of WeakHashMap in >>>CachedIntrospectionResults help? >>> >>>Dmitriy. >>> >>>Tim Kettering wrote: >>> >>> >>>>I posted this to the users list last week and did not receive any >>>>reply on it, so I'm posting it again here on the developer list, in >>>>hopes i could get an reply from someone here. I'm trying to >>>>determine if its something I should be doing myself, or if hte >>>>spring context should be cleaning up those resources by itself on >>>>the .close() call. Further profiling shows that there are >>>>duplicate instances of SQLError and hibernate proxy classes hanging >>>>around afterwards too. Other objects do get cleaned up properly. >>>>-------- >>>>Hi everyone, >>>>We're (meaning me) looking into some resource leaks that are >>>>occuring when our webapp context gets reloaded. I found that >>>>context.close() needs to be called on the destroy() method of >>>>plugin we're using, and it works for a good majority of the objects >>>>we were seeing leaked, but there are some objects that I'm unable >>>>to make go away. Object in question is the: >>>>org.springframework.beans.CachedIntrospectionResults >>>>Whenever I reload the context - the profiler I'm using shows that I >>>>have essentially a duplicate group of those objects (same instance >>>>count) as the original, and successive reloads will continue to >>>>duplicate this. >>>>The profiler also shows the final reference to those objects like this: >>>>100% - 1008 bytes - 63 alloc. >>>>org.springframework.context.support.ClassPathXmlApplicationContext.<in >>>>it > >>>>So basically I guess what I'm asking is for ideas or suggestions on >>>>how I could get those to clean up. This bean doesnt show up in the >>>>Spring javadocs. And looking in CVS says its a package level bean, >>>>not for application use, so I'm thinking that closing the context >>>>should (in theory) clean this up? Thanks in advance. >>>>-tim >>>>------------------------------------------------------- >>>>This SF.Net email is sponsored by: Oracle 10g >>>>Get certified on the hottest thing ever to hit the market... Oracle >>>>10g. Take an Oracle 10g class now, and we'll give you the exam FREE. >>>>http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click >>>>_______________________________________________ >>>>Springframework-developer mailing list >>>>Spr...@li... >>>>https://lists.sourceforge.net/lists/listinfo/springframework-developer >>> >>> >>> >>> >>>------------------------------------------------------- >>>This SF.Net email is sponsored by: Oracle 10g >>>Get certified on the hottest thing ever to hit the market... Oracle >>>10g. Take an Oracle 10g class now, and we'll give you the exam FREE. >>>http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click >>>_______________________________________________ >>>Springframework-developer mailing list >>>Spr...@li... >>>https://lists.sourceforge.net/lists/listinfo/springframework-developer >>> >> >> >> >>------------------------------------------------------- >>This SF.Net email is sponsored by: Oracle 10g >>Get certified on the hottest thing ever to hit the market... Oracle 10g. >>Take an Oracle 10g class now, and we'll give you the exam FREE. >>http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click >>_______________________________________________ >>Springframework-developer mailing list >>Spr...@li... >>https://lists.sourceforge.net/lists/listinfo/springframework-developer > > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: Oracle 10g > Get certified on the hottest thing ever to hit the market... Oracle 10g. > Take an Oracle 10g class now, and we'll give you the exam FREE. > http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click > _______________________________________________ > Springframework-developer mailing list > Spr...@li... > https://lists.sourceforge.net/lists/listinfo/springframework-developer > > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: Oracle 10g > Get certified on the hottest thing ever to hit the market... Oracle 10g. > Take an Oracle 10g class now, and we'll give you the exam FREE. > http://ads.osdn.com/?ad_id149&alloc_id66&op=ick > _______________________________________________ > Springframework-developer mailing list > Spr...@li... > https://lists.sourceforge.net/lists/listinfo/springframework-developer > > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: Oracle 10g > Get certified on the hottest thing ever to hit the market... Oracle 10g. > Take an Oracle 10g class now, and we'll give you the exam FREE. > http://ads.osdn.com/?ad_id149&alloc_id66&op=ick > _______________________________________________ > Springframework-developer mailing list > Spr...@li... > https://lists.sourceforge.net/lists/listinfo/springframework-developer > > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: Oracle 10g > Get certified on the hottest thing ever to hit the market... Oracle 10g. > Take an Oracle 10g class now, and we'll give you the exam FREE. > http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click > _______________________________________________ > Springframework-developer mailing list > Spr...@li... > https://lists.sourceforge.net/lists/listinfo/springframework-developer > > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: Oracle 10g > Get certified on the hottest thing ever to hit the market... Oracle 10g. > Take an Oracle 10g class now, and we'll give you the exam FREE. > http://ads.osdn.com/?ad_id149&alloc_id66&op=ick > _______________________________________________ > Springframework-developer mailing list > Spr...@li... > https://lists.sourceforge.net/lists/listinfo/springframework-developer > > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: Oracle 10g > Get certified on the hottest thing ever to hit the market... Oracle 10g. > Take an Oracle 10g class now, and we'll give you the exam FREE. > http://ads.osdn.com/?ad_id149&alloc_id66&op=ick > _______________________________________________ > Springframework-developer mailing list > Spr...@li... > https://lists.sourceforge.net/lists/listinfo/springframework-developer > > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: Oracle 10g > Get certified on the hottest thing ever to hit the market... Oracle 10g. > Take an Oracle 10g class now, and we'll give you the exam FREE. > http://ads.osdn.com/?ad_id149&alloc_id66&op=click > _______________________________________________ > Springframework-developer mailing list > Spr...@li... > https://lists.sourceforge.net/lists/listinfo/springframework-developer |
|
From: <jue...@we...> - 2004-05-24 17:29:16
|
Tim, As I'm about to release Spring 1.0.2 tomorrow, it would be great if you = could give the current CVS HEAD a try promptly. I'm pretty sure that it = alleviates the situation as much as possible, but a verification always = helps :-) Juergen -----Original Message----- From: spr...@li... [mailto:spr...@li...]On Behalf Of Tim Kettering Sent: Monday, May 24, 2004 4:47 PM To: spr...@li... Subject: Re: [Springframework-developer] Cleanup of context resources on webapp reload Jurgen, Thanks for looking into this issue. When I originally reported on it, =20 and didnt hear much discussion on it, I had started to think that i was = the only person experiencing the memory leak and that I was coding =20 something wrong. Your experiences w/ JProfiler are very similar to what I was seeing =20 when I was running JProfiler against our code, although you've =20 obviously delved deeper into the exact reasons for those leaks. I will = give the latest build of spring a try on our code and hopefully it =20 should help alleviate the situation some. I also wanted to add a comment in respect to what Guillaume said below =20 that we've consistently seen OOM errors coming from Tomcat after X =20 amount of reloads done by the test server. And this can happen over =20 the span of several days, so I do not believe that those resources are =20 going to be GC'ed at all, unless the definition of "took a while" means = a week or so. :) Resin may be doing something different however. -tim On May 24, 2004, at 8:48 AM, Guillaume Poirier wrote: > Well, I find it really weird, because I've tested such behavior before = =20 > on > Resin 2.1.10, and while it took a while before the resources were =20 > garbage > collected, they were always eventually collected. And if I added a > System.gc() when the ServletContext was loaded, all my singletons were = =20 > being > collected right away. During all your tests, did you notice higher =20 > memory > consumption or did you actually get a OutOfMemoryError ? May be it's = > the > profiler that keeps a reference on some resource(s) for some reasons? > > But if it is really a ClassLoader issue, shouldn't you be able to =20 > simulate > the situation outside a Servlet Container? I've run the attached =20 > test, and > while I create a memory leak quickly if I keep a reference on the > ClassLoader, I am not able to produce one without holding a ref on it. > > Or is my test flawed? > > Guillaume > > ----- Original Message ----- > From: "j=FCrgen h=F6ller [werk3AT]" <jue...@we...> > To: <spr...@li...> > Sent: Monday, May 24, 2004 4:41 AM > Subject: Re: [Springframework-developer] Cleanup of context resources = > on > webapp reload > > > Just finished tests on Resin 2.1.11 - exactly same behavior as with =20 > Tomcat. > This *is* a general class loader respectively garbage collection = issue, > rather than a server-specific leak. > > Consequently, with the changes I've committed yesterday, there is the = > same > significant benefit as with Tomcat. The only remaining issue are the =20 > "full > object" constants like ClassFilter.TRUE. > > Juergen > > > ________________________________ > > Von: spr...@li... im Auftrag = > von > j=FCrgen h=F6ller [werk3AT] > Gesendet: Mo 24.05.2004 09:20 > An: spr...@li... > Betreff: Re: [Springframework-developer] Cleanup of context resources = > on > webapp reload > > > > FYI, I've tested both Tomcat 5.0.18 and 4.1.27 - same behavior with =20 > both. > > BTW, a couple of related posts from the Resin mailing list: > > http://www.caucho.com/support/resin-interest/0404/0117.html > http://www.caucho.com/support/resin-interest/0111/0164.html > <http://www.caucho.com/support/resin-interest/0111/0164.html> > > Juergen > > > ________________________________ > > Von: spr...@li... im Auftrag = > von > j=FCrgen h=F6ller [werk3AT] > Gesendet: Mo 24.05.2004 08:42 > An: spr...@li... > Betreff: Re: [Springframework-developer] Cleanup of context resources = > on > webapp reload > > > > I've told JProfiler to explicitly run garbage collection - a number of > times, actually - before I've had a look at the heap. So I'm sure that = =20 > those > remaining objects were not garbage-collected, and probably would have = > stayed > around in the VM forever... > > I'm aware that it seems odd, but this issue just seems to affect =20 > specific > static fields: BeanWrapperImpl's defaultEditors did not cause a leak, = > but > CachedIntrospectionResults' classCache did. Normal constants or static > logger fields didn't, but "full object" constants like =20 > ClassFilters.TRUE > did. > > CachedIntrospectionResults' classCache uses the Class as key; the =20 > value, a > CachedIntrospectionResult object, also refers to the key Class. > Consequently, I had to use a WeakHashMap *with WeakReferences as =20 > values* to > see proper garbage collection. Note that this static cache contains > instances of its containing class as values. > > I've run all my tests x times to make sure that I could trust my eyes. = =20 > I've > even undone the WeakReference changes again, and voila, there were the = =20 > leaks > again. I'd be happy to learn more about how the garbage collector = works > here... All I can state at this point of time is that the changes did = > cause > an obvious difference in terms of resource leaks. > > Juergen > > > ________________________________ > > Von: spr...@li... im Auftrag = > von > Guillaume Poirier > Gesendet: Mo 24.05.2004 05:17 > An: spr...@li... > Betreff: Re: [Springframework-developer] Cleanup of context resources = > on > webapp reload > > > > The "resource leak" caused by a singleton when a webapp's classloader = > is > thrown away is only temorary, the unused classes and the classloader =20 > will be > eventually garbage collected and the resources will be freed. The = only > thing that could prevent that is if there was a something in the =20 > server's > classloader that still had a reference on an object or a class of the = > child > classloader. > > Unless there's a bug in Tomcat or in the application code, I really =20 > can't > see how the classes won't be eventually garbage collected when the JVM = =20 > needs > memory. And anyway, if there was indeed a leak because =20 > SQLErrorCodesFactory > is a singleton, why wouldn't there be one for each static fields such = > as > constants? > > Are you sure that JProfiler does not disable garbage collecting in =20 > order to > make its profiling? I know that in many of the JVMPI method calls are = =20 > done > with garbage collecting off. I suspect the above to be the cause of =20 > the > "resource leak", rather than any singleton that Spring might be using. > > Guillaume > > ----- Original Message ----- > From: "j=FCrgen h=F6ller [werk3AT]" <jue...@we...> > To: <spr...@li...> > Sent: Sunday, May 23, 2004 4:45 PM > Subject: Re: [Springframework-developer] Cleanup of context resources = > on > webapp reload > > > I've just spent about 10 hours profiling Spring, using the Image =20 > Database > and Petclinic samples. (BTW, I've used an evaluation version of =20 > JProfiler > from ej-technologies - nice product!) > > Although I still don't completely understand the garbage collection > behavior, I've figured out the following issues. Each of them simply > prevents the respective classes from getting garbage collected on > destruction of the class loader (e.g. on Tomcat web app shutdown). > > - A classic singleton with a class variable holding the object. I've > reworked GlobalAdvisorAdapterRegistry and SQLErrorCodesFactory to hold = =20 > the > respective singleton as a WeakReference. > > - A static cache. I've reworked CachedIntrospectionResults to use a > WeakHashMap with WeakReferences as values. > > - A ThreadLocal with a default other than null. I've reworked > TransactionSynchronizationManager to use null as default for the =20 > resource > map, setting a HashMap there on demand, removing the entire HashMap =20 > when > unbinding the last resource. > > - Constants that define a full object. We have a number of those, for > example ClassFilters.TRUE and AdvisedSupport.EMPTY_TARGET_SOURCE. I've = =20 > tried > for quite a while, but I haven't been able to figure out a way to =20 > define > such constants such that they will be garbage collected. > > The latter programming style is not uncommon, so I really don't =20 > understand > why it causes trouble with garbage collection. Hibernate uses a = similar > style for its FlushMode, for example. > > In general, other frameworks like CGLIB, Hibernate, Velocity have huge > resource leaks on web app shutdown, while just the constants issue =20 > remains > with Spring now. As long as those huge third-party leaks are not =20 > addressed, > I'm not worried at all by the single remaining Spring issue. > > As I initially said, we shouldn't exaggerate the problem, as it =20 > basically > just affects hot reloading of web apps - mainly a development feature > anyway. We need to make that clear to users too, to avoid comments a = la > "Spring is not usable for real apps because it leaks on hot =20 > redeployment". > > Please, everybody, give the current CVS head a sanity check tomorrow. = > There > shouldn't be any issues: the test suite passes, the sample apps run > properly. Still, I'd feel more comfortable if we make sure that no =20 > subtle > side effects have been introduced. > > For this reason, I will delay release 1.0.2 till tomorrow night. > > Juergen > > > ________________________________ > > Von: spr...@li... im Auftrag = > von > j=FCrgen h=F6ller [werk3AT] > Gesendet: Sa 22.05.2004 15:52 > An: spr...@li... > Betreff: Re: [Springframework-developer] Cleanup of context resources = > on > webapp reload > > > > So basically, all static caches cause resource leaks when restarting a > Tomcat web app? I wonder why this happens... The class loader should > completely dissolve all classes that it has loaded in its lifetime, > including static caches. Or have I misunderstood something here? = Anyone > having in-detail experience with handling such a scenario? > > Juergen > > > ________________________________ > > Von: spr...@li... im Auftrag = > von > Dmitriy Kopylenko > Gesendet: Di 04.05.2004 18:02 > An: spr...@li... > Betreff: Re: [Springframework-developer] Cleanup of context resources = > on > webapp reload > > > > Well, > > for instance SQLErrorCodesFactory is a singleton(GoF, not Spring) = which > caches SQLErrorCodes internally in the Map with strong references. =20 > Again, I > don't know if trying > to use WeakHashMap there would do the trick... > > Dmitriy > > Tim Kettering wrote: > >> >> I looked at it some more this morning, and basically what I did was >> start up tomcat w/ the webapp in the profiler, then after it was done >> starting up I used tomcat's manager to stop the context. This should >> destroy all resources related to the context. Here is a list of >> spring related stuff that still were in memory after the context was >> closed. Other stuff was cleaned up just fine. >> >> org.springframework.beans.CachedIntrospectionResults >> org.springframework.jdbc.support.SQLCodes >> org.springframework.core.Constants >> org.springframework.aop.framework.AdvisedSupport$1 >> org.springframework.aop.framework.adapter.BeforeAdviceAdapter >> org.springframework.aop.framework.adapter.AfterReturningAdviceAdapter >> org.springframework.jdbc.support.SQLErrorCodesFactory >> = org.springframework.transaction.support.TransactionSynchronizationMana=20 >> ge >> r$1 >> org.springframework.transaction.interceptor.RollbackRuleAttribute >> org.springframework.aop.framework.adapter.ThrosAdviceAdapter >> org.springframework.aop.Pointcut$1 >> = org.springframework.aop.framework.adapter.GlobalAdvisorAdapterRegistry >> >> On May 3, 2004, at 3:28 PM, Dmitriy Kopylenko wrote: >> >>> I'm just wondering, would the use of WeakHashMap in >>> CachedIntrospectionResults help? >>> >>> Dmitriy. >>> >>> Tim Kettering wrote: >>> >>>> I posted this to the users list last week and did not receive any >>>> reply on it, so I'm posting it again here on the developer list, = in >>>> hopes i could get an reply from someone here. I'm trying to >>>> determine if its something I should be doing myself, or if hte >>>> spring context should be cleaning up those resources by itself on >>>> the .close() call. Further profiling shows that there are >>>> duplicate instances of SQLError and hibernate proxy classes =20 >>>> hanging >>>> around afterwards too. Other objects do get cleaned up properly. >>>> -------- >>>> Hi everyone, >>>> We're (meaning me) looking into some resource leaks that are >>>> occuring when our webapp context gets reloaded. I found that >>>> context.close() needs to be called on the destroy() method of >>>> plugin we're using, and it works for a good majority of the =20 >>>> objects >>>> we were seeing leaked, but there are some objects that I'm unable >>>> to make go away. Object in question is the: >>>> org.springframework.beans.CachedIntrospectionResults >>>> Whenever I reload the context - the profiler I'm using shows that I >>>> have essentially a duplicate group of those objects (same instance >>>> count) as the original, and successive reloads will continue to >>>> duplicate this. >>>> The profiler also shows the final reference to those objects like = >>>> this: >>>> 100% - 1008 bytes - 63 alloc. >>>> = org.springframework.context.support.ClassPathXmlApplicationContext.<=20 >>>> in >>>> it > >>>> So basically I guess what I'm asking is for ideas or suggestions on >>>> how I could get those to clean up. This bean doesnt show up in = the >>>> Spring javadocs. And looking in CVS says its a package level = bean, >>>> not for application use, so I'm thinking that closing the context >>>> should (in theory) clean this up? Thanks in advance. >>>> -tim >>>> ------------------------------------------------------- >>>> This SF.Net email is sponsored by: Oracle 10g >>>> Get certified on the hottest thing ever to hit the market... Oracle >>>> 10g. Take an Oracle 10g class now, and we'll give you the exam = FREE. >>>> http://ads.osdn.com/?ad_id=3D3149&alloc_id=3D8166&op=3Dclick >>>> _______________________________________________ >>>> Springframework-developer mailing list >>>> Spr...@li... >>>> https://lists.sourceforge.net/lists/listinfo/springframework-=20 >>>> developer >>> >>> >>> >>> >>> ------------------------------------------------------- >>> This SF.Net email is sponsored by: Oracle 10g >>> Get certified on the hottest thing ever to hit the market... Oracle >>> 10g. Take an Oracle 10g class now, and we'll give you the exam FREE. >>> http://ads.osdn.com/?ad_id=3D3149&alloc_id=3D8166&op=3Dclick >>> _______________________________________________ >>> Springframework-developer mailing list >>> Spr...@li... >>> https://lists.sourceforge.net/lists/listinfo/springframework-=20 >>> developer >>> >> >> >> >> ------------------------------------------------------- >> This SF.Net email is sponsored by: Oracle 10g >> Get certified on the hottest thing ever to hit the market... Oracle =20 >> 10g. >> Take an Oracle 10g class now, and we'll give you the exam FREE. >> http://ads.osdn.com/?ad_id=3D3149&alloc_id=3D8166&op=3Dclick >> _______________________________________________ >> Springframework-developer mailing list >> Spr...@li... >> = https://lists.sourceforge.net/lists/listinfo/springframework-developer > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: Oracle 10g > Get certified on the hottest thing ever to hit the market... Oracle =20 > 10g. > Take an Oracle 10g class now, and we'll give you the exam FREE. > http://ads.osdn.com/?ad_id=3D3149&alloc_id=3D8166&op=3Dclick > _______________________________________________ > Springframework-developer mailing list > Spr...@li... > https://lists.sourceforge.net/lists/listinfo/springframework-developer > > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: Oracle 10g > Get certified on the hottest thing ever to hit the market... Oracle =20 > 10g. > Take an Oracle 10g class now, and we'll give you the exam FREE. > http://ads.osdn.com/?ad_id149&alloc_id=9966&op=3Dick > _______________________________________________ > Springframework-developer mailing list > Spr...@li... > https://lists.sourceforge.net/lists/listinfo/springframework-developer > > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: Oracle 10g > Get certified on the hottest thing ever to hit the market... Oracle =20 > 10g. > Take an Oracle 10g class now, and we'll give you the exam FREE. > http://ads.osdn.com/?ad_id149&alloc_id=9966&op=3Dick > _______________________________________________ > Springframework-developer mailing list > Spr...@li... > https://lists.sourceforge.net/lists/listinfo/springframework-developer > > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: Oracle 10g > Get certified on the hottest thing ever to hit the market... Oracle =20 > 10g. > Take an Oracle 10g class now, and we'll give you the exam FREE. > http://ads.osdn.com/?ad_id=3D3149&alloc_id=3D8166&op=3Dclick > _______________________________________________ > Springframework-developer mailing list > Spr...@li... > https://lists.sourceforge.net/lists/listinfo/springframework-developer > > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: Oracle 10g > Get certified on the hottest thing ever to hit the market... Oracle =20 > 10g. > Take an Oracle 10g class now, and we'll give you the exam FREE. > http://ads.osdn.com/?ad_id149&alloc_id=9966&op=3Dick > _______________________________________________ > Springframework-developer mailing list > Spr...@li... > https://lists.sourceforge.net/lists/listinfo/springframework-developer > > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: Oracle 10g > Get certified on the hottest thing ever to hit the market... Oracle =20 > 10g. > Take an Oracle 10g class now, and we'll give you the exam FREE. > http://ads.osdn.com/?ad_id149&alloc_id=9966&op=3Dick > _______________________________________________ > Springframework-developer mailing list > Spr...@li... > https://lists.sourceforge.net/lists/listinfo/springframework-developer > > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: Oracle 10g > Get certified on the hottest thing ever to hit the market... Oracle =20 > 10g. > Take an Oracle 10g class now, and we'll give you the exam FREE. > http://ads.osdn.com/?ad_id149&alloc_id=9966&op=3Dick > _______________________________________________ > Springframework-developer mailing list > Spr...@li... > https://lists.sourceforge.net/lists/listinfo/springframework-developer > <classloader.tar.gz> ------------------------------------------------------- This SF.Net email is sponsored by: Oracle 10g Get certified on the hottest thing ever to hit the market... Oracle 10g. = Take an Oracle 10g class now, and we'll give you the exam FREE. http://ads.osdn.com/?ad_id149&alloc_id=8166&op=3Dick _______________________________________________ Springframework-developer mailing list Spr...@li... https://lists.sourceforge.net/lists/listinfo/springframework-developer |
|
From: Tim K. <tim...@vi...> - 2004-05-24 18:20:19
|
I'd love to check it out, but I'm jammed up w/ other stuff until end of =20= this week. Being that we're just restarting Tomcat on every deploy, =20 this problem isn't a high priority for us at the moment - the OOMs only =20= occur when reloading the context in Tomcat. But I will check it out =20 with the 1.0.2 release as soon as possible and let you know of my =20 findings. Thanks again for looking into it. -tim On May 24, 2004, at 1:28 PM, j=9Frgen h=9Aller [werk3AT] wrote: > Tim, > > As I'm about to release Spring 1.0.2 tomorrow, it would be great if =20= > you could give the current CVS HEAD a try promptly. I'm pretty sure =20= > that it alleviates the situation as much as possible, but a =20 > verification always helps :-) > > Juergen > > > -----Original Message----- > From: spr...@li... > [mailto:spr...@li...]On = Behalf > Of Tim Kettering > Sent: Monday, May 24, 2004 4:47 PM > To: spr...@li... > Subject: Re: [Springframework-developer] Cleanup of context resources =20= > on > webapp reload > > > Jurgen, > > Thanks for looking into this issue. When I originally reported on it, > and didnt hear much discussion on it, I had started to think that i = was > the only person experiencing the memory leak and that I was coding > something wrong. > > Your experiences w/ JProfiler are very similar to what I was seeing > when I was running JProfiler against our code, although you've > obviously delved deeper into the exact reasons for those leaks. I = will > give the latest build of spring a try on our code and hopefully it > should help alleviate the situation some. > > I also wanted to add a comment in respect to what Guillaume said below > that we've consistently seen OOM errors coming from Tomcat after X > amount of reloads done by the test server. And this can happen over > the span of several days, so I do not believe that those resources are > going to be GC'ed at all, unless the definition of "took a while" = means > a week or so. :) Resin may be doing something different however. > > -tim > > > On May 24, 2004, at 8:48 AM, Guillaume Poirier wrote: > >> Well, I find it really weird, because I've tested such behavior = before >> on >> Resin 2.1.10, and while it took a while before the resources were >> garbage >> collected, they were always eventually collected. And if I added a >> System.gc() when the ServletContext was loaded, all my singletons = were >> being >> collected right away. During all your tests, did you notice higher >> memory >> consumption or did you actually get a OutOfMemoryError ? May be it's >> the >> profiler that keeps a reference on some resource(s) for some reasons? >> >> But if it is really a ClassLoader issue, shouldn't you be able to >> simulate >> the situation outside a Servlet Container? I've run the attached >> test, and >> while I create a memory leak quickly if I keep a reference on the >> ClassLoader, I am not able to produce one without holding a ref on = it. >> >> Or is my test flawed? >> >> Guillaume >> >> ----- Original Message ----- >> From: "j=9Frgen h=9Aller [werk3AT]" <jue...@we...> >> To: <spr...@li...> >> Sent: Monday, May 24, 2004 4:41 AM >> Subject: Re: [Springframework-developer] Cleanup of context resources >> on >> webapp reload >> >> >> Just finished tests on Resin 2.1.11 - exactly same behavior as with >> Tomcat. >> This *is* a general class loader respectively garbage collection =20 >> issue, >> rather than a server-specific leak. >> >> Consequently, with the changes I've committed yesterday, there is the >> same >> significant benefit as with Tomcat. The only remaining issue are the >> "full >> object" constants like ClassFilter.TRUE. >> >> Juergen >> >> >> ________________________________ >> >> Von: spr...@li... im Auftrag >> von >> j=9Frgen h=9Aller [werk3AT] >> Gesendet: Mo 24.05.2004 09:20 >> An: spr...@li... >> Betreff: Re: [Springframework-developer] Cleanup of context resources >> on >> webapp reload >> >> >> >> FYI, I've tested both Tomcat 5.0.18 and 4.1.27 - same behavior with >> both. >> >> BTW, a couple of related posts from the Resin mailing list: >> >> http://www.caucho.com/support/resin-interest/0404/0117.html >> http://www.caucho.com/support/resin-interest/0111/0164.html >> <http://www.caucho.com/support/resin-interest/0111/0164.html> >> >> Juergen >> >> >> ________________________________ >> >> Von: spr...@li... im Auftrag >> von >> j=9Frgen h=9Aller [werk3AT] >> Gesendet: Mo 24.05.2004 08:42 >> An: spr...@li... >> Betreff: Re: [Springframework-developer] Cleanup of context resources >> on >> webapp reload >> >> >> >> I've told JProfiler to explicitly run garbage collection - a number = of >> times, actually - before I've had a look at the heap. So I'm sure = that >> those >> remaining objects were not garbage-collected, and probably would have >> stayed >> around in the VM forever... >> >> I'm aware that it seems odd, but this issue just seems to affect >> specific >> static fields: BeanWrapperImpl's defaultEditors did not cause a leak, >> but >> CachedIntrospectionResults' classCache did. Normal constants or = static >> logger fields didn't, but "full object" constants like >> ClassFilters.TRUE >> did. >> >> CachedIntrospectionResults' classCache uses the Class as key; the >> value, a >> CachedIntrospectionResult object, also refers to the key Class. >> Consequently, I had to use a WeakHashMap *with WeakReferences as >> values* to >> see proper garbage collection. Note that this static cache contains >> instances of its containing class as values. >> >> I've run all my tests x times to make sure that I could trust my = eyes. >> I've >> even undone the WeakReference changes again, and voila, there were = the >> leaks >> again. I'd be happy to learn more about how the garbage collector =20 >> works >> here... All I can state at this point of time is that the changes did >> cause >> an obvious difference in terms of resource leaks. >> >> Juergen >> >> >> ________________________________ >> >> Von: spr...@li... im Auftrag >> von >> Guillaume Poirier >> Gesendet: Mo 24.05.2004 05:17 >> An: spr...@li... >> Betreff: Re: [Springframework-developer] Cleanup of context resources >> on >> webapp reload >> >> >> >> The "resource leak" caused by a singleton when a webapp's classloader >> is >> thrown away is only temorary, the unused classes and the classloader >> will be >> eventually garbage collected and the resources will be freed. The =20= >> only >> thing that could prevent that is if there was a something in the >> server's >> classloader that still had a reference on an object or a class of the >> child >> classloader. >> >> Unless there's a bug in Tomcat or in the application code, I really >> can't >> see how the classes won't be eventually garbage collected when the = JVM >> needs >> memory. And anyway, if there was indeed a leak because >> SQLErrorCodesFactory >> is a singleton, why wouldn't there be one for each static fields such >> as >> constants? >> >> Are you sure that JProfiler does not disable garbage collecting in >> order to >> make its profiling? I know that in many of the JVMPI method calls = are >> done >> with garbage collecting off. I suspect the above to be the cause of >> the >> "resource leak", rather than any singleton that Spring might be = using. >> >> Guillaume >> >> ----- Original Message ----- >> From: "j=9Frgen h=9Aller [werk3AT]" <jue...@we...> >> To: <spr...@li...> >> Sent: Sunday, May 23, 2004 4:45 PM >> Subject: Re: [Springframework-developer] Cleanup of context resources >> on >> webapp reload >> >> >> I've just spent about 10 hours profiling Spring, using the Image >> Database >> and Petclinic samples. (BTW, I've used an evaluation version of >> JProfiler >> from ej-technologies - nice product!) >> >> Although I still don't completely understand the garbage collection >> behavior, I've figured out the following issues. Each of them simply >> prevents the respective classes from getting garbage collected on >> destruction of the class loader (e.g. on Tomcat web app shutdown). >> >> - A classic singleton with a class variable holding the object. I've >> reworked GlobalAdvisorAdapterRegistry and SQLErrorCodesFactory to = hold >> the >> respective singleton as a WeakReference. >> >> - A static cache. I've reworked CachedIntrospectionResults to use a >> WeakHashMap with WeakReferences as values. >> >> - A ThreadLocal with a default other than null. I've reworked >> TransactionSynchronizationManager to use null as default for the >> resource >> map, setting a HashMap there on demand, removing the entire HashMap >> when >> unbinding the last resource. >> >> - Constants that define a full object. We have a number of those, for >> example ClassFilters.TRUE and AdvisedSupport.EMPTY_TARGET_SOURCE. = I've >> tried >> for quite a while, but I haven't been able to figure out a way to >> define >> such constants such that they will be garbage collected. >> >> The latter programming style is not uncommon, so I really don't >> understand >> why it causes trouble with garbage collection. Hibernate uses a =20 >> similar >> style for its FlushMode, for example. >> >> In general, other frameworks like CGLIB, Hibernate, Velocity have = huge >> resource leaks on web app shutdown, while just the constants issue >> remains >> with Spring now. As long as those huge third-party leaks are not >> addressed, >> I'm not worried at all by the single remaining Spring issue. >> >> As I initially said, we shouldn't exaggerate the problem, as it >> basically >> just affects hot reloading of web apps - mainly a development feature >> anyway. We need to make that clear to users too, to avoid comments a =20= >> la >> "Spring is not usable for real apps because it leaks on hot >> redeployment". >> >> Please, everybody, give the current CVS head a sanity check tomorrow. >> There >> shouldn't be any issues: the test suite passes, the sample apps run >> properly. Still, I'd feel more comfortable if we make sure that no >> subtle >> side effects have been introduced. >> >> For this reason, I will delay release 1.0.2 till tomorrow night. >> >> Juergen >> >> >> ________________________________ >> >> Von: spr...@li... im Auftrag >> von >> j=9Frgen h=9Aller [werk3AT] >> Gesendet: Sa 22.05.2004 15:52 >> An: spr...@li... >> Betreff: Re: [Springframework-developer] Cleanup of context resources >> on >> webapp reload >> >> >> >> So basically, all static caches cause resource leaks when restarting = a >> Tomcat web app? I wonder why this happens... The class loader should >> completely dissolve all classes that it has loaded in its lifetime, >> including static caches. Or have I misunderstood something here? =20 >> Anyone >> having in-detail experience with handling such a scenario? >> >> Juergen >> >> >> ________________________________ >> >> Von: spr...@li... im Auftrag >> von >> Dmitriy Kopylenko >> Gesendet: Di 04.05.2004 18:02 >> An: spr...@li... >> Betreff: Re: [Springframework-developer] Cleanup of context resources >> on >> webapp reload >> >> >> >> Well, >> >> for instance SQLErrorCodesFactory is a singleton(GoF, not Spring) =20 >> which >> caches SQLErrorCodes internally in the Map with strong references. >> Again, I >> don't know if trying >> to use WeakHashMap there would do the trick... >> >> Dmitriy >> >> Tim Kettering wrote: >> >>> >>> I looked at it some more this morning, and basically what I did was >>> start up tomcat w/ the webapp in the profiler, then after it was = done >>> starting up I used tomcat's manager to stop the context. This = should >>> destroy all resources related to the context. Here is a list of >>> spring related stuff that still were in memory after the context was >>> closed. Other stuff was cleaned up just fine. >>> >>> org.springframework.beans.CachedIntrospectionResults >>> org.springframework.jdbc.support.SQLCodes >>> org.springframework.core.Constants >>> org.springframework.aop.framework.AdvisedSupport$1 >>> org.springframework.aop.framework.adapter.BeforeAdviceAdapter >>> = org.springframework.aop.framework.adapter.AfterReturningAdviceAdapter >>> org.springframework.jdbc.support.SQLErrorCodesFactory >>> = org.springframework.transaction.support.TransactionSynchronizationMan=20 >>> a >>> ge >>> r$1 >>> org.springframework.transaction.interceptor.RollbackRuleAttribute >>> org.springframework.aop.framework.adapter.ThrosAdviceAdapter >>> org.springframework.aop.Pointcut$1 >>> = org.springframework.aop.framework.adapter.GlobalAdvisorAdapterRegistr=20 >>> y >>> >>> On May 3, 2004, at 3:28 PM, Dmitriy Kopylenko wrote: >>> >>>> I'm just wondering, would the use of WeakHashMap in >>>> CachedIntrospectionResults help? >>>> >>>> Dmitriy. >>>> >>>> Tim Kettering wrote: >>>> >>>>> I posted this to the users list last week and did not receive any >>>>> reply on it, so I'm posting it again here on the developer list, =20= >>>>> in >>>>> hopes i could get an reply from someone here. I'm trying to >>>>> determine if its something I should be doing myself, or if hte >>>>> spring context should be cleaning up those resources by itself on >>>>> the .close() call. Further profiling shows that there are >>>>> duplicate instances of SQLError and hibernate proxy classes >>>>> hanging >>>>> around afterwards too. Other objects do get cleaned up = properly. >>>>> -------- >>>>> Hi everyone, >>>>> We're (meaning me) looking into some resource leaks that are >>>>> occuring when our webapp context gets reloaded. I found that >>>>> context.close() needs to be called on the destroy() method of >>>>> plugin we're using, and it works for a good majority of the >>>>> objects >>>>> we were seeing leaked, but there are some objects that I'm = unable >>>>> to make go away. Object in question is the: >>>>> org.springframework.beans.CachedIntrospectionResults >>>>> Whenever I reload the context - the profiler I'm using shows that = I >>>>> have essentially a duplicate group of those objects (same instance >>>>> count) as the original, and successive reloads will continue to >>>>> duplicate this. >>>>> The profiler also shows the final reference to those objects like >>>>> this: >>>>> 100% - 1008 bytes - 63 alloc. >>>>> = org.springframework.context.support.ClassPathXmlApplicationContext.=20 >>>>> < >>>>> in >>>>> it > >>>>> So basically I guess what I'm asking is for ideas or suggestions = on >>>>> how I could get those to clean up. This bean doesnt show up in =20= >>>>> the >>>>> Spring javadocs. And looking in CVS says its a package level =20 >>>>> bean, >>>>> not for application use, so I'm thinking that closing the context >>>>> should (in theory) clean this up? Thanks in advance. >>>>> -tim >>>>> ------------------------------------------------------- >>>>> This SF.Net email is sponsored by: Oracle 10g >>>>> Get certified on the hottest thing ever to hit the market... = Oracle >>>>> 10g. Take an Oracle 10g class now, and we'll give you the exam =20 >>>>> FREE. >>>>> http://ads.osdn.com/?ad_id=3D3149&alloc_id=3D8166&op=3Dclick >>>>> _______________________________________________ >>>>> Springframework-developer mailing list >>>>> Spr...@li... >>>>> https://lists.sourceforge.net/lists/listinfo/springframework- >>>>> developer >>>> >>>> >>>> >>>> >>>> ------------------------------------------------------- >>>> This SF.Net email is sponsored by: Oracle 10g >>>> Get certified on the hottest thing ever to hit the market... Oracle >>>> 10g. Take an Oracle 10g class now, and we'll give you the exam = FREE. >>>> http://ads.osdn.com/?ad_id=3D3149&alloc_id=3D8166&op=3Dclick >>>> _______________________________________________ >>>> Springframework-developer mailing list >>>> Spr...@li... >>>> https://lists.sourceforge.net/lists/listinfo/springframework- >>>> developer >>>> >>> >>> >>> >>> ------------------------------------------------------- >>> This SF.Net email is sponsored by: Oracle 10g >>> Get certified on the hottest thing ever to hit the market... Oracle >>> 10g. >>> Take an Oracle 10g class now, and we'll give you the exam FREE. >>> http://ads.osdn.com/?ad_id=3D3149&alloc_id=3D8166&op=3Dclick >>> _______________________________________________ >>> Springframework-developer mailing list >>> Spr...@li... >>> https://lists.sourceforge.net/lists/listinfo/springframework-=20 >>> developer >> >> >> >> ------------------------------------------------------- >> This SF.Net email is sponsored by: Oracle 10g >> Get certified on the hottest thing ever to hit the market... Oracle >> 10g. >> Take an Oracle 10g class now, and we'll give you the exam FREE. >> http://ads.osdn.com/?ad_id=3D3149&alloc_id=3D8166&op=3Dclick >> _______________________________________________ >> Springframework-developer mailing list >> Spr...@li... >> = https://lists.sourceforge.net/lists/listinfo/springframework-developer >> >> >> >> >> ------------------------------------------------------- >> This SF.Net email is sponsored by: Oracle 10g >> Get certified on the hottest thing ever to hit the market... Oracle >> 10g. >> Take an Oracle 10g class now, and we'll give you the exam FREE. >> http://ads.osdn.com/?ad_id149&alloc_id=FA66&op=3Dick >> _______________________________________________ >> Springframework-developer mailing list >> Spr...@li... >> = https://lists.sourceforge.net/lists/listinfo/springframework-developer >> >> >> >> >> ------------------------------------------------------- >> This SF.Net email is sponsored by: Oracle 10g >> Get certified on the hottest thing ever to hit the market... Oracle >> 10g. >> Take an Oracle 10g class now, and we'll give you the exam FREE. >> http://ads.osdn.com/?ad_id149&alloc_id=FA66&op=3Dick >> _______________________________________________ >> Springframework-developer mailing list >> Spr...@li... >> = https://lists.sourceforge.net/lists/listinfo/springframework-developer >> >> >> >> >> ------------------------------------------------------- >> This SF.Net email is sponsored by: Oracle 10g >> Get certified on the hottest thing ever to hit the market... Oracle >> 10g. >> Take an Oracle 10g class now, and we'll give you the exam FREE. >> http://ads.osdn.com/?ad_id=3D3149&alloc_id=3D8166&op=3Dclick >> _______________________________________________ >> Springframework-developer mailing list >> Spr...@li... >> = https://lists.sourceforge.net/lists/listinfo/springframework-developer >> >> >> >> >> ------------------------------------------------------- >> This SF.Net email is sponsored by: Oracle 10g >> Get certified on the hottest thing ever to hit the market... Oracle >> 10g. >> Take an Oracle 10g class now, and we'll give you the exam FREE. >> http://ads.osdn.com/?ad_id149&alloc_id=FA66&op=3Dick >> _______________________________________________ >> Springframework-developer mailing list >> Spr...@li... >> = https://lists.sourceforge.net/lists/listinfo/springframework-developer >> >> >> >> >> ------------------------------------------------------- >> This SF.Net email is sponsored by: Oracle 10g >> Get certified on the hottest thing ever to hit the market... Oracle >> 10g. >> Take an Oracle 10g class now, and we'll give you the exam FREE. >> http://ads.osdn.com/?ad_id149&alloc_id=FA66&op=3Dick >> _______________________________________________ >> Springframework-developer mailing list >> Spr...@li... >> = https://lists.sourceforge.net/lists/listinfo/springframework-developer >> >> >> >> >> ------------------------------------------------------- >> This SF.Net email is sponsored by: Oracle 10g >> Get certified on the hottest thing ever to hit the market... Oracle >> 10g. >> Take an Oracle 10g class now, and we'll give you the exam FREE. >> http://ads.osdn.com/?ad_id149&alloc_id=FA66&op=3Dick >> _______________________________________________ >> Springframework-developer mailing list >> Spr...@li... >> = https://lists.sourceforge.net/lists/listinfo/springframework-developer >> <classloader.tar.gz> > > > ------------------------------------------------------- > This SF.Net email is sponsored by: Oracle 10g > Get certified on the hottest thing ever to hit the market... Oracle =20= > 10g. > Take an Oracle 10g class now, and we'll give you the exam FREE. > http://ads.osdn.com/?ad_id149&alloc_id=AA66&op=3Dick > _______________________________________________ > Springframework-developer mailing list > Spr...@li... > https://lists.sourceforge.net/lists/listinfo/springframework-developer > > > ------------------------------------------------------- > This SF.Net email is sponsored by: Oracle 10g > Get certified on the hottest thing ever to hit the market... Oracle =20= > 10g. > Take an Oracle 10g class now, and we'll give you the exam FREE. > http://ads.osdn.com/?ad_id149&alloc_id=AA66&op=3Dclick > _______________________________________________ > Springframework-developer mailing list > Spr...@li... > https://lists.sourceforge.net/lists/listinfo/springframework-developer > |
|
From: <jue...@we...> - 2004-05-24 18:16:24
|
As you will notice in my follow-ups to the mail you quoted, I have = already verified the issue, done a lot of profiling, and adapted = affected Spring code as far as possible. This will make it into release = 1.0.2, to be released tomorrow. For details, see my other mails. Juergen -----Original Message----- From: spr...@li... [mailto:spr...@li...]On Behalf Of Seth Ladd Sent: Monday, May 24, 2004 7:59 PM To: spr...@li... Subject: Re: [Springframework-developer] Cleanup of context resources on webapp reload -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 j=FCrgen h=F6ller [werk3AT] wrote: | So basically, all static caches cause resource leaks when restarting a Tomcat web app? I wonder why this happens... The class loader should completely dissolve all classes that it has loaded in its lifetime, including static caches. Or have I misunderstood something here? Anyone having in-detail experience with handling such a scenario? | That has been my experience. I've noticed the static init blocks from BeanWrapperImpl never leaving scope after a webapp uninstall. Again, an easy way to verify this is to just redeploy the spring-minimal.war over and over. This will generate a OOM. Seth -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.3-nr1 (Windows XP) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFAsjfxKZsFSwtW+wIRAiRTAJ9CBVRbheW6McGOLKbunWEyXeESZwCfU4sO tzdNXKEXcoYjFUuLxkjX/Ig=3D =3Dzu5p -----END PGP SIGNATURE----- ------------------------------------------------------- This SF.Net email is sponsored by: Oracle 10g Get certified on the hottest thing ever to hit the market... Oracle 10g. = Take an Oracle 10g class now, and we'll give you the exam FREE. http://ads.osdn.com/?ad_id=3D3149&alloc_id=3D8166&op=3Dclick _______________________________________________ Springframework-developer mailing list Spr...@li... https://lists.sourceforge.net/lists/listinfo/springframework-developer |
|
From: Seth L. <se...@eh...> - 2004-05-24 18:27:41
|
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 jürgen höller [werk3AT] wrote: | As you will notice in my follow-ups to the mail you quoted, I have already verified the issue, done a lot of profiling, and adapted affected Spring code as far as possible. This will make it into release 1.0.2, to be released tomorrow. For details, see my other mails. Yep, sorry about that... that's the problem with responding to weekend piles of email before reading through them all. :) Seth -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.3-nr1 (Windows XP) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFAsj8KKZsFSwtW+wIRAtOnAJ0YbYZfFnzIZCJueRNQaq1IWjujCgCffCOf EBsfqB2nxqI5WBngrGbPux4= =5XmN -----END PGP SIGNATURE----- |
|
From: <jue...@we...> - 2004-05-24 18:50:06
|
Don't worry, I knew that you had just been halfway through :-) Juergen -----Original Message----- From: spr...@li... [mailto:spr...@li...]On Behalf Of Seth Ladd Sent: Monday, May 24, 2004 8:30 PM To: spr...@li... Subject: Re: [Springframework-developer] Cleanup of context resources on webapp reload -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 j=FCrgen h=F6ller [werk3AT] wrote: | As you will notice in my follow-ups to the mail you quoted, I have already verified the issue, done a lot of profiling, and adapted affected Spring code as far as possible. This will make it into release 1.0.2, to be released tomorrow. For details, see my other mails. Yep, sorry about that... that's the problem with responding to weekend piles of email before reading through them all. :) Seth -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.3-nr1 (Windows XP) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFAsj8KKZsFSwtW+wIRAtOnAJ0YbYZfFnzIZCJueRNQaq1IWjujCgCffCOf EBsfqB2nxqI5WBngrGbPux4=3D =3D5XmN -----END PGP SIGNATURE----- ------------------------------------------------------- This SF.Net email is sponsored by: Oracle 10g Get certified on the hottest thing ever to hit the market... Oracle 10g. = Take an Oracle 10g class now, and we'll give you the exam FREE. http://ads.osdn.com/?ad_id=3D3149&alloc_id=3D8166&op=3Dclick _______________________________________________ Springframework-developer mailing list Spr...@li... https://lists.sourceforge.net/lists/listinfo/springframework-developer |
|
From: Rodrigo K. <ku...@re...> - 2004-05-27 21:42:18
|
Hi. I tried to produce some sort of classloading hack to minimize this problem but I found that it's nether Spring's or Tomcat's fault. The problem can be caused by simple use of j2se APIs. All your patches won't really reduce the problem as no class can be collected before it's classloader, the spec says that and the ClassLoader implementation from Sun enforce. Most of the times the OOM during hotdeploy is caused by not having enouth memory for the class semi-space, or whatever it's name is, and not for regular objects. Some API's have caching and thus break class unloading, an example is java.beans.Introspector, you must call Introspector.flushCaches() during context destroy on Sun JVM, and this is a bug as it should use weak references. I believe the solution is to look for these classes and never use any non-standard classes the container makes available. Anyway, it will only work if your container provides classloading that is not broken like some AS do. |
|
From: <jue...@we...> - 2004-05-29 18:29:09
|
Guillaume, =20 I've prototypically added an Introspector.flushCaches call to context = shutdown: I don't see any difference in the profiler. That's not too = surprising, as the originally leaking classes are not managed by beans = facilities in the first place: for example, SQLErrorCodesFactory, which = is just used internally by SQLErrorCodeSQLExceptionTranslator. =20 Of course, since my changes from a week ago, those classes don't leak = anymore, as they hold their singleton instance in a WeakReference now... = I still don't understand why this is necessary, but I'm 100% sure that = it does make a difference on both Sun JDK 1.4.2 and Sun JDK 1.3.1. I've = also tried various GC configuration options - always the same effect. =20 Juergen =20 ________________________________ Von: spr...@li... im Auftrag = von Guillaume Poirier Gesendet: Sa 29.05.2004 06:12 An: spr...@li... Betreff: Re: [Springframework-developer] Cleanup of context resources on = webapp reload I experimented some more about this cleanup issue, and I was able to = narrow down the problem to the caching done by the java.beans.Introspector. It stores the BeanInfo instances in a WeakHashMap, but in that Map implementation, only the keys uses WeakReference, the values are stored = with hard references since BeanInfo has an hard reference on the class it = gives info about (indirectly through BeanDescriptor and others), any time Introspector.getBeanInfo(Class) is used, that class and it's static = members will not ever be able to be garbage collected. I kind of remember = someone mentioning something related to this in the mailling list, but I cannot = find the mail. I wonder if there's other case where the java[x] classes = might have an hard reference on a class or its instances. A fix for this particular problem is to have a ServletContextListener call Introspector.flushCaches() when the context is destroyed. It seems like a known issue at Sun : http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=3D4291376 http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=3D4730581 http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=3D4809008 So, unless I'm missing something here, that means fixes like using synchronization and WeakReference on singleton probably won't help much = if at all. The only way that I can see for a class not to be gargage = collected when no more active thread use it, is if another ClassLoader has an hard reference to the class instance, or an instance of that class. Having = the class itself have an hard reference on a its own singleton has no = effect, it's a circular reference that will not prevent the class or the = instance to be gargabe collected when neither is being refered to by something else. Guillaume ------------------------------------------------------- This SF.Net email is sponsored by: Oracle 10g Get certified on the hottest thing ever to hit the market... Oracle 10g. Take an Oracle 10g class now, and we'll give you the exam FREE. http://ads.osdn.com/?ad_id=3D3149&alloc_id=3D8166&op=3Dclick _______________________________________________ Springframework-developer mailing list Spr...@li... https://lists.sourceforge.net/lists/listinfo/springframework-developer |
|
From: Guillaume P. <gpo...@gl...> - 2004-05-30 05:01:44
|
Juergen, I'm curious about how exactly you assert what is leaking, what is
not, and exactly what is it better with WeakReference? Are you using a
profiler that tells you the non-garbage collected objects in the JVM or do
you use other means?
Concerning SQLErrorCodesFactory, while the class' constructor does cause a
leak (indirectly by the use of XmlBeanFactory), it is not itself the leak,
it's the class instance of SQLErrorCodes that prevent the ClassLoader's
collection. I did a test calling
SQLErrorCodesFactory.getInstance().getErrorCodes("DB2") in a child
ClassLoader, and it caused a leak when the ClassLoader is thrown away. Then
I tried to create my own SQLErrorCodesFactory implementation, and it kept
leaking until I stopped using the XmlBeanFactory to create the instances of
SQLErrorCodes. Then I tried to just use an Introspector directly to set the
properties on the SQLErrorCodes instances by reflection, and it leaked just
as the SQLErrorCodesFactory. You have to keep in mind that if e.g.
Introspector has an hard reference on SQLErrorCodes.class, then none of the
class loaded by its ClassLoader can be collected. So it would be the cause
of SQLErrorCodesFactory singleton to be kept alive, not because
SQLFactoryErrorCodes has an hard reference on it, but because the
ClassLoader does, and SQLErrorCodes.class has an hard reference on the
ClassLoader, and the Introspector has an hard reference on
SQLErrorCodes.class. So the singleton of SQLFactoryCodesFactory is really
kept alive because Introspector has an hard reference on the class instance
of SQLErrorCodes.
I might misunderstand the situation, but as I see it, you're working on the
symptom rather than the cause. If you allow the singleton to be garbage
collected when the class itself is retained, then you might save some
memory, but it's just like adding memory to the JVM to solve a leak, it will
only delay the OutOfMemoryError, it won't prevent it. However, if you allow
the ClassLoader to be collected by removing any hard reference to its
classes, that would prevent the leak to even take place at all.
That's why I'm curious as to why you say it's better with WeakReference on
singleton and Introspector.flushCaches() does nothing, how do you make that
assertion? I realize that if there's something else than Introspector that
has an hard reference on a class of the ClassLoader being thown away,
flushing the Introspector's cache will have no effect on the leak, it would
still leak as fast. But while the WeakReference on the singleton will delay
the OutOfMemoryError, does it really help that much, since anyway all the
Class definitions and static members cannot be collected? You have to
consider that coding defensively on this might reduce performance because of
more object creation and use of synchronization, while also complicating the
code. Is it really worth it, did your tests really showed a significant
effect on a typical application?
Guillaume
----- Original Message -----
From: "jürgen höller [werk3AT]" <jue...@we...>
To: <spr...@li...>
Sent: Saturday, May 29, 2004 2:28 PM
Subject: Re: [Springframework-developer] Cleanup of context resources on
webapp reload
Guillaume,
I've prototypically added an Introspector.flushCaches call to context
shutdown: I don't see any difference in the profiler. That's not too
surprising, as the originally leaking classes are not managed by beans
facilities in the first place: for example, SQLErrorCodesFactory, which is
just used internally by SQLErrorCodeSQLExceptionTranslator.
Of course, since my changes from a week ago, those classes don't leak
anymore, as they hold their singleton instance in a WeakReference now... I
still don't understand why this is necessary, but I'm 100% sure that it does
make a difference on both Sun JDK 1.4.2 and Sun JDK 1.3.1. I've also tried
various GC configuration options - always the same effect.
Juergen
________________________________
Von: spr...@li... im Auftrag von
Guillaume Poirier
Gesendet: Sa 29.05.2004 06:12
An: spr...@li...
Betreff: Re: [Springframework-developer] Cleanup of context resources on
webapp reload
I experimented some more about this cleanup issue, and I was able to narrow
down the problem to the caching done by the java.beans.Introspector. It
stores the BeanInfo instances in a WeakHashMap, but in that Map
implementation, only the keys uses WeakReference, the values are stored with
hard references since BeanInfo has an hard reference on the class it gives
info about (indirectly through BeanDescriptor and others), any time
Introspector.getBeanInfo(Class) is used, that class and it's static members
will not ever be able to be garbage collected. I kind of remember someone
mentioning something related to this in the mailling list, but I cannot find
the mail. I wonder if there's other case where the java[x] classes might
have an hard reference on a class or its instances. A fix for this
particular problem is to have a ServletContextListener call
Introspector.flushCaches() when the context is destroyed.
It seems like a known issue at Sun :
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4291376
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4730581
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4809008
So, unless I'm missing something here, that means fixes like using
synchronization and WeakReference on singleton probably won't help much if
at all. The only way that I can see for a class not to be gargage collected
when no more active thread use it, is if another ClassLoader has an hard
reference to the class instance, or an instance of that class. Having the
class itself have an hard reference on a its own singleton has no effect,
it's a circular reference that will not prevent the class or the instance to
be gargabe collected when neither is being refered to by something else.
Guillaume
-------------------------------------------------------
This SF.Net email is sponsored by: Oracle 10g
Get certified on the hottest thing ever to hit the market... Oracle 10g.
Take an Oracle 10g class now, and we'll give you the exam FREE.
http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click
_______________________________________________
Springframework-developer mailing list
Spr...@li...
https://lists.sourceforge.net/lists/listinfo/springframework-developer
-------------------------------------------------------
This SF.Net email is sponsored by: Oracle 10g
Get certified on the hottest thing ever to hit the market... Oracle 10g.
Take an Oracle 10g class now, and we'll give you the exam FREE.
http://ads.osdn.com/?ad_id149&alloc_id66&op=ick
_______________________________________________
Springframework-developer mailing list
Spr...@li...
https://lists.sourceforge.net/lists/listinfo/springframework-developer
|
