Menu
▾
▴
Re: [Springframework-developer] Cleanup of context resources on webapp reload
|
From: Tim K. <tim...@vi...> - 2004-05-24 18:20:19
|
I'd love to check it out, but I'm jammed up w/ other stuff until end of =20= this week. Being that we're just restarting Tomcat on every deploy, =20 this problem isn't a high priority for us at the moment - the OOMs only =20= occur when reloading the context in Tomcat. But I will check it out =20 with the 1.0.2 release as soon as possible and let you know of my =20 findings. Thanks again for looking into it. -tim On May 24, 2004, at 1:28 PM, j=9Frgen h=9Aller [werk3AT] wrote: > Tim, > > As I'm about to release Spring 1.0.2 tomorrow, it would be great if =20= > you could give the current CVS HEAD a try promptly. I'm pretty sure =20= > that it alleviates the situation as much as possible, but a =20 > verification always helps :-) > > Juergen > > > -----Original Message----- > From: spr...@li... > [mailto:spr...@li...]On = Behalf > Of Tim Kettering > Sent: Monday, May 24, 2004 4:47 PM > To: spr...@li... > Subject: Re: [Springframework-developer] Cleanup of context resources =20= > on > webapp reload > > > Jurgen, > > Thanks for looking into this issue. When I originally reported on it, > and didnt hear much discussion on it, I had started to think that i = was > the only person experiencing the memory leak and that I was coding > something wrong. > > Your experiences w/ JProfiler are very similar to what I was seeing > when I was running JProfiler against our code, although you've > obviously delved deeper into the exact reasons for those leaks. I = will > give the latest build of spring a try on our code and hopefully it > should help alleviate the situation some. > > I also wanted to add a comment in respect to what Guillaume said below > that we've consistently seen OOM errors coming from Tomcat after X > amount of reloads done by the test server. And this can happen over > the span of several days, so I do not believe that those resources are > going to be GC'ed at all, unless the definition of "took a while" = means > a week or so. :) Resin may be doing something different however. > > -tim > > > On May 24, 2004, at 8:48 AM, Guillaume Poirier wrote: > >> Well, I find it really weird, because I've tested such behavior = before >> on >> Resin 2.1.10, and while it took a while before the resources were >> garbage >> collected, they were always eventually collected. And if I added a >> System.gc() when the ServletContext was loaded, all my singletons = were >> being >> collected right away. During all your tests, did you notice higher >> memory >> consumption or did you actually get a OutOfMemoryError ? May be it's >> the >> profiler that keeps a reference on some resource(s) for some reasons? >> >> But if it is really a ClassLoader issue, shouldn't you be able to >> simulate >> the situation outside a Servlet Container? I've run the attached >> test, and >> while I create a memory leak quickly if I keep a reference on the >> ClassLoader, I am not able to produce one without holding a ref on = it. >> >> Or is my test flawed? >> >> Guillaume >> >> ----- Original Message ----- >> From: "j=9Frgen h=9Aller [werk3AT]" <jue...@we...> >> To: <spr...@li...> >> Sent: Monday, May 24, 2004 4:41 AM >> Subject: Re: [Springframework-developer] Cleanup of context resources >> on >> webapp reload >> >> >> Just finished tests on Resin 2.1.11 - exactly same behavior as with >> Tomcat. >> This *is* a general class loader respectively garbage collection =20 >> issue, >> rather than a server-specific leak. >> >> Consequently, with the changes I've committed yesterday, there is the >> same >> significant benefit as with Tomcat. The only remaining issue are the >> "full >> object" constants like ClassFilter.TRUE. >> >> Juergen >> >> >> ________________________________ >> >> Von: spr...@li... im Auftrag >> von >> j=9Frgen h=9Aller [werk3AT] >> Gesendet: Mo 24.05.2004 09:20 >> An: spr...@li... >> Betreff: Re: [Springframework-developer] Cleanup of context resources >> on >> webapp reload >> >> >> >> FYI, I've tested both Tomcat 5.0.18 and 4.1.27 - same behavior with >> both. >> >> BTW, a couple of related posts from the Resin mailing list: >> >> http://www.caucho.com/support/resin-interest/0404/0117.html >> http://www.caucho.com/support/resin-interest/0111/0164.html >> <http://www.caucho.com/support/resin-interest/0111/0164.html> >> >> Juergen >> >> >> ________________________________ >> >> Von: spr...@li... im Auftrag >> von >> j=9Frgen h=9Aller [werk3AT] >> Gesendet: Mo 24.05.2004 08:42 >> An: spr...@li... >> Betreff: Re: [Springframework-developer] Cleanup of context resources >> on >> webapp reload >> >> >> >> I've told JProfiler to explicitly run garbage collection - a number = of >> times, actually - before I've had a look at the heap. So I'm sure = that >> those >> remaining objects were not garbage-collected, and probably would have >> stayed >> around in the VM forever... >> >> I'm aware that it seems odd, but this issue just seems to affect >> specific >> static fields: BeanWrapperImpl's defaultEditors did not cause a leak, >> but >> CachedIntrospectionResults' classCache did. Normal constants or = static >> logger fields didn't, but "full object" constants like >> ClassFilters.TRUE >> did. >> >> CachedIntrospectionResults' classCache uses the Class as key; the >> value, a >> CachedIntrospectionResult object, also refers to the key Class. >> Consequently, I had to use a WeakHashMap *with WeakReferences as >> values* to >> see proper garbage collection. Note that this static cache contains >> instances of its containing class as values. >> >> I've run all my tests x times to make sure that I could trust my = eyes. >> I've >> even undone the WeakReference changes again, and voila, there were = the >> leaks >> again. I'd be happy to learn more about how the garbage collector =20 >> works >> here... All I can state at this point of time is that the changes did >> cause >> an obvious difference in terms of resource leaks. >> >> Juergen >> >> >> ________________________________ >> >> Von: spr...@li... im Auftrag >> von >> Guillaume Poirier >> Gesendet: Mo 24.05.2004 05:17 >> An: spr...@li... >> Betreff: Re: [Springframework-developer] Cleanup of context resources >> on >> webapp reload >> >> >> >> The "resource leak" caused by a singleton when a webapp's classloader >> is >> thrown away is only temorary, the unused classes and the classloader >> will be >> eventually garbage collected and the resources will be freed. The =20= >> only >> thing that could prevent that is if there was a something in the >> server's >> classloader that still had a reference on an object or a class of the >> child >> classloader. >> >> Unless there's a bug in Tomcat or in the application code, I really >> can't >> see how the classes won't be eventually garbage collected when the = JVM >> needs >> memory. And anyway, if there was indeed a leak because >> SQLErrorCodesFactory >> is a singleton, why wouldn't there be one for each static fields such >> as >> constants? >> >> Are you sure that JProfiler does not disable garbage collecting in >> order to >> make its profiling? I know that in many of the JVMPI method calls = are >> done >> with garbage collecting off. I suspect the above to be the cause of >> the >> "resource leak", rather than any singleton that Spring might be = using. >> >> Guillaume >> >> ----- Original Message ----- >> From: "j=9Frgen h=9Aller [werk3AT]" <jue...@we...> >> To: <spr...@li...> >> Sent: Sunday, May 23, 2004 4:45 PM >> Subject: Re: [Springframework-developer] Cleanup of context resources >> on >> webapp reload >> >> >> I've just spent about 10 hours profiling Spring, using the Image >> Database >> and Petclinic samples. (BTW, I've used an evaluation version of >> JProfiler >> from ej-technologies - nice product!) >> >> Although I still don't completely understand the garbage collection >> behavior, I've figured out the following issues. Each of them simply >> prevents the respective classes from getting garbage collected on >> destruction of the class loader (e.g. on Tomcat web app shutdown). >> >> - A classic singleton with a class variable holding the object. I've >> reworked GlobalAdvisorAdapterRegistry and SQLErrorCodesFactory to = hold >> the >> respective singleton as a WeakReference. >> >> - A static cache. I've reworked CachedIntrospectionResults to use a >> WeakHashMap with WeakReferences as values. >> >> - A ThreadLocal with a default other than null. I've reworked >> TransactionSynchronizationManager to use null as default for the >> resource >> map, setting a HashMap there on demand, removing the entire HashMap >> when >> unbinding the last resource. >> >> - Constants that define a full object. We have a number of those, for >> example ClassFilters.TRUE and AdvisedSupport.EMPTY_TARGET_SOURCE. = I've >> tried >> for quite a while, but I haven't been able to figure out a way to >> define >> such constants such that they will be garbage collected. >> >> The latter programming style is not uncommon, so I really don't >> understand >> why it causes trouble with garbage collection. Hibernate uses a =20 >> similar >> style for its FlushMode, for example. >> >> In general, other frameworks like CGLIB, Hibernate, Velocity have = huge >> resource leaks on web app shutdown, while just the constants issue >> remains >> with Spring now. As long as those huge third-party leaks are not >> addressed, >> I'm not worried at all by the single remaining Spring issue. >> >> As I initially said, we shouldn't exaggerate the problem, as it >> basically >> just affects hot reloading of web apps - mainly a development feature >> anyway. We need to make that clear to users too, to avoid comments a =20= >> la >> "Spring is not usable for real apps because it leaks on hot >> redeployment". >> >> Please, everybody, give the current CVS head a sanity check tomorrow. >> There >> shouldn't be any issues: the test suite passes, the sample apps run >> properly. Still, I'd feel more comfortable if we make sure that no >> subtle >> side effects have been introduced. >> >> For this reason, I will delay release 1.0.2 till tomorrow night. >> >> Juergen >> >> >> ________________________________ >> >> Von: spr...@li... im Auftrag >> von >> j=9Frgen h=9Aller [werk3AT] >> Gesendet: Sa 22.05.2004 15:52 >> An: spr...@li... >> Betreff: Re: [Springframework-developer] Cleanup of context resources >> on >> webapp reload >> >> >> >> So basically, all static caches cause resource leaks when restarting = a >> Tomcat web app? I wonder why this happens... The class loader should >> completely dissolve all classes that it has loaded in its lifetime, >> including static caches. Or have I misunderstood something here? =20 >> Anyone >> having in-detail experience with handling such a scenario? >> >> Juergen >> >> >> ________________________________ >> >> Von: spr...@li... im Auftrag >> von >> Dmitriy Kopylenko >> Gesendet: Di 04.05.2004 18:02 >> An: spr...@li... >> Betreff: Re: [Springframework-developer] Cleanup of context resources >> on >> webapp reload >> >> >> >> Well, >> >> for instance SQLErrorCodesFactory is a singleton(GoF, not Spring) =20 >> which >> caches SQLErrorCodes internally in the Map with strong references. >> Again, I >> don't know if trying >> to use WeakHashMap there would do the trick... >> >> Dmitriy >> >> Tim Kettering wrote: >> >>> >>> I looked at it some more this morning, and basically what I did was >>> start up tomcat w/ the webapp in the profiler, then after it was = done >>> starting up I used tomcat's manager to stop the context. This = should >>> destroy all resources related to the context. Here is a list of >>> spring related stuff that still were in memory after the context was >>> closed. Other stuff was cleaned up just fine. >>> >>> org.springframework.beans.CachedIntrospectionResults >>> org.springframework.jdbc.support.SQLCodes >>> org.springframework.core.Constants >>> org.springframework.aop.framework.AdvisedSupport$1 >>> org.springframework.aop.framework.adapter.BeforeAdviceAdapter >>> = org.springframework.aop.framework.adapter.AfterReturningAdviceAdapter >>> org.springframework.jdbc.support.SQLErrorCodesFactory >>> = org.springframework.transaction.support.TransactionSynchronizationMan=20 >>> a >>> ge >>> r$1 >>> org.springframework.transaction.interceptor.RollbackRuleAttribute >>> org.springframework.aop.framework.adapter.ThrosAdviceAdapter >>> org.springframework.aop.Pointcut$1 >>> = org.springframework.aop.framework.adapter.GlobalAdvisorAdapterRegistr=20 >>> y >>> >>> On May 3, 2004, at 3:28 PM, Dmitriy Kopylenko wrote: >>> >>>> I'm just wondering, would the use of WeakHashMap in >>>> CachedIntrospectionResults help? >>>> >>>> Dmitriy. >>>> >>>> Tim Kettering wrote: >>>> >>>>> I posted this to the users list last week and did not receive any >>>>> reply on it, so I'm posting it again here on the developer list, =20= >>>>> in >>>>> hopes i could get an reply from someone here. I'm trying to >>>>> determine if its something I should be doing myself, or if hte >>>>> spring context should be cleaning up those resources by itself on >>>>> the .close() call. Further profiling shows that there are >>>>> duplicate instances of SQLError and hibernate proxy classes >>>>> hanging >>>>> around afterwards too. Other objects do get cleaned up = properly. >>>>> -------- >>>>> Hi everyone, >>>>> We're (meaning me) looking into some resource leaks that are >>>>> occuring when our webapp context gets reloaded. I found that >>>>> context.close() needs to be called on the destroy() method of >>>>> plugin we're using, and it works for a good majority of the >>>>> objects >>>>> we were seeing leaked, but there are some objects that I'm = unable >>>>> to make go away. Object in question is the: >>>>> org.springframework.beans.CachedIntrospectionResults >>>>> Whenever I reload the context - the profiler I'm using shows that = I >>>>> have essentially a duplicate group of those objects (same instance >>>>> count) as the original, and successive reloads will continue to >>>>> duplicate this. >>>>> The profiler also shows the final reference to those objects like >>>>> this: >>>>> 100% - 1008 bytes - 63 alloc. >>>>> = org.springframework.context.support.ClassPathXmlApplicationContext.=20 >>>>> < >>>>> in >>>>> it > >>>>> So basically I guess what I'm asking is for ideas or suggestions = on >>>>> how I could get those to clean up. This bean doesnt show up in =20= >>>>> the >>>>> Spring javadocs. And looking in CVS says its a package level =20 >>>>> bean, >>>>> not for application use, so I'm thinking that closing the context >>>>> should (in theory) clean this up? Thanks in advance. >>>>> -tim >>>>> ------------------------------------------------------- >>>>> This SF.Net email is sponsored by: Oracle 10g >>>>> Get certified on the hottest thing ever to hit the market... = Oracle >>>>> 10g. Take an Oracle 10g class now, and we'll give you the exam =20 >>>>> FREE. >>>>> http://ads.osdn.com/?ad_id=3D3149&alloc_id=3D8166&op=3Dclick >>>>> _______________________________________________ >>>>> Springframework-developer mailing list >>>>> Spr...@li... >>>>> https://lists.sourceforge.net/lists/listinfo/springframework- >>>>> developer >>>> >>>> >>>> >>>> >>>> ------------------------------------------------------- >>>> This SF.Net email is sponsored by: Oracle 10g >>>> Get certified on the hottest thing ever to hit the market... Oracle >>>> 10g. Take an Oracle 10g class now, and we'll give you the exam = FREE. >>>> http://ads.osdn.com/?ad_id=3D3149&alloc_id=3D8166&op=3Dclick >>>> _______________________________________________ >>>> Springframework-developer mailing list >>>> Spr...@li... >>>> https://lists.sourceforge.net/lists/listinfo/springframework- >>>> developer >>>> >>> >>> >>> >>> ------------------------------------------------------- >>> This SF.Net email is sponsored by: Oracle 10g >>> Get certified on the hottest thing ever to hit the market... Oracle >>> 10g. >>> Take an Oracle 10g class now, and we'll give you the exam FREE. >>> http://ads.osdn.com/?ad_id=3D3149&alloc_id=3D8166&op=3Dclick >>> _______________________________________________ >>> Springframework-developer mailing list >>> Spr...@li... >>> https://lists.sourceforge.net/lists/listinfo/springframework-=20 >>> developer >> >> >> >> ------------------------------------------------------- >> This SF.Net email is sponsored by: Oracle 10g >> Get certified on the hottest thing ever to hit the market... Oracle >> 10g. >> Take an Oracle 10g class now, and we'll give you the exam FREE. >> http://ads.osdn.com/?ad_id=3D3149&alloc_id=3D8166&op=3Dclick >> _______________________________________________ >> Springframework-developer mailing list >> Spr...@li... >> = https://lists.sourceforge.net/lists/listinfo/springframework-developer >> >> >> >> >> ------------------------------------------------------- >> This SF.Net email is sponsored by: Oracle 10g >> Get certified on the hottest thing ever to hit the market... Oracle >> 10g. >> Take an Oracle 10g class now, and we'll give you the exam FREE. >> http://ads.osdn.com/?ad_id149&alloc_id=FA66&op=3Dick >> _______________________________________________ >> Springframework-developer mailing list >> Spr...@li... >> = https://lists.sourceforge.net/lists/listinfo/springframework-developer >> >> >> >> >> ------------------------------------------------------- >> This SF.Net email is sponsored by: Oracle 10g >> Get certified on the hottest thing ever to hit the market... Oracle >> 10g. >> Take an Oracle 10g class now, and we'll give you the exam FREE. >> http://ads.osdn.com/?ad_id149&alloc_id=FA66&op=3Dick >> _______________________________________________ >> Springframework-developer mailing list >> Spr...@li... >> = https://lists.sourceforge.net/lists/listinfo/springframework-developer >> >> >> >> >> ------------------------------------------------------- >> This SF.Net email is sponsored by: Oracle 10g >> Get certified on the hottest thing ever to hit the market... Oracle >> 10g. >> Take an Oracle 10g class now, and we'll give you the exam FREE. >> http://ads.osdn.com/?ad_id=3D3149&alloc_id=3D8166&op=3Dclick >> _______________________________________________ >> Springframework-developer mailing list >> Spr...@li... >> = https://lists.sourceforge.net/lists/listinfo/springframework-developer >> >> >> >> >> ------------------------------------------------------- >> This SF.Net email is sponsored by: Oracle 10g >> Get certified on the hottest thing ever to hit the market... Oracle >> 10g. >> Take an Oracle 10g class now, and we'll give you the exam FREE. >> http://ads.osdn.com/?ad_id149&alloc_id=FA66&op=3Dick >> _______________________________________________ >> Springframework-developer mailing list >> Spr...@li... >> = https://lists.sourceforge.net/lists/listinfo/springframework-developer >> >> >> >> >> ------------------------------------------------------- >> This SF.Net email is sponsored by: Oracle 10g >> Get certified on the hottest thing ever to hit the market... Oracle >> 10g. >> Take an Oracle 10g class now, and we'll give you the exam FREE. >> http://ads.osdn.com/?ad_id149&alloc_id=FA66&op=3Dick >> _______________________________________________ >> Springframework-developer mailing list >> Spr...@li... >> = https://lists.sourceforge.net/lists/listinfo/springframework-developer >> >> >> >> >> ------------------------------------------------------- >> This SF.Net email is sponsored by: Oracle 10g >> Get certified on the hottest thing ever to hit the market... Oracle >> 10g. >> Take an Oracle 10g class now, and we'll give you the exam FREE. >> http://ads.osdn.com/?ad_id149&alloc_id=FA66&op=3Dick >> _______________________________________________ >> Springframework-developer mailing list >> Spr...@li... >> = https://lists.sourceforge.net/lists/listinfo/springframework-developer >> <classloader.tar.gz> > > > ------------------------------------------------------- > This SF.Net email is sponsored by: Oracle 10g > Get certified on the hottest thing ever to hit the market... Oracle =20= > 10g. > Take an Oracle 10g class now, and we'll give you the exam FREE. > http://ads.osdn.com/?ad_id149&alloc_id=AA66&op=3Dick > _______________________________________________ > Springframework-developer mailing list > Spr...@li... > https://lists.sourceforge.net/lists/listinfo/springframework-developer > > > ------------------------------------------------------- > This SF.Net email is sponsored by: Oracle 10g > Get certified on the hottest thing ever to hit the market... Oracle =20= > 10g. > Take an Oracle 10g class now, and we'll give you the exam FREE. > http://ads.osdn.com/?ad_id149&alloc_id=AA66&op=3Dclick > _______________________________________________ > Springframework-developer mailing list > Spr...@li... > https://lists.sourceforge.net/lists/listinfo/springframework-developer > |
