Menu
▾
▴
springframework-developer
|
From: <jue...@we...> - 2004-10-19 17:08:36
|
BTW, I've polished Cglib2AopProxy's source code quite a bit: for =
example, it logs at debug level now (rather than info), and uses =
formatting consistent with the rest of Spring's sources.
I've actually already done this polishing a second time: The earlier one =
got lost. It shows up in CVS, but then seems to have got overwritten by =
a subsequent commit. There was also a JDK 1.4 dependency fix in there =
(Boolean.valueOf with a boolean argument), which got lost too.
So please, when you merge before a commit, double-check that you're not =
losing updates that someone else applied before you! It wasn't a big =
issue this time, but what if we don't notice it next time...
Juergen
-----Original Message-----
From: j=FCrgen h=F6ller [werk3AT]=20
Sent: Tuesday, October 19, 2004 6:35 PM
To: spr...@li...
Subject: CGLIB memory usage within class loader
I've just been made aware of the following by a werk3 colleague: When =
repeatedly creating CGLIB proxies for the same target class but for =
different advices - within the same class loader -, you'll create new =
proxied classes all the time, which won't get removed for the lifetime =
of the classes.
The generated classes themselves are not the problem here. However, each =
of those proxy classes seems to hold strong references to its advices =
and target object, through holding the ProxyCallbackFilter instance that =
was passed into the Enhancer on proxy creation. Note that the proxy =
*class* holds that reference, not the proxy *instance*.
The effect is that you can easily run out of memory if your advices or =
target objects occupy a large amount of memory, as none of those objects =
will get garbage collected as long as the CGLIB-generated proxy class is =
still hanging around in the class loader. This does *not* happen at all =
with JDK dynamic proxies.
Of course, such excessive proxy creation is not the usual case in a =
production application. Web app restart is not a problem either, as the =
class loader will be shut down (releasing the CGLIB-generated proxy =
classes). The usual scenario where this memory leak becomes a problem is =
test suites, with application contexts getting created per test method.
The following code snippet reproduces the issue:
while (true) {
ProxyFactory proxyFactory =3D new ProxyFactory();
proxyFactory.setTarget(new TestBean() {
private byte[] field =3D new byte[1000000];
});
proxyFactory.addAdvice(new DebugInterceptor());
proxyFactory.setProxyTargetClass(true);
TestBean tb =3D (TestBean) proxyFactory.getProxy();
Thread.sleep(100);
}
Depending on the max memory available to the VM, you'll run into an =
OutOfMemoryError sooner or later, as the target TestBean instance with =
its byte array of size 1000000 never gets garbage collected. Note that =
this does *not* happen when removing the addAdvice call, as we're just =
generating a single CGLIB proxy class then.
Is this behavior unavoidable? What do we recommend for such testing =
scenarios then? In particular for integration tests with Hibernate, a 5 =
MB leak per context creation is not really acceptable. Of course it's =
preferable to share the application context as far as possible, but =
there are still valid use cases for repeated creation...
Juergen
|
|
From: <jue...@we...> - 2004-10-19 18:02:10
|
I've not committed this yet, as I still need to merge changes in that =
were committed this afternoon ;-)
Juergen
-----Original Message-----
From: j=FCrgen h=F6ller [werk3AT]=20
Sent: Tuesday, October 19, 2004 7:11 PM
To: 'spr...@li...'
Subject: RE: CGLIB memory usage within class loader
BTW, I've polished Cglib2AopProxy's source code quite a bit: for =
example, it logs at debug level now (rather than info), and uses =
formatting consistent with the rest of Spring's sources.
I've actually already done this polishing a second time: The earlier one =
got lost. It shows up in CVS, but then seems to have got overwritten by =
a subsequent commit. There was also a JDK 1.4 dependency fix in there =
(Boolean.valueOf with a boolean argument), which got lost too.
So please, when you merge before a commit, double-check that you're not =
losing updates that someone else applied before you! It wasn't a big =
issue this time, but what if we don't notice it next time...
Juergen
-----Original Message-----
From: j=FCrgen h=F6ller [werk3AT]=20
Sent: Tuesday, October 19, 2004 6:35 PM
To: spr...@li...
Subject: CGLIB memory usage within class loader
I've just been made aware of the following by a werk3 colleague: When =
repeatedly creating CGLIB proxies for the same target class but for =
different advices - within the same class loader -, you'll create new =
proxied classes all the time, which won't get removed for the lifetime =
of the classes.
The generated classes themselves are not the problem here. However, each =
of those proxy classes seems to hold strong references to its advices =
and target object, through holding the ProxyCallbackFilter instance that =
was passed into the Enhancer on proxy creation. Note that the proxy =
*class* holds that reference, not the proxy *instance*.
The effect is that you can easily run out of memory if your advices or =
target objects occupy a large amount of memory, as none of those objects =
will get garbage collected as long as the CGLIB-generated proxy class is =
still hanging around in the class loader. This does *not* happen at all =
with JDK dynamic proxies.
Of course, such excessive proxy creation is not the usual case in a =
production application. Web app restart is not a problem either, as the =
class loader will be shut down (releasing the CGLIB-generated proxy =
classes). The usual scenario where this memory leak becomes a problem is =
test suites, with application contexts getting created per test method.
The following code snippet reproduces the issue:
while (true) {
ProxyFactory proxyFactory =3D new ProxyFactory();
proxyFactory.setTarget(new TestBean() {
private byte[] field =3D new byte[1000000];
});
proxyFactory.addAdvice(new DebugInterceptor());
proxyFactory.setProxyTargetClass(true);
TestBean tb =3D (TestBean) proxyFactory.getProxy();
Thread.sleep(100);
}
Depending on the max memory available to the VM, you'll run into an =
OutOfMemoryError sooner or later, as the target TestBean instance with =
its byte array of size 1000000 never gets garbage collected. Note that =
this does *not* happen when removing the addAdvice call, as we're just =
generating a single CGLIB proxy class then.
Is this behavior unavoidable? What do we recommend for such testing =
scenarios then? In particular for integration tests with Hibernate, a 5 =
MB leak per context creation is not really acceptable. Of course it's =
preferable to share the application context as far as possible, but =
there are still valid use cases for repeated creation...
Juergen
|
|
From: Rob R. <rob...@ur...> - 2004-10-19 18:05:53
|
Juergen - we found ourselves having the same problem with
creating contexts per test method, though we weren't using CGLIB
proxies through Spring. CGLIB was still involved because of
Hibernate, and we found that the JVM for the integration test
would run out of memory after about 20-25 tests that each
created a new context and then did some operations involving
Hibernate. We ended up extending TestCase and TestSuite so that
a single context is created and shared among test methods; this
helped us get around the problem.
We also get this problem when restarting just the webapp in
Tomcat. I know this was discussed before, and I thought we
shouldn't be affected because we only use JDK proxies, but it
still occurs for us.
Is there any chance that this will eventually be fixed in CGLIB?
We can get by with one context for all test methods, but as you
state, it would be nice for it not to be a problem when each
test method does need to create its own context.
Rob
---- On Tue, 19 Oct 2004,
=?iso-8859-1?Q?j=FCrgen_h=F6ller_=5Bwerk3AT=5D?=
(jue...@we...) wrote:
> I've just been made aware of the following by a werk3
colleague: When repeatedly creating CGLIB
> proxies for the same target class but for different advices -
within the same class loader -, you'll
> create new proxied classes all the time, which won't get
removed for the lifetime of the classes.
>
> The generated classes themselves are not the problem here.
However, each of those proxy classes seems
> to hold strong references to its advices and target object,
through holding the ProxyCallbackFilter
> instance that was passed into the Enhancer on proxy creation.
Note that the proxy *class* holds that
> reference, not the proxy *instance*.
>
> The effect is that you can easily run out of memory if your
advices or target objects occupy a large
> amount of memory, as none of those objects will get garbage
collected as long as the CGLIB-generated
> proxy class is still hanging around in the class loader. This
does *not* happen at all with JDK
> dynamic proxies.
>
> Of course, such excessive proxy creation is not the usual case
in a production application. Web app
> restart is not a problem either, as the class loader will be
shut down (releasing the CGLIB-generated
> proxy classes). The usual scenario where this memory leak
becomes a problem is test suites, with
> application contexts getting created per test method.
>
> The following code snippet reproduces the issue:
>
> while (true) {
> ProxyFactory proxyFactory = new ProxyFactory();
> proxyFactory.setTarget(new TestBean() {
> private byte[] field = new byte[1000000];
> });
> proxyFactory.addAdvice(new DebugInterceptor());
> proxyFactory.setProxyTargetClass(true);
> TestBean tb = (TestBean) proxyFactory.getProxy();
> Thread.sleep(100);
> }
>
> Depending on the max memory available to the VM, you'll run
into an OutOfMemoryError sooner or later,
> as the target TestBean instance with its byte array of size
1000000 never gets garbage collected.
> Note that this does *not* happen when removing the addAdvice
call, as we're just generating a single
> CGLIB proxy class then.
>
> Is this behavior unavoidable? What do we recommend for such
testing scenarios then? In particular for
> integration tests with Hibernate, a 5 MB leak per context
creation is not really acceptable. Of
> course it's preferable to share the application context as far
as possible, but there are still valid
> use cases for repeated creation...
>
> Juergen
>
>
> -------------------------------------------------------
> This SF.net email is sponsored by: IT Product Guide on
ITManagersJournal
> Use IT products in your business? Tell us what you think of
them. Give us
> Your Opinions, Get Free ThinkGeek Gift Certificates! Click to
find out more
> http://productguide.itmanagersjournal.com/guidepromo.tmpl
> _______________________________________________
> Springframework-developer mailing list
> Spr...@li...
>
https://lists.sourceforge.net/lists/listinfo/springframework-developer
>
>
>
|
|
From: Chris N. <ch...@si...> - 2004-10-19 19:32:50
|
Rob Rudin wrote: > Is there any chance that this will eventually be fixed in CGLIB? > We can get by with one context for all test methods, but as you > state, it would be nice for it not to be a problem when each > test method does need to create its own context. CGLIB keeps an internal WeakReference-based per-ClassLoader cache of all generated classes, and tries to use it whenever possible. But if anything about the requested proxy changes, it does the safe thing and generates a new class. As Rob Harrop alluded to, for CGLIB to realize that one proxy is the same as another, among other things the Callback and CallbackFilter objects must implement equals/hashCode properly. Hibernate may have some issues in this regard, so you may want to file a bug with them. Chris |
|
From: Rob H. <ro...@ca...> - 2004-10-19 20:00:25
|
To add to what Chris is saying, the best way to keep memory usage is to implement equals() on your interceptors so that proxy classes can be reused as much as possible. I am going to look into a solution that will use weak references from the proxy class but strong references from proxy instances. This way once a proxy class is no longer used the advice chain and target can still be GCd. In the rare case when the advice chain is collected but the proxy class could still apply we can instruct CGLIB to create a new proxy class anyway - this will effectively remove this problem in all but the MOST extreme cases. I will work on this as soon as I get back to work. I need access to a profiler to make this solution work properly and I don't have one at home. Rob Chris Nokleberg wrote: >Rob Rudin wrote: > > >>Is there any chance that this will eventually be fixed in CGLIB? >>We can get by with one context for all test methods, but as you >>state, it would be nice for it not to be a problem when each >>test method does need to create its own context. >> >> > >CGLIB keeps an internal WeakReference-based per-ClassLoader cache of all >generated classes, and tries to use it whenever possible. But if anything >about the requested proxy changes, it does the safe thing and generates a >new class. As Rob Harrop alluded to, for CGLIB to realize that one proxy is >the same as another, among other things the Callback and CallbackFilter >objects must implement equals/hashCode properly. Hibernate may have some >issues in this regard, so you may want to file a bug with them. > >Chris > > > > >------------------------------------------------------- >This SF.net email is sponsored by: IT Product Guide on ITManagersJournal >Use IT products in your business? Tell us what you think of them. Give us >Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more >http://productguide.itmanagersjournal.com/guidepromo.tmpl >_______________________________________________ >Springframework-developer mailing list >Spr...@li... >https://lists.sourceforge.net/lists/listinfo/springframework-developer > > > > |
|
From: <tho...@tr...> - 2004-10-19 21:11:46
|
Rob, Don't know if you've seen this, but there is a free version of JProbe available for download. It's limited to 5 users, and you have to download a new version when it expires in six months. Good enough for home use :) http://www.quest.com/jprobe/profiler_freeware.asp Thomas Quoting Rob Harrop <ro...@ca...>: > To add to what Chris is saying, the best way to keep memory usage is to > implement equals() on your interceptors so that proxy classes can be > reused as much as possible. I am going to look into a solution that will > use weak references from the proxy class but strong references from > proxy instances. This way once a proxy class is no longer used the > advice chain and target can still be GCd. In the rare case when the > advice chain is collected but the proxy class could still apply we can > instruct CGLIB to create a new proxy class anyway - this will > effectively remove this problem in all but the MOST extreme cases. > > I will work on this as soon as I get back to work. I need access to a > profiler to make this solution work properly and I don't have one at home. > > Rob > > Chris Nokleberg wrote: > > >Rob Rudin wrote: > > > > > >>Is there any chance that this will eventually be fixed in CGLIB? > >>We can get by with one context for all test methods, but as you > >>state, it would be nice for it not to be a problem when each > >>test method does need to create its own context. > >> > >> > > > >CGLIB keeps an internal WeakReference-based per-ClassLoader cache of all > >generated classes, and tries to use it whenever possible. But if anything > >about the requested proxy changes, it does the safe thing and generates a > >new class. As Rob Harrop alluded to, for CGLIB to realize that one proxy is > >the same as another, among other things the Callback and CallbackFilter > >objects must implement equals/hashCode properly. Hibernate may have some > >issues in this regard, so you may want to file a bug with them. > > > >Chris > > > > > > > > > >------------------------------------------------------- > >This SF.net email is sponsored by: IT Product Guide on ITManagersJournal > >Use IT products in your business? Tell us what you think of them. Give us > >Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more > >http://productguide.itmanagersjournal.com/guidepromo.tmpl > >_______________________________________________ > >Springframework-developer mailing list > >Spr...@li... > >https://lists.sourceforge.net/lists/listinfo/springframework-developer > > > > > > > > > > > ------------------------------------------------------- > This SF.net email is sponsored by: IT Product Guide on ITManagersJournal > Use IT products in your business? Tell us what you think of them. Give us > Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more > http://productguide.itmanagersjournal.com/guidepromo.tmpl > _______________________________________________ > Springframework-developer mailing list > Spr...@li... > https://lists.sourceforge.net/lists/listinfo/springframework-developer > |
|
From: Seth L. <set...@gm...> - 2004-10-19 20:06:11
|
On Tue, 19 Oct 2004 14:05:47 -0400, Rob Rudin <rob...@ur...> wrote: > Juergen - we found ourselves having the same problem with > creating contexts per test method, though we weren't using CGLIB > proxies through Spring. CGLIB was still involved because of > Hibernate, and we found that the JVM for the integration test > would run out of memory after about 20-25 tests that each > created a new context and then did some operations involving > Hibernate. We ended up extending TestCase and TestSuite so that > a single context is created and shared among test methods; this > helped us get around the problem. > > We also get this problem when restarting just the webapp in > Tomcat. I know this was discussed before, and I thought we > shouldn't be affected because we only use JDK proxies, but it > still occurs for us. > > Is there any chance that this will eventually be fixed in CGLIB? > We can get by with one context for all test methods, but as you > state, it would be nice for it not to be a problem when each > test method does need to create its own context. We also have experienced this problem. We had to create one single ApplicationContext for all tests. Otherwise... BOOM... OOM. And we still see OOM w/ redeployments in Tomcat. The more I look into it, the more it points to a CGLIB problem from Hibernate. Only when I include Hibernate into my deployments or somehow use CGLIB do I get OOM after a few redeployments or test runs. Seth |
|
From: Rob H. <ro...@ca...> - 2004-10-19 20:52:30
|
I'll look at the Hibernate code as well it may be that proxy classes aren't getting reused as often as possible. Rob Seth Ladd wrote: >On Tue, 19 Oct 2004 14:05:47 -0400, Rob Rudin <rob...@ur...> wrote: > > >>Juergen - we found ourselves having the same problem with >>creating contexts per test method, though we weren't using CGLIB >>proxies through Spring. CGLIB was still involved because of >>Hibernate, and we found that the JVM for the integration test >>would run out of memory after about 20-25 tests that each >>created a new context and then did some operations involving >>Hibernate. We ended up extending TestCase and TestSuite so that >>a single context is created and shared among test methods; this >>helped us get around the problem. >> >>We also get this problem when restarting just the webapp in >>Tomcat. I know this was discussed before, and I thought we >>shouldn't be affected because we only use JDK proxies, but it >>still occurs for us. >> >>Is there any chance that this will eventually be fixed in CGLIB? >>We can get by with one context for all test methods, but as you >>state, it would be nice for it not to be a problem when each >>test method does need to create its own context. >> >> > >We also have experienced this problem. We had to create one single >ApplicationContext for all tests. Otherwise... BOOM... OOM. > >And we still see OOM w/ redeployments in Tomcat. The more I look into >it, the more it points to a CGLIB problem from Hibernate. Only when I >include Hibernate into my deployments or somehow use CGLIB do I get >OOM after a few redeployments or test runs. > >Seth > > >------------------------------------------------------- >This SF.net email is sponsored by: IT Product Guide on ITManagersJournal >Use IT products in your business? Tell us what you think of them. Give us >Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more >http://productguide.itmanagersjournal.com/guidepromo.tmpl >_______________________________________________ >Springframework-developer mailing list >Spr...@li... >https://lists.sourceforge.net/lists/listinfo/springframework-developer > > > > |
|
From: Guillaume P. <gpo...@gl...> - 2004-10-19 22:41:49
|
> And we still see OOM w/ redeployments in Tomcat. The more I look into > it, the more it points to a CGLIB problem from Hibernate. Only when I > include Hibernate into my deployments or somehow use CGLIB do I get > OOM after a few redeployments or test runs. If you use Hibernate, then there's at least something else that will prevent the old webapp's classloader to be collected when you application is reloaded. That's Dom4j, it uses ThreadLocal stuff without ever removing it, which prevent the ClassLoader to be collected for as long as the pooled threads are not destroyed. However, that particular problem doesn't cause any leak if the webapp's ClassLoader isn't thrown away. Guillaume ----- Original Message ----- From: "Seth Ladd" <set...@gm...> To: <spr...@li...> Sent: Tuesday, October 19, 2004 4:06 PM Subject: Re: [Springframework-developer] CGLIB memory usage within class loader > On Tue, 19 Oct 2004 14:05:47 -0400, Rob Rudin <rob...@ur...> wrote: >> Juergen - we found ourselves having the same problem with >> creating contexts per test method, though we weren't using CGLIB >> proxies through Spring. CGLIB was still involved because of >> Hibernate, and we found that the JVM for the integration test >> would run out of memory after about 20-25 tests that each >> created a new context and then did some operations involving >> Hibernate. We ended up extending TestCase and TestSuite so that >> a single context is created and shared among test methods; this >> helped us get around the problem. >> >> We also get this problem when restarting just the webapp in >> Tomcat. I know this was discussed before, and I thought we >> shouldn't be affected because we only use JDK proxies, but it >> still occurs for us. >> >> Is there any chance that this will eventually be fixed in CGLIB? >> We can get by with one context for all test methods, but as you >> state, it would be nice for it not to be a problem when each >> test method does need to create its own context. > > We also have experienced this problem. We had to create one single > ApplicationContext for all tests. Otherwise... BOOM... OOM. > > And we still see OOM w/ redeployments in Tomcat. The more I look into > it, the more it points to a CGLIB problem from Hibernate. Only when I > include Hibernate into my deployments or somehow use CGLIB do I get > OOM after a few redeployments or test runs. > > Seth > > > ------------------------------------------------------- > This SF.net email is sponsored by: IT Product Guide on ITManagersJournal > Use IT products in your business? Tell us what you think of them. Give us > Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out > more > http://productguide.itmanagersjournal.com/guidepromo.tmpl > _______________________________________________ > Springframework-developer mailing list > Spr...@li... > https://lists.sourceforge.net/lists/listinfo/springframework-developer |
|
From: Seth L. <set...@gm...> - 2004-10-19 23:58:22
|
On Tue, 19 Oct 2004 18:41:48 -0400, Guillaume Poirier <gpo...@gl...> wrote: > > And we still see OOM w/ redeployments in Tomcat. The more I look into > > it, the more it points to a CGLIB problem from Hibernate. Only when I > > include Hibernate into my deployments or somehow use CGLIB do I get > > OOM after a few redeployments or test runs. > > If you use Hibernate, then there's at least something else that will prevent > the old webapp's classloader to be collected when you application is > reloaded. That's Dom4j, it uses ThreadLocal stuff without ever removing it, > which prevent the ClassLoader to be collected for as long as the pooled > threads are not destroyed. > > However, that particular problem doesn't cause any leak if the webapp's > ClassLoader isn't thrown away. How does it not create a leak if the classloader is not going away? The real cause of the OOM exceptions is that the WebappClassLoader instances never go away. Thanks! Seth |
|
From: Guillaume P. <gpo...@gl...> - 2004-10-20 02:10:23
|
> How does it not create a leak if the classloader is not going away? > The real cause of the OOM exceptions is that the WebappClassLoader > instances never go away. I meant if the container (e.g. Tomcat) doesn't throw the ClassLoader away, i.e. if it's still in use... Of course, if the webapp is reloaded and the old ClassLoader isn't collected, that's a leak. What I was saying is the problem with Dom4j cause a leak when you reload the webapp, but cause no leak if you never use hot-reload. (While the CGLIB leak apparently does the opposite) Guillaume ----- Original Message ----- From: "Seth Ladd" <set...@gm...> To: <spr...@li...> Sent: Tuesday, October 19, 2004 7:58 PM Subject: Re: [Springframework-developer] CGLIB memory usage within class loader > On Tue, 19 Oct 2004 18:41:48 -0400, Guillaume Poirier > <gpo...@gl...> wrote: >> > And we still see OOM w/ redeployments in Tomcat. The more I look into >> > it, the more it points to a CGLIB problem from Hibernate. Only when I >> > include Hibernate into my deployments or somehow use CGLIB do I get >> > OOM after a few redeployments or test runs. >> >> If you use Hibernate, then there's at least something else that will >> prevent >> the old webapp's classloader to be collected when you application is >> reloaded. That's Dom4j, it uses ThreadLocal stuff without ever removing >> it, >> which prevent the ClassLoader to be collected for as long as the pooled >> threads are not destroyed. >> >> However, that particular problem doesn't cause any leak if the webapp's >> ClassLoader isn't thrown away. > > How does it not create a leak if the classloader is not going away? > The real cause of the OOM exceptions is that the WebappClassLoader > instances never go away. > > Thanks! > Seth > > > ------------------------------------------------------- > This SF.net email is sponsored by: IT Product Guide on ITManagersJournal > Use IT products in your business? Tell us what you think of them. Give us > Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out > more > http://productguide.itmanagersjournal.com/guidepromo.tmpl > _______________________________________________ > Springframework-developer mailing list > Spr...@li... > https://lists.sourceforge.net/lists/listinfo/springframework-developer |
|
From: Seth L. <set...@gm...> - 2004-10-20 18:26:18
|
On Tue, 19 Oct 2004 22:10:20 -0400, Guillaume Poirier <gpo...@gl...> wrote: > > How does it not create a leak if the classloader is not going away? > > The real cause of the OOM exceptions is that the WebappClassLoader > > instances never go away. > > I meant if the container (e.g. Tomcat) doesn't throw the ClassLoader away, > i.e. if it's still in use... > Of course, if the webapp is reloaded and the old ClassLoader isn't > collected, that's a leak. > > What I was saying is the problem with Dom4j cause a leak when you reload the > webapp, > but cause no leak if you never use hot-reload. (While the CGLIB leak > apparently does the opposite) Ahh... that's what I thought. Thanks for the clarification! I only wish A) there was a way to explicitly blow away the classloader or B) have hibernate be redeployable. One option is to place all the Hibernate related classes inside Tomcat's shared Classloader. This might reduce the amount of times I need to restart. Seth |
|
From: Guillaume P. <gpo...@gl...> - 2004-10-21 22:25:22
|
> I only wish A) there was a way to explicitly blow away the classloader > or B) have hibernate be redeployable. One option is to place all the > Hibernate related classes inside Tomcat's shared Classloader. This > might reduce the amount of times I need to restart. Well, if you care enough, I'm pretty sure you could create a ContextListener that would cleanup when the context is destroyed. It would invole reflection to hack in private fields of the Threads though, so the solution might fail in different JVM, and wouldn't work well with SecurityManagers. Basically, what you would have to do is to get the list of all the system's thread (through ThreadGroup), and for each of them, iterate the "threadLocals" map, and the "inheritableThreadLocals", to cleanup any instance for which the ClassLoader is the current webapp or of child of it. You might also have to cleanup a few other things though, such as if you have any JDBC drivers in your webapp's ClassLoader, they need to be explicitly unregistered, otherwise they will not be garbage collected. I create a prototype of a ContextListener that does that if you want. Guillaume ----- Original Message ----- From: "Seth Ladd" <set...@gm...> To: <spr...@li...> Sent: Wednesday, October 20, 2004 2:26 PM Subject: Re: [Springframework-developer] CGLIB memory usage within class loader > On Tue, 19 Oct 2004 22:10:20 -0400, Guillaume Poirier > <gpo...@gl...> wrote: >> > How does it not create a leak if the classloader is not going away? >> > The real cause of the OOM exceptions is that the WebappClassLoader >> > instances never go away. >> >> I meant if the container (e.g. Tomcat) doesn't throw the ClassLoader >> away, >> i.e. if it's still in use... >> Of course, if the webapp is reloaded and the old ClassLoader isn't >> collected, that's a leak. >> >> What I was saying is the problem with Dom4j cause a leak when you reload >> the >> webapp, >> but cause no leak if you never use hot-reload. (While the CGLIB leak >> apparently does the opposite) > > Ahh... that's what I thought. Thanks for the clarification! > > I only wish A) there was a way to explicitly blow away the classloader > or B) have hibernate be redeployable. One option is to place all the > Hibernate related classes inside Tomcat's shared Classloader. This > might reduce the amount of times I need to restart. > > Seth > > > ------------------------------------------------------- > This SF.net email is sponsored by: IT Product Guide on ITManagersJournal > Use IT products in your business? Tell us what you think of them. Give us > Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out > more > http://productguide.itmanagersjournal.com/guidepromo.tmpl > _______________________________________________ > Springframework-developer mailing list > Spr...@li... > https://lists.sourceforge.net/lists/listinfo/springframework-developer |
|
From: Guillaume P. <gpo...@gl...> - 2004-10-22 04:50:26
Attachments:
cleanup.tar.gz
|
I just created a ContextListener that cleanup ThreadLocal stuff when a webapp is shut down. It's attached to this email if you wish to use it. However, there's one issue I'm not sure how to deal with, that's concurrency. The ThreadLocal$ThreadLocalMap is accessed without synchronization by the ThreadLocal class, because there's only one per thread, and it's only manipulated by the owning thread. Since my cleanup hack might be executed from another thread, there's a risk for concurrent access. When reading, the problem isn't too bad. The worse case is stale data, which I don't even think could lead to unsuccessful cleanup, and certainly it wouldn't corrupt other webapps. However, once a value for a ThreadLocal is found that needs to be cleaned, then a concurrent write is needed. There, I cannot really see how it's possible to handle concurrency without risk for data corruption. The only way around this I can see is to suspend the thread using JPDA. Of course, if it's for development and you're the only user, then there's no real risk of concurrency problem anyway. Guillaume ----- Original Message ----- From: "Guillaume Poirier" <gpo...@gl...> To: <spr...@li...> Sent: Thursday, October 21, 2004 6:25 PM Subject: Re: [Springframework-developer] CGLIB memory usage within class loader >> I only wish A) there was a way to explicitly blow away the classloader >> or B) have hibernate be redeployable. One option is to place all the >> Hibernate related classes inside Tomcat's shared Classloader. This >> might reduce the amount of times I need to restart. > > Well, if you care enough, I'm pretty sure you could create a > ContextListener > that would cleanup when the context is destroyed. It would invole > reflection > to hack in private fields of the Threads though, so the solution might > fail > in different JVM, and wouldn't work well with SecurityManagers. Basically, > what you would have to do is to get the list of all the system's thread > (through > ThreadGroup), and for each of them, iterate the "threadLocals" map, and > the "inheritableThreadLocals", to cleanup any instance for which the > ClassLoader is the current webapp or of child of it. > > You might also have to cleanup a few other things though, such as if you > have > any JDBC drivers in your webapp's ClassLoader, they need to be explicitly > unregistered, otherwise they will not be garbage collected. > > I create a prototype of a ContextListener that does that if you want. > > Guillaume > > ----- Original Message ----- > From: "Seth Ladd" <set...@gm...> > To: <spr...@li...> > Sent: Wednesday, October 20, 2004 2:26 PM > Subject: Re: [Springframework-developer] CGLIB memory usage within class > loader > > >> On Tue, 19 Oct 2004 22:10:20 -0400, Guillaume Poirier >> <gpo...@gl...> wrote: >>> > How does it not create a leak if the classloader is not going away? >>> > The real cause of the OOM exceptions is that the WebappClassLoader >>> > instances never go away. >>> >>> I meant if the container (e.g. Tomcat) doesn't throw the ClassLoader >>> away, >>> i.e. if it's still in use... >>> Of course, if the webapp is reloaded and the old ClassLoader isn't >>> collected, that's a leak. >>> >>> What I was saying is the problem with Dom4j cause a leak when you reload >>> the >>> webapp, >>> but cause no leak if you never use hot-reload. (While the CGLIB leak >>> apparently does the opposite) >> >> Ahh... that's what I thought. Thanks for the clarification! >> >> I only wish A) there was a way to explicitly blow away the classloader >> or B) have hibernate be redeployable. One option is to place all the >> Hibernate related classes inside Tomcat's shared Classloader. This >> might reduce the amount of times I need to restart. >> >> Seth >> >> >> ------------------------------------------------------- >> This SF.net email is sponsored by: IT Product Guide on ITManagersJournal >> Use IT products in your business? Tell us what you think of them. Give us >> Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out >> more >> http://productguide.itmanagersjournal.com/guidepromo.tmpl >> _______________________________________________ >> Springframework-developer mailing list >> Spr...@li... >> https://lists.sourceforge.net/lists/listinfo/springframework-developer > > > > ------------------------------------------------------- > This SF.net email is sponsored by: IT Product Guide on ITManagersJournal > Use IT products in your business? Tell us what you think of them. Give us > Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out > more > http://productguide.itmanagersjournal.com/guidepromo.tmpl > _______________________________________________ > Springframework-developer mailing list > Spr...@li... > https://lists.sourceforge.net/lists/listinfo/springframework-developer |
|
From: Patrick B. <pbu...@gm...> - 2004-10-22 16:20:12
|
I just want to get this straight, since we run into this memory leak a lot here when redeploying our app a lot in development: The leak is caused by ThreadLocals that aren't cleaned up in Dom4j? If so, I wonder what could be done in the Dom4j library to help this situation, like maybe an API call that tells it to "clean up"? Thanks, Patrick On Fri, 22 Oct 2004 00:50:33 -0400, Guillaume Poirier <gpo...@gl...> wrote: > ----- Original Message ----- > From: "Guillaume Poirier" <gpo...@gl...> > To: <spr...@li...> > Sent: Thursday, October 21, 2004 6:25 PM > Subject: Re: [Springframework-developer] CGLIB memory usage within class > loader > > >> I only wish A) there was a way to explicitly blow away the classloader > >> or B) have hibernate be redeployable. One option is to place all the > >> Hibernate related classes inside Tomcat's shared Classloader. This > >> might reduce the amount of times I need to restart. > > > > Well, if you care enough, I'm pretty sure you could create a > > ContextListener > > that would cleanup when the context is destroyed. It would invole > > reflection > > to hack in private fields of the Threads though, so the solution might > > fail > > in different JVM, and wouldn't work well with SecurityManagers. Basically, > > what you would have to do is to get the list of all the system's thread > > (through > > ThreadGroup), and for each of them, iterate the "threadLocals" map, and > > the "inheritableThreadLocals", to cleanup any instance for which the > > ClassLoader is the current webapp or of child of it. > > > > You might also have to cleanup a few other things though, such as if you > > have > > any JDBC drivers in your webapp's ClassLoader, they need to be explicitly > > unregistered, otherwise they will not be garbage collected. > > > > I create a prototype of a ContextListener that does that if you want. > > > > Guillaume |
|
From: Guillaume P. <gpo...@gl...> - 2004-10-23 04:33:25
|
Patrick, >I just want to get this straight, since we run into this memory leak a > lot here when redeploying our app a lot in development: The leak is > caused by ThreadLocals that aren't cleaned up in Dom4j? That's correct. But there are other potential causes for such leak, there might be other things than DOM4J that would need to be fixed. > If so, I wonder what could be done in the Dom4j library to help this > situation, like maybe an API call that tells it to "clean up"? I don't really think it could work, because you can only clean up a ThreadLocal variable from the same thread you put it in. I took a look at DOM4J's source code, and it seems like they use ThreadLocal for thread safe cache (per-thread cache). There's three classes that use thing pattern. In two places it's for the factory instance that is returned from a static getInstance(). It doesn't seem appropriate to me to be using per-thread caching there. For the last use, I'm not really sure why it's needed, they are caching a instances of QNameCache, in which synchronization is used. If that class was intended to be thread safe, then I'm not sure why per-thread caching is used, and if it wasn't intended to be used concurrently, then I don't see why synchronization is used. To me, the best solution seems for DOM4J to just get rid of their current uses of ThreadLocal variables, I don't see any value in it. Guillaume ----- Original Message ----- From: "Patrick Burleson" <pbu...@gm...> To: <spr...@li...> Sent: Friday, October 22, 2004 12:20 PM Subject: Re: [Springframework-developer] CGLIB memory usage within class loader >I just want to get this straight, since we run into this memory leak a > lot here when redeploying our app a lot in development: The leak is > caused by ThreadLocals that aren't cleaned up in Dom4j? > > If so, I wonder what could be done in the Dom4j library to help this > situation, like maybe an API call that tells it to "clean up"? > > Thanks, > Patrick > > > On Fri, 22 Oct 2004 00:50:33 -0400, Guillaume Poirier > <gpo...@gl...> wrote: >> ----- Original Message ----- >> From: "Guillaume Poirier" <gpo...@gl...> >> To: <spr...@li...> >> Sent: Thursday, October 21, 2004 6:25 PM >> Subject: Re: [Springframework-developer] CGLIB memory usage within class >> loader >> >> >> I only wish A) there was a way to explicitly blow away the classloader >> >> or B) have hibernate be redeployable. One option is to place all the >> >> Hibernate related classes inside Tomcat's shared Classloader. This >> >> might reduce the amount of times I need to restart. >> > >> > Well, if you care enough, I'm pretty sure you could create a >> > ContextListener >> > that would cleanup when the context is destroyed. It would invole >> > reflection >> > to hack in private fields of the Threads though, so the solution might >> > fail >> > in different JVM, and wouldn't work well with SecurityManagers. >> > Basically, >> > what you would have to do is to get the list of all the system's thread >> > (through >> > ThreadGroup), and for each of them, iterate the "threadLocals" map, and >> > the "inheritableThreadLocals", to cleanup any instance for which the >> > ClassLoader is the current webapp or of child of it. >> > >> > You might also have to cleanup a few other things though, such as if >> > you >> > have >> > any JDBC drivers in your webapp's ClassLoader, they need to be >> > explicitly >> > unregistered, otherwise they will not be garbage collected. >> > >> > I create a prototype of a ContextListener that does that if you want. >> > >> > Guillaume > > > ------------------------------------------------------- > This SF.net email is sponsored by: IT Product Guide on ITManagersJournal > Use IT products in your business? Tell us what you think of them. Give us > Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out > more > http://productguide.itmanagersjournal.com/guidepromo.tmpl > _______________________________________________ > Springframework-developer mailing list > Spr...@li... > https://lists.sourceforge.net/lists/listinfo/springframework-developer |
|
From: Rob H. <ro...@ca...> - 2004-10-19 18:06:19
|
Jurgen/All,
I'll check into this some more and see if I can reduce the memory usage
of the proxy classes. On another note I can't see the changes that
Jurgen has made, I made some changes about two hours ago and they seem
to have committed.
Rob
jürgen höller [werk3AT] wrote:
>BTW, I've polished Cglib2AopProxy's source code quite a bit: for example, it logs at debug level now (rather than info), and uses formatting consistent with the rest of Spring's sources.
>
>I've actually already done this polishing a second time: The earlier one got lost. It shows up in CVS, but then seems to have got overwritten by a subsequent commit. There was also a JDK 1.4 dependency fix in there (Boolean.valueOf with a boolean argument), which got lost too.
>
>So please, when you merge before a commit, double-check that you're not losing updates that someone else applied before you! It wasn't a big issue this time, but what if we don't notice it next time...
>
>Juergen
>
>
>-----Original Message-----
>From: jürgen höller [werk3AT]
>Sent: Tuesday, October 19, 2004 6:35 PM
>To: spr...@li...
>Subject: CGLIB memory usage within class loader
>
>
>I've just been made aware of the following by a werk3 colleague: When repeatedly creating CGLIB proxies for the same target class but for different advices - within the same class loader -, you'll create new proxied classes all the time, which won't get removed for the lifetime of the classes.
>
>The generated classes themselves are not the problem here. However, each of those proxy classes seems to hold strong references to its advices and target object, through holding the ProxyCallbackFilter instance that was passed into the Enhancer on proxy creation. Note that the proxy *class* holds that reference, not the proxy *instance*.
>
>The effect is that you can easily run out of memory if your advices or target objects occupy a large amount of memory, as none of those objects will get garbage collected as long as the CGLIB-generated proxy class is still hanging around in the class loader. This does *not* happen at all with JDK dynamic proxies.
>
>Of course, such excessive proxy creation is not the usual case in a production application. Web app restart is not a problem either, as the class loader will be shut down (releasing the CGLIB-generated proxy classes). The usual scenario where this memory leak becomes a problem is test suites, with application contexts getting created per test method.
>
>The following code snippet reproduces the issue:
>
> while (true) {
> ProxyFactory proxyFactory = new ProxyFactory();
> proxyFactory.setTarget(new TestBean() {
> private byte[] field = new byte[1000000];
> });
> proxyFactory.addAdvice(new DebugInterceptor());
> proxyFactory.setProxyTargetClass(true);
> TestBean tb = (TestBean) proxyFactory.getProxy();
> Thread.sleep(100);
> }
>
>Depending on the max memory available to the VM, you'll run into an OutOfMemoryError sooner or later, as the target TestBean instance with its byte array of size 1000000 never gets garbage collected. Note that this does *not* happen when removing the addAdvice call, as we're just generating a single CGLIB proxy class then.
>
>Is this behavior unavoidable? What do we recommend for such testing scenarios then? In particular for integration tests with Hibernate, a 5 MB leak per context creation is not really acceptable. Of course it's preferable to share the application context as far as possible, but there are still valid use cases for repeated creation...
>
>Juergen
>
>
>-------------------------------------------------------
>This SF.net email is sponsored by: IT Product Guide on ITManagersJournal
>Use IT products in your business? Tell us what you think of them. Give us
>Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more
>http://productguide.itmanagersjournal.com/guidepromo.tmpl
>_______________________________________________
>Springframework-developer mailing list
>Spr...@li...
>https://lists.sourceforge.net/lists/listinfo/springframework-developer
>
>
>
>
|
