I wish I knew the Exact answer ...  I did many things and the problem eventually went away.
My app is pretty complicated so I cant say for sure what was necessary and what was a coincidence.
 
 
First off I did have something like a variable inside the interpreter which referenced the container ...
It was actually an argument to a bound method ... but I think that's the same thing ...
I got rid of that and instead used a ThreadLocal variable.
 
Then I did what Stewart suggested
 
protected void finalize()
 {
if( this.interpreter != null )
{
    this.interpreter.getNameSpace().clear();
   this.interpreter = null;
   }
}
 
This didn't seem to help by itself ... Still memory rose and wouldn't GC ... This was where I was about to give up but got encouraged to keep trying and use a Memory profiler ...
 
so I went further.  I had several places where caches of objects were being kept, which in turn held onto interpreters.  I added similar methods to those classes.   Also I added a "dispose" method to several of these classes which null'd out all their member variables.
 
Then I was still having some memory problems and used JProfiler and found that I had one line of code in a Bsh method call (to my code) which wasn't closing a prepared statement ... since the Connection was being cached in a connection cache, these lost statements were not garbage collected ...
 
Then I tried cleaning up my use of Interpreter .... in my app I have 2 levels of scripts,   top level scripts ("Campaign") and lower level scripts ("State") ... in a possibly misguided attempt at efficiency these shared the same interpreter, and the State scripts were implemented as bound methods ... I got rid of that and instead used separate Interpreter instances for each state instead of sharing the parent interpreter.   I still cache them, but not for as long a time.  In my app there are convenient points where the execution thread waits for more messages, at that point I release all the cached interpreters ... I destroy and recreate them more often (calling my new "dispose" method on the containing classes).
This is a bit of a performance hit, but not much compared to evergrowing memory ...
 
I'm not entirely sure which of these changes was really  necessary ... or even a good idea ... but all together totally solved the problem.
I just completed a test which sent out 40,000 emails to a bounced address ... processed the bounced emails and 5 state changes (each of them implemented as bsh scripts) ... about 500,000 overall script creations and invocations and the memory is still flat ...
 
 
 
----- Original Message -----
From: Alexey Zinger
To: beanshell-users@lists.sourceforge.net
Sent: Tuesday, September 26, 2006 6:20 PM
Subject: Re: [Beanshell-users] Beanshell GC problems

I think many of us would love to know what fix was.  What kinds of circular references were they?  Or is it something specifically pertaining to your app?

David Lee <dlee@calldei.com> wrote:
Thanks to Stewart and Nick I got this fixed !!!
Turned out to be much more subtle then a single circular reference ...
thanks to JProfiler I was able to spot several places creating big circular
chains ...
Why doesn't Java find these ? I still haven't figured out ... the JVM docs
say it can handle circular references ... but apparently some are just too
confounding !
I get to keep BeanShell !!!! Whoo Hoo !!!! I was NOT looking forward to
writing class loaders, java compiler interfaces, cache management, file
updating and all that stuff ... in a hurry.

Anyway All is well ... ran 500,000 iterations and now memory is stable at
about 50MB ... where before 100,000 ramped up to 200M+ ... and better yet
... its FLAT ... I can GC and looks like most everything is reclaimed ...

Live is good again and I can take the bullets out of the gun :)

FYI, at 5:00 AM CST I tried to download ANY Java profiler tool to use ...
Happy to pay for it if it works. I tried Quest JProbe, YourKit Profiler
and JProfiler.
I could download all 3 ... no problem ... but when requesting the license
key ... JProfiler sent it within 30 seconds, the other two I'm still
waiting 4 hours later ...

Guess who gets my $ .... I'm sure the other two work fine ... but when it
comes to "Have to have it RIGHT NOW ... before my clients get out of bed and
find the app server isn't running " ... timely activation is a strong
selling factor.

-David Lee





> Well, that's fair enough, but I'd look it over one more time. Tenacity
> sometimes wins through.
> I've attached some png pictures which may help you, taken from when I
> trouble shooted my situation. It did take me 2-3 days to completely
> solve, even after a correct diagnosis.
>
> beanshell_self_ref.png is a JProfiler image showing the exact problem
> in a clear way. A chain of references in memory, consisting of 7
> objects which are kind of in a "stay-in-memory-club". As a group they
> are completely orphaned, but they reference each other in a circle. I
> think that somewhere the bsh must be keeping some static references.
>
> getNamespace().clear() does break this circle. I have a wrapper class
> (a Calculator class - this is what I am using bsh for) which isolates
> bsh from the rest of the app (in case I ever decided to use Rhino or
> something else).
> I'm creating and destroying Calculator objects all over the place, but
> I'm very strictly controlling their creation and destruction. This is
> kind of like when I was a C++ programmer where there is no GC and you
> must control memory allocation yourself - every "new" keyword needed a
> "delete" keyword, etc, and you had to think about detecting when an
> object is no longer required.
> When I destroy a Calculator object, (ie, set it to null for GC) I
> always first call myCalc.destroy(), which cleans up the bsh
> references, and then the GC can do its work cleanly.
> The two pictures jvm_problem and jvm_solved show what was happening
> before and after I implemented this. I am thinking you have a similar
> situation. As you can see, my app was crashing after 10 minutes of
> use.
>
> The final picture is simply to show understanding for if you do decide
> to rip bsh out of your app!
>
> Stewart
>
>
> On 26/09/06, David Lee wrote:
>> Thank you very much Stewart, unfortunately it didnt work in my case :(
>> Looks like I'm going to have to rip BeanShell out and go the
>> javac/classloader route ... better make an extra pot of coffee ... and I
>> was
>> getting to really really like BeanShell :(
>>
>>
>> -David
>>
>>
>>
>> ----- Original Message -----
>> From: "Stewart Cambridge"
>> To: "David A. Lee" ;
>>
>> Sent: Tuesday, September 26, 2006 5:05 AM
>> Subject: Re: [Beanshell-users] Beanshell GC problems
>>
>>
>> > http://groups.google.co.uk/group/comp.lang.java.programmer/browse_thread/thread/5142bb824b09e544/ea06d4d7205a24a8
>> >
>> > It is the recursive self-references which cause the hang up.
>> > The solution I use is that whatever is holding a reference to the
>> > interpreter has to manually clean up, before it is destroyed itself.
>> >
>> > Basically this is a wrapper class around the interpreter object.
>> >
>> > protected void finalize()
>> > {
>> > if( this.interpreter != null )
>> > {
>> > this.interpreter.getNameSpace().clear();
>> > this.interpreter = null;
>> > }
>> > }
>> >
>> > HTH
>> >
>> > On 25/09/06, David A. Lee wrote:
>> >>
>> >> Have you found any way to get around your problem with Beanshell
>> >> interpreters not getting garbage collected ? I'm running into what I
>> >> think
>> >> is the same problem ... memory quickly goes up to 300MB+ and even
>> >> after
>> >> all
>> >> references to the interpreter are gone, gc doesnt clean it up ...
>> >>
>> >> Any suggestions greatly welcome !
>> >>
>> >> -David Lee
>> >> ------------------------------------------------
>> >> David A. Lee
>> >> VP Engineering
>> >> Nexstra, Inc.
>> >> dlee@nexstra.com
>> >> www.nexstra.com
>> >
>> > -------------------------------------------------------------------------
>> > Take Surveys. Earn Cash. Influence the Future of IT
>> > Join SourceForge.net's Techsay panel and you'll get the chance to share
>> > your
>> > opinions on IT & business topics through brief surveys -- and earn cash
>> > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
>> > _______________________________________________
>> > Beanshell-users mailing list
>> > Beanshell-users@lists.sourceforge.net
>> > https://lists.sourceforge.net/lists/listinfo/beanshell-users
>> >
>>
>>
>


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys -- and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
Beanshell-users mailing list
Beanshell-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/beanshell-users



Alexey
2001 Honda CBR600F4i (CCS)
1992 Kawasaki EX500
http://azinger.blogspot.com
http://bsheet.sourceforge.net


Do you Yahoo!?
Everyone is raving about the all-new Yahoo! Mail.


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys -- and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV


_______________________________________________
Beanshell-users mailing list
Beanshell-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/beanshell-users