[DrJava] [ drjava-Bugs-522331 ] Massive memory leakage (fwd)

[Brian S. <bs...@bs...>]

Thought you might find this interesting. (The problem was entirely my
fault, of course.)

---------- Forwarded message ----------
Date: Mon, 25 Feb 2002 08:32:50 -0800
From: no...@so...
To: no...@so...
Subject: [ drjava-Bugs-522331 ] Massive memory leakage

Bugs item #522331, was opened at 2002-02-24 21:43
You can respond by visiting:
http://sourceforge.net/tracker/?func=detail&atid=438935&aid=522331&group_id=44253

Category: Performance
Group: Serious
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Brian Stoler (brianstoler)
Assigned to: Brian Stoler (brianstoler)
Summary: Massive memory leakage

Initial Comment:
I think DrJava is leaking memory. I just started
getting some OutOfMemoryErrors when compiling something
in a long-running DrJava process. That process was
using 105 MB of memory.

I then started a new DrJava process, and it's using
32MB of RAM.

(Note that all of this only is concerning the main JVM;
the interactions JVM is a whole other game.)

I think it's leaking memory. Let's see how process RAM
use changes as I compile, to see if that's what is
leaking memory:

OK, loaded one file, and memory use went up less than a
megabyte.

Now I compiled it once, and memory use went up to 48MB.

Compiled the same file again, and memory use is now 52MB.

Further compiles of the same file: 55, 58, 61, 64, 66,
70, 73, 76, 80, 84, 87, 89, 92, 96, 99, 100, 103, 106,
108, and the next one causes an OutOfMemoryError.

I think this is a memory leak cause by compiling. :)

----------------------------------------------------------------------

>Comment By: Brian Stoler (brianstoler)
Date: 2002-02-25 08:32

Message:
Logged In: YES
user_id=343403

Fix committed in drjava-20020225-1623.

On my last "ant commit", the java process never used more
than 95MB of RAM. While that's a lot, the commit process
does a LOT, all inside on JVM, and it doesn't necessarily
try to hard to clean up garbage. So I think it's a lot
better. :)

----------------------------------------------------------------------

Comment By: Brian Stoler (brianstoler)
Date: 2002-02-25 08:22

Message:
Logged In: YES
user_id=343403

OK, with this fix, I just ran 50 straight compiles on
jdk1.4, and the memory use of the drjava process on my
machine reached and then stayed fixed at 60MB. I call this
bug fixed.

Note that this bug demonstrates the perils of designing code
that expects one runtime environment and then using it in
another. The compiler was not designed to be run over and
over again in the same JVM, and so it leaks memory like
crazy (I presume via some statics). In this case, the way
around it is to always load the compiler via a custom
classloader.

----------------------------------------------------------------------

Comment By: Brian Stoler (brianstoler)
Date: 2002-02-25 07:59

Message:
Logged In: YES
user_id=343403

Just made standard javac go through a proxy, and here are
the results:

1803k / 3588k
2172k / 5693k
4601k / 8990k
2455k / 10027k
8450k / 15245k
3864k / 15245k
9434k / 15245k
5537k / 15507k
9295k / 16646k
5255k / 16646k
8793k / 16646k
1754k / 16646k
1754k / 11964k
1754k / 6438k

I think this solved the problem. :) Now I just need to make
GJv6 use a proxy (not that anyone uses GJ).

I think this fix should speed up the test cases that use
compilation and also keep them from using so much memory
(which required -Xmx256M before!).

----------------------------------------------------------------------

Comment By: Brian Stoler (brianstoler)
Date: 2002-02-25 07:49

Message:
Logged In: YES
user_id=343403

I think that second part of my last comment is actually
wrong. I instrumented DrJava.java with a thread to print the
heap usage every 10 seconds (and call System.gc too), and it
appears that, although memory use can grow with each
compile, it's not permanent. Here are some (used / total)
measurements of the Java heap over time (duplicates deleted)
when I compiled over and over again using JSR-14 (user):

1779k / 5746k
2115k / 8941k
1761k / 8699k
5008k / 8998k
2777k / 9502k
5324k / 10448k
4910k / 13103k
2327k / 10964k
1744k / 6402k
5679k / 10141k
4820k / 13160k
8433k / 14716k
9285k / 16613k
16302k / 16613k
8033k / 16613k
1752k / 15654k
1753k / 11935k
1753k / 6418k


I attribute the weird, wild fluctuations to the vaguaries of
GC. (This was on Linux/Sun-1.4final.) Also, after like 30
compiles, the memory image of the process seemed fixed at 50 MB.

Now let's try the same test using javac (no proxy):

1748k / 3706k
5792k / 10055k
18204k / 32411k
32490k / 57712k
48988k / 66650k
57293k / 66650k

(I loaded a file and then hit F5 in rapid succession.)

Oh my, this is the problem! So we need to make the standard
javac compiler option use a proxy, so that the compiler
classes can be fully unloaded/reloaded.



----------------------------------------------------------------------

Comment By: Brian Stoler (brianstoler)
Date: 2002-02-25 07:17

Message:
Logged In: YES
user_id=343403

OK, done some more testing. First, this general trend is
reproducable on my Linux machine on both sun-1.4 and ibm-1.3.

Now. My first guess was that this would only show up when I
compiled using a CompilerProxy extension, so I tried
compiling with vanilla javac-from-classpath; but the same
problem persisted.

Then I thought that, if I made sure that the CompilerProxy
threw away its classloader after each compile, I could
ensure the compiler classes would be unloaded and thus all
resources would for sure be cleaned up. So I changed
CompilerProxy to re-create the classloader on every compile,
and it does definately allow the compiler classes to be
unloaded (as seen via java -verbose) -- but the memory use
still keeps growing, although possibly at a slower rate.

----------------------------------------------------------------------

You can respond by visiting:
http://sourceforge.net/tracker/?func=detail&atid=438935&aid=522331&group_id=44253