|
From: Brian S. <bs...@bs...> - 2002-02-25 16:33:49
|
Thought you might find this interesting. (The problem was entirely my fault, of course.) ---------- Forwarded message ---------- Date: Mon, 25 Feb 2002 08:32:50 -0800 From: no...@so... To: no...@so... Subject: [ drjava-Bugs-522331 ] Massive memory leakage Bugs item #522331, was opened at 2002-02-24 21:43 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=438935&aid=522331&group_id=44253 Category: Performance Group: Serious >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Brian Stoler (brianstoler) Assigned to: Brian Stoler (brianstoler) Summary: Massive memory leakage Initial Comment: I think DrJava is leaking memory. I just started getting some OutOfMemoryErrors when compiling something in a long-running DrJava process. That process was using 105 MB of memory. I then started a new DrJava process, and it's using 32MB of RAM. (Note that all of this only is concerning the main JVM; the interactions JVM is a whole other game.) I think it's leaking memory. Let's see how process RAM use changes as I compile, to see if that's what is leaking memory: OK, loaded one file, and memory use went up less than a megabyte. Now I compiled it once, and memory use went up to 48MB. Compiled the same file again, and memory use is now 52MB. Further compiles of the same file: 55, 58, 61, 64, 66, 70, 73, 76, 80, 84, 87, 89, 92, 96, 99, 100, 103, 106, 108, and the next one causes an OutOfMemoryError. I think this is a memory leak cause by compiling. :) ---------------------------------------------------------------------- >Comment By: Brian Stoler (brianstoler) Date: 2002-02-25 08:32 Message: Logged In: YES user_id=343403 Fix committed in drjava-20020225-1623. On my last "ant commit", the java process never used more than 95MB of RAM. While that's a lot, the commit process does a LOT, all inside on JVM, and it doesn't necessarily try to hard to clean up garbage. So I think it's a lot better. :) ---------------------------------------------------------------------- Comment By: Brian Stoler (brianstoler) Date: 2002-02-25 08:22 Message: Logged In: YES user_id=343403 OK, with this fix, I just ran 50 straight compiles on jdk1.4, and the memory use of the drjava process on my machine reached and then stayed fixed at 60MB. I call this bug fixed. Note that this bug demonstrates the perils of designing code that expects one runtime environment and then using it in another. The compiler was not designed to be run over and over again in the same JVM, and so it leaks memory like crazy (I presume via some statics). In this case, the way around it is to always load the compiler via a custom classloader. ---------------------------------------------------------------------- Comment By: Brian Stoler (brianstoler) Date: 2002-02-25 07:59 Message: Logged In: YES user_id=343403 Just made standard javac go through a proxy, and here are the results: 1803k / 3588k 2172k / 5693k 4601k / 8990k 2455k / 10027k 8450k / 15245k 3864k / 15245k 9434k / 15245k 5537k / 15507k 9295k / 16646k 5255k / 16646k 8793k / 16646k 1754k / 16646k 1754k / 11964k 1754k / 6438k I think this solved the problem. :) Now I just need to make GJv6 use a proxy (not that anyone uses GJ). I think this fix should speed up the test cases that use compilation and also keep them from using so much memory (which required -Xmx256M before!). ---------------------------------------------------------------------- Comment By: Brian Stoler (brianstoler) Date: 2002-02-25 07:49 Message: Logged In: YES user_id=343403 I think that second part of my last comment is actually wrong. I instrumented DrJava.java with a thread to print the heap usage every 10 seconds (and call System.gc too), and it appears that, although memory use can grow with each compile, it's not permanent. Here are some (used / total) measurements of the Java heap over time (duplicates deleted) when I compiled over and over again using JSR-14 (user): 1779k / 5746k 2115k / 8941k 1761k / 8699k 5008k / 8998k 2777k / 9502k 5324k / 10448k 4910k / 13103k 2327k / 10964k 1744k / 6402k 5679k / 10141k 4820k / 13160k 8433k / 14716k 9285k / 16613k 16302k / 16613k 8033k / 16613k 1752k / 15654k 1753k / 11935k 1753k / 6418k I attribute the weird, wild fluctuations to the vaguaries of GC. (This was on Linux/Sun-1.4final.) Also, after like 30 compiles, the memory image of the process seemed fixed at 50 MB. Now let's try the same test using javac (no proxy): 1748k / 3706k 5792k / 10055k 18204k / 32411k 32490k / 57712k 48988k / 66650k 57293k / 66650k (I loaded a file and then hit F5 in rapid succession.) Oh my, this is the problem! So we need to make the standard javac compiler option use a proxy, so that the compiler classes can be fully unloaded/reloaded. ---------------------------------------------------------------------- Comment By: Brian Stoler (brianstoler) Date: 2002-02-25 07:17 Message: Logged In: YES user_id=343403 OK, done some more testing. First, this general trend is reproducable on my Linux machine on both sun-1.4 and ibm-1.3. Now. My first guess was that this would only show up when I compiled using a CompilerProxy extension, so I tried compiling with vanilla javac-from-classpath; but the same problem persisted. Then I thought that, if I made sure that the CompilerProxy threw away its classloader after each compile, I could ensure the compiler classes would be unloaded and thus all resources would for sure be cleaned up. So I changed CompilerProxy to re-create the classloader on every compile, and it does definately allow the compiler classes to be unloaded (as seen via java -verbose) -- but the memory use still keeps growing, although possibly at a slower rate. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=438935&aid=522331&group_id=44253 |