[Htmlunit-user] Lingering FrameWindow instances in WebClient

[Sebastian C. <seb...@ou...>]

Greetings,

I have automatically generated test cases which fetches a page, runs 
tests on that page, drops all references to that page, calls 
closeAllWindows on the WebClient, fetches the next page, runs tests on 
it depending on the outcome of the tests for the previous page, &c. The 
WebClient instance has a fairly long life time. JavaScript is enabled 
and some if the tested sites use a lot of it. I'm using HtmlUnit 2.13.

My problem is that the heap fills up after some time of doing this. The 
heap is set to a couple of GBs. Looking at the heap dump, I can see that 
the private instance variable windows_ in the WebClient object is full 
of FrameWindow objects. These objects in turn have references to 
HtmlPage objects. These objects will never be collected.

There's also a lot of WeakReference-objects in jobManagers_. This is 
less of an issue than the content of windows_ though, but it seems to be 
related.

I don't know if this is a bug or if I'm doing something wrong. A thought 
I had was to create a new WebClient object every N calls to getPage, but 
I would rather not do this. I tried adding the following code to 
WebClient#closeAllWindows(), at the end of it:

+        /* Clear all but the current WebWindow */
+        for(Iterator<WebWindow> it = windows_.iterator(); it.hasNext();) {
+               WebWindow window = it.next();
+               if (window != currentWindow_) {
+                       it.remove();
+                       fireWindowClosed(new WebWindowEvent(window,
+                               WebWindowEvent.CLOSE,
+                               window.getEnclosedPage(), null));
+               }
+        }
+
+        /* clear all the job managers */
+        for(Iterator<WeakReference<JavaScriptJobManager>> it =
+                   jobManagers_.iterator(); it.hasNext();) {
+               JavaScriptJobManager manager = it.next().get();
+               if (manager != null) {
+                       manager.shutdown();
+               }
+
+               it.remove();
+        }

My understanding of HtmlUnit is a bit limited, so I don't know if it's a 
good idea. It *seems* to work, heap usage is reduced and nothing 
crashes, except sometimes JS seems to get stuck in a loop, which may or 
may not have something to do with shutting down the JavaScriptJobManagers.

I don't know if I'm doing something wrong that causes the windows_ 
variable to fill up. I'm calling closeAllWindows() between every test, 
but without the code above, windows_ fills up after a while. I also feel 
that I have too little to go on to provide a bug report, should it be a 
bug. That's why I'm asking on the Users list.

I'm sorry to say that I have not been able to reproduce this problem 
consistently because of the complexity of the tests I'm running, and I 
can't share any heap dumps or code. Sorry.

//Sebastian