Menu
▾
▴
htmlunit-user
|
From: Sebastian C. <seb...@ou...> - 2013-12-19 09:14:50
|
Greetings,
I have automatically generated test cases which fetches a page, runs
tests on that page, drops all references to that page, calls
closeAllWindows on the WebClient, fetches the next page, runs tests on
it depending on the outcome of the tests for the previous page, &c. The
WebClient instance has a fairly long life time. JavaScript is enabled
and some if the tested sites use a lot of it. I'm using HtmlUnit 2.13.
My problem is that the heap fills up after some time of doing this. The
heap is set to a couple of GBs. Looking at the heap dump, I can see that
the private instance variable windows_ in the WebClient object is full
of FrameWindow objects. These objects in turn have references to
HtmlPage objects. These objects will never be collected.
There's also a lot of WeakReference-objects in jobManagers_. This is
less of an issue than the content of windows_ though, but it seems to be
related.
I don't know if this is a bug or if I'm doing something wrong. A thought
I had was to create a new WebClient object every N calls to getPage, but
I would rather not do this. I tried adding the following code to
WebClient#closeAllWindows(), at the end of it:
+ /* Clear all but the current WebWindow */
+ for(Iterator<WebWindow> it = windows_.iterator(); it.hasNext();) {
+ WebWindow window = it.next();
+ if (window != currentWindow_) {
+ it.remove();
+ fireWindowClosed(new WebWindowEvent(window,
+ WebWindowEvent.CLOSE,
+ window.getEnclosedPage(), null));
+ }
+ }
+
+ /* clear all the job managers */
+ for(Iterator<WeakReference<JavaScriptJobManager>> it =
+ jobManagers_.iterator(); it.hasNext();) {
+ JavaScriptJobManager manager = it.next().get();
+ if (manager != null) {
+ manager.shutdown();
+ }
+
+ it.remove();
+ }
My understanding of HtmlUnit is a bit limited, so I don't know if it's a
good idea. It *seems* to work, heap usage is reduced and nothing
crashes, except sometimes JS seems to get stuck in a loop, which may or
may not have something to do with shutting down the JavaScriptJobManagers.
I don't know if I'm doing something wrong that causes the windows_
variable to fill up. I'm calling closeAllWindows() between every test,
but without the code above, windows_ fills up after a while. I also feel
that I have too little to go on to provide a bug report, should it be a
bug. That's why I'm asking on the Users list.
I'm sorry to say that I have not been able to reproduce this problem
consistently because of the complexity of the tests I'm running, and I
can't share any heap dumps or code. Sorry.
//Sebastian
|
|
From: Ahmed A. <asa...@ya...> - 2013-12-19 10:40:15
|
Hi Sebastian,
The below test case works fine, even if there is a JS exception in frame1 for example.
Can you provide more details?
/**
* @throws Exception if the test fails
*/
@Test
public void gc() throws Exception {
final String html =
"<html><head><title>frames</title></head>\n"
+ "<frameset cols='180,*'>\n"
+ "<frame name='f1' src='1.html'/>\n"
+ "<frame name='f2' src='2.html'/>\n"
+ "</frameset>\n"
+ "</html>";
final String frame1 = "<html><head><title>1</title></head>\n"
+ "<body>1"
+ "<script>\n"
+ " parent.frames['f2'].location.href = '3.html';\n"
+ "</script>\n"
+ "</body></html>";
final String frame3 = "<html><head><title>page 3</title></head><body></body></html>";
final WebClient webClient = getWebClient();
final MockWebConnection conn = new MockWebConnection();
webClient.setWebConnection(conn);
conn.setDefaultResponse("<html><head><title>default</title></head><body></body></html>");
conn.setResponse(URL_FIRST, html);
conn.setResponse(new URL(URL_FIRST, "1.html"), frame1);
conn.setResponse(new URL(URL_FIRST, "3.html"), frame3);
webClient.getPage(URL_FIRST);
Assert.assertEquals(3, webClient.getWebWindows().size());
webClient.closeAllWindows();
Assert.assertEquals(1, webClient.getWebWindows().size());
}
Yours,
Ahmed
________________________________
From: Sebastian Cato <seb...@ou...>
To: htm...@li...
Sent: Thursday, December 19, 2013 12:14 PM
Subject: [Htmlunit-user] Lingering FrameWindow instances in WebClient
Greetings,
I have automatically generated test cases which fetches a page, runs
tests on that page, drops all references to that page, calls
closeAllWindows on the WebClient, fetches the next page, runs tests on
it depending on the outcome of the tests for the previous page, &c. The
WebClient instance has a fairly long life time. JavaScript is enabled
and some if the tested sites use a lot of it. I'm using HtmlUnit 2.13.
My problem is that the heap fills up after some time of doing this. The
heap is set to a couple of GBs. Looking at the heap dump, I can see that
the private instance variable windows_ in the WebClient object is full
of FrameWindow objects. These objects in turn have references to
HtmlPage objects. These objects will never be collected.
There's also a lot of WeakReference-objects in jobManagers_. This is
less of an issue than the content of windows_ though, but it seems to be
related.
I don't know if this is a bug or if I'm doing something wrong. A thought
I had was to create a new WebClient object every N calls to getPage, but
I would rather not do this. I tried adding the following code to
WebClient#closeAllWindows(), at the end of it:
+ /* Clear all but the current WebWindow */
+ for(Iterator<WebWindow> it = windows_.iterator(); it.hasNext();) {
+ WebWindow window = it.next();
+ if (window != currentWindow_) {
+ it.remove();
+ fireWindowClosed(new WebWindowEvent(window,
+ WebWindowEvent.CLOSE,
+ window.getEnclosedPage(), null));
+ }
+ }
+
+ /* clear all the job managers */
+ for(Iterator<WeakReference<JavaScriptJobManager>> it =
+ jobManagers_.iterator(); it.hasNext();) {
+ JavaScriptJobManager manager = it.next().get();
+ if (manager != null) {
+ manager.shutdown();
+ }
+
+ it.remove();
+ }
My understanding of HtmlUnit is a bit limited, so I don't know if it's a
good idea. It *seems* to work, heap usage is reduced and nothing
crashes, except sometimes JS seems to get stuck in a loop, which may or
may not have something to do with shutting down the JavaScriptJobManagers.
I don't know if I'm doing something wrong that causes the windows_
variable to fill up. I'm calling closeAllWindows() between every test,
but without the code above, windows_ fills up after a while. I also feel
that I have too little to go on to provide a bug report, should it be a
bug. That's why I'm asking on the Users list.
I'm sorry to say that I have not been able to reproduce this problem
consistently because of the complexity of the tests I'm running, and I
can't share any heap dumps or code. Sorry.
//Sebastian
------------------------------------------------------------------------------
Rapidly troubleshoot problems before they affect your business. Most IT
organizations don't have a clear picture of how application performance
affects their revenue. With AppDynamics, you get 100% visibility into your
Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
_______________________________________________
Htmlunit-user mailing list
Htm...@li...
https://lists.sourceforge.net/lists/listinfo/htmlunit-user |
|
From: Sebastian C. <seb...@ou...> - 2013-12-19 11:32:44
|
Hello, Not at this time. I'm working on locating the problem. All I have is a heap dump which I cannot share. I've tried to replicate it, but so far no luck. Will keep you informed if/when I find something. In the mean time, would emptying windows_ on closeAllWindows() work or would that cause other issues? I've built a version locally that does this (without the JavaScriptJobManager part posted earlier) and tested it a bit and it solves the heap problem I'm having. Not saying it should be added to the repo because the problem may be on my side, and it would also be nice to locate the cause of the problem before changing anything, but if it doesn't have any negative consequences it would make my Christmas a bit easier :) Cheers, Sebastian On 2013-12-19 11:40, Ahmed Ashour wrote: > > Hi Sebastian, > > > The below test case works fine, even if there is a JS exception in > frame1 for example. > > > Can you provide more details? > > |
×
Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.
Thanks for helping keep SourceForge clean.
X