#1577 Excessive usage of network resources

[Joerg Werner]

Since v2.14, HtmlUnit appears to open a new network connection for each request. Furthermore, connections are not closed orderly, but simply abandoned. So after a while, not only the HtmlUnit process may have many open sockets, but also the server process. Especially the latter is quite a serious issue.

There is no specific test case that I could provide. Simply request the same (existing) page again and again in a loop and watch the output of netstat or similar tools:

WebClient webClient = new WebClient(BrowserVersion.FIREFOX_17);
for (int i = 0; i < 10000; i++)
{
HtmlPage page = webClient.getPage("http://localhost:8080/foo");
}

[Ahmed Ashour]

HttpWebConnectionTest.remotePort() test was added as @NotYetImplemented

[Ahmed Ashour]

Thanks a lot for reporting, fixed in SVN.

Please re-open if you have any concern.

[Joerg Werner]

Hi Ahmed,

I would like to re-open this issue. The current implementation uses a static pooling connection manager that is shared among all web clients. I think that each web client/web connection should get an own pooling connection manager.

Thanks,
J.

[Ahmed Ashour]

Hi Joerg,

You are right, and that's the cause of the current failing test.

The connection manager will be per HttpWebConnection, hope to commit it shortly.

[Ahmed Ashour]

Committed, please provide feedback if you have any concern.

[Joerg Werner]

The issue is gone now. Thanks, Ahmed!

[Joerg Werner]

Hi Ahmed,

when rechecking your fix I wasn't careful enough. I did only check that connections will be properly reused for subsequent requests. However, part 2 of my original submission, the orderly shutdown of connections when the web client is closed, is still open. Maybe closing the connection pool in HttpWebConnection.shutdown() will do the trick.

Thanks,
J.

[Ahmed Ashour]

Hi Joerg,

How about now?

I couldn't think of a quick test case :(

[Joerg Werner]

Me, neither... But the good news is: the fix works like a charm. Thanks.