#416 Cache files like a browser does

[Remon Sinnema]

Browsers caches files, like javascript files or images, to speed up displaying pages. I think it would be very useful if HtmlUnit also provided this functionality.

I currently run into performance problems with pages that include a lot of JavaScript files, like Dojo. One test that resursively clicks through all links in my web application (a couple of hunderd pages), takes almost three hours to run :-(

[Marc Guillemot]

Logged In: YES
user_id=402164
Originator: NO

I agree that it would be needed. I think that it should belong to commons-httpclient rather than to htmlunit directly and I've opened a issue for this a long time ago:
http://issues.apache.org/jira/browse/HTTPCLIENT-427

This could be implemented in htmlunit too if someone has time for it ;-(. I think that it is not that easy to handle correctly all cache related headers.

As a quick workaround you can wrap the default WebConnection into you own ones that caches for instance all js files.

[Remon Sinnema]

HttpWebConnection that caches text files

[Remon Sinnema]

Logged In: YES
user_id=451078
Originator: YES

It seems that comons-httpclient is unwilling to add this feature, since "we're not a browser".

In the meantime, I implemented a simple HttpWebConnection descendant that caches text files, see attachment.
File Added: CachingHttpWebConnection.java

[Remon Sinnema]

Logged In: YES
user_id=451078
Originator: YES

Use it as follows:
webClient.setWebConnection(new CachingHttpWebConnection(webClient, ".+\.js"));

[Marc Guillemot]

Logged In: YES
user_id=402164
Originator: NO

Interesting.

You could even make it more generic: it doesn't need to extend HttpWebConnection but should just be a wrapper around the existing web connection. The WebConnectionWrapper provides the base facility for this purpose.

[Remon Sinnema]

Logged In: YES
user_id=451078
Originator: YES

I'll leave that up to you guys, I'm perfectly happy with my current solution. Feel free to use this code in HtmlUnit.

[SourceForge Robot]

Logged In: YES
user_id=1312539
Originator: NO

This Tracker item was closed automatically by the system. It was
previously set to a Pending status, and the original submitter
did not respond within 14 days (the time period specified by
the administrator of this Tracker).

[Remon Sinnema]

Logged In: YES
user_id=451078
Originator: YES

It should not use StringWebResponse, but the original response instead. StringWebResponse always returns text/html as mime type.