Currently HttpWebConnection dictates the size threshold of responses that stay in memory vs go to disk (private static final long MAX_IN_MEMORY = 500 * 1024;), this threshold should be configurable at runtime.
As an example why, unless you explicitly call response.cleanUp() religiously, you can quickly find your system filled with large temp files, because if anything causes your JVM to exit abruptly, file.deleteOnExit() will fail to delete all these temp files. Further, for a long running JVM you don't want to wait to delete them until exit time.
Suggestions:
1) Enable MAX_IN_MEMORY to be configured either on the WebClient object, or on the WebRequest object (or both, where the latter overrides the former).
2) Add a finalize handler to OnFile that calls cleanUp(), so that when the garbage collector removes the object is also removes the temp file.
Both done, thanks for your suggestions.
Thanks for such a fast fix!