[EsiGate-devel] Replacing EsiGate cache by Apache HttpClient native cache implementation

[François-Xavier B. <fra...@ce...>]

Hi all,

As mentioned in this Mantis ticket : 
https://sourceforge.net/apps/mantisbt/webassembletool/view.php?id=47 
there is now a HTTP cache implementation included in Apache HttpClient 
project. Switching to this implementation instead of our own would have 
several benefits :
- reduce our code base, current implementation represents 20% of the 
code of EsiGate
- reduce the work needed to maintain it
- share our effort with HttpClient cache developers

But there are also some features we will loose or have to rewrite :
- EsiGate currently caches also error pages, temporary redirects and 
network errors in order to improve performance. I don't see how to 
achieve this with HttpClient cache.
- possibility to use of a local filesystem instead of the backend server 
when it is down. This cannot be done with HttpClient cache but this 
feature is not often used in production.
- load-balancing feature that has just been added in the trunk has to be 
refactored (maybe using "route" mechanism in HttpClient)
- use stale cache entries when the server is down

About the use of stale cache entries, I just discovered that HttpClient 
Cache implements RFC 5861, an extension to HTTP 1.1 specification 
enabling to reuse a cache entry when the backend server is down or just 
while the cache entry is being refreshed. This behavior looks like to 
Varnish "grace" and "saint" mode
http://tools.ietf.org/html/rfc5861
http://www.gossamer-threads.com/lists/varnish/misc/24061?do=post_view_threaded#24061
We could take advantage of this feature just by adding http headers to 
the requests and I think it would improve a lot our performance.

Some feedback ?

FX