#31 If-Modified-Since support

[Ken Lalonde]

WebKit makes no use of the If-Modified-Since HTTP header,
which makes static content service inefficient.
Of course, you can move your static content to a separate
HTTP server, but adding support for If-Modified-Since
buys most of the benefits and costs little.

The attached patch modifies mod_webkit to pass
the If-Modified-Since header via the request environment
dictionary. While arguably less than elegant,
this does the trick.

The patch to UnknownFileTypeServlet checks the
IMS value against the file's mtime. If it matches,
it sets "Status 304" and sends no content.
(It also corrects some unlikely but possible bugs.)

[Ken Lalonde]

Patch against today's WebKit CVS

[Ken Lalonde]

Patch against today's WebKit CVS

[Ken Lalonde]

Logged In: YES
user_id=203927

Here is an improved version of the above patch.
This version moves the
If-Modified-Since logic up to the HTTPServlet level,
so it may be used by other factories. I've modified PSP
so that "static" PSP pages may be cached at the client.
("Static" pages are those that don't use any dynamic PSP
hooks.)

There's also a fix to a typo in Session.py.

[Ken Lalonde]

Patch against today's Webware CVS

[Ken Lalonde]

Logged In: YES
user_id=203927

Here is a third version of the patch, against today's CVS.
The PSP caching is now off by default, to avoid breaking
existing code.
I've added a patch to mod_webkit2.
The patch to UnknownFileTypeServlet includes changes made in the
meantime.

[Stuart Donaldson]

Logged In: YES
user_id=326269

Ken,
Have you any test scripts to verify that the IMS is working
as expected?

Also, is there more to the isAuthenticated() method? It
does not appear to be related to IMS.

There are some other issues addressed in the
UnknownFileTypeServlet apparently. Can you provide a
description of changes in functionality here? It's not
clear how much of that is needed.

-- Thanks --

[Ken Lalonde]

Logged In: YES
user_id=203927

> Have you any test scripts to verify that the IMS
> is working as expected?

I've attached a simple script below (test-ims.py)
that will verify that a given HTTP server is handling
If-Modified-Since as expected.

> Also, is there more to the isAuthenticated() method?
> It does not appear to be related to IMS.

Dang -- that's an internal thing that shouldn't
have been part of the patch.
Just delete the call and definition.

> There are some other issues addressed in the
> UnknownFileTypeServlet apparently. Can you provide a
> description of changes in functionality here?
> It's not clear how much of that is needed.

The only essential change to UnknownFileTypeServlet
for IMS support is the addition of the
lastModified() method. But I think the other
changes are worthwhile too:
- the file in question is now opened exactly once,
and open() failure is handled correctly.
- The "CheckDate" config option is no longer used.
If found in the cache, the file mtime is always
checked against the cached value. The time
saved avoiding a stat() call is negligible,
and certainly not worth the hassle of returning
stale data.
- when a cached file changes, the old entry is
deleted completely. The old code just updated
the cached mtime and content, so files that grow
larger than MaxCacheContentSize would still stay
in the cache.
- the Content-Length HTTP header is set for all requests,
not just HEAD.
- the changes in HTTPServlet remove the need to add
the Last-Modified header.
- when sending the data to the client, we send no
more than the advertized amount (in case the file
has grown), and never loop on early EOF
(if it has shrunk). Both cases are unlikely,
but better safe...

Also, I recommend changing the default value of the
CacheContent config setting to 0.
Caching file data in memory is a waste of time,
since the memory-to-disk bandwidth isn't the bottleneck.
The O/S can handle file caching much more efficiently,
and if speed is really an issue, you're far better off
serving static data outside of WebKit.

[Ken Lalonde]

Script to verify IMS support

[Stuart Donaldson]

Logged In: YES
user_id=326269

I have applied this to CVS with the exception of the portion
which applied to PSP. I took the iconcept from the test
script and created a WebKit/Testing/TestIMS.py test.

The PSP patch for IMS still required that the PSP page
implement a method CachePSPByModTime. It seems if you need
to implement a method such as that, then you should just be
able to implement a lastModified method of your own. It
might be useful to provide a compiled time to the PSP
servlet to assist in this.

I am closing this patch, but will pose the question to the
devel list for comment.