#3 Rescrape headers now and again to look for changed files

[Anonymous]

If a day moves URL, we currently do rescrape it and
update (this happens when daily editions are bundled
into volumes). However, we do not speculative recrawl
all pages looking for changes.

We should do this every now and again for the entire
backlog of stuff. And only update things if the date
in the HTTP header has changed - if cunning we can do
the kind of HTTP request which is for just the header.