Heritrix: Internet Archive Web Crawler

This issue is now discussed in the new JIRA tracker at
http://webteam.archive.org/jira/browse/HER-42 -- please add further
comments at that location.

Logged In: YES
user_id=144912

Testing with artificially small values results in expected
behavior: DNS and robots info refetched at the right times.
Closing.

Logged In: YES
user_id=896276

Most of this is fixed, but I am not sure that the DNS expiry
is ever checked and I'm not sure where this should be done.
Probably the right place is in the PreConditionEnforcer. The
code to be added should be something like:
if <DNS.expired> {
curi.setForcedPrerequisiteUri(<DNS-uri>);
}