Heritrix: Internet Archive Web Crawler

This issue is now discussed in the new JIRA tracker at
http://webteam.archive.org/jira/browse/HER-439 -- please add further
comments at that location.

Logged In: YES
user_id=1220421

I must admit that it has been a bad idea from the beginning
to add a default value for the "maximumNumberOfKeys()"
invariant in the abstract QueueAssignmentPolicy class.

While it instantly made this feature available to all policy
implementations, there shouldn't be a default value at all.
Instead, we might define something like a int
UNBOUNDED_NUMBER_OF_KEYS = -1 constant in the interface,
which each implementation is allowed return.

Logged In: YES
user_id=924942

Thanks for patch. Sorry busted your assignment policy.
Below is commit.

Regards whether it should become an interface, how would you
suggest that the default behavior of maximumNumberOfKeys be
done -- each implementation would return
DEFAULT_MAX_NUMBER_OF_KEYS?

Fix for '[ 1219715 ] [patch] Signature change broke
BucketQueueAssignmentPolicy'Contributed by Christian
Kohlschuetter. Here is comment from Christian:

" The change of QueueAssignmentPolicy.getClassKey's
method signature from CrawlURI to CandidateURI has
broken BucketQueueAssignmentPolicy, making it behave
like HostnameQueueAssignmentPolicy (it inherited from
that class)"

*
src/java/org/archive/crawler/frontier/BucketQueueAssignmentPolicy.java
(getClassKey): Get host from ServerCache.