I understand that when I see the "url rejected" in the output, that means it is in the excluded list. I have excluded /calendar/ from htdig, however, it seems to find it necessarry to repeatedly hit /calendar/index.php?page=XX&day=XX&month=XX&year=XX (where XX are repeadetly replaced with increasing values, so far from 2001 to 2007). Is this because it is finding those links somewhere or is it just doing a zombie-like, brain-dead, recursive search on the url which is then rejected?
Sjan Evardsson
Alaska Pacific University