[htdig] Unable to build connection with fqdn.of.host:80

[Phydeaux <re...@ta...>]

Hi!

We've been using htdig for a while and we're running release 3.1.6 to index a set of related
web sites that are split across three servers. In the last few days it seems that the digging
gets stuck somewhere on the host that handles our mailing lists. There are several thousand
URLs on that server and somewhere in the middle it throws errors like this:

3962:3985:0:http://fqdn.of.host/full/url/first.html: Unable to build connection with fqdn.of.host:80
 no server running
3963:3986:0:http://fqdn.of.host/full/url/next.html:  no server running

All subsequent attempts by the digger at obtaining data from this host fail with "no server
running." I have checked and the server is, in fact, running. Furthermore, the server is not
under much of a load and it can be reached just fine from other machines and various
web browsers. If I do stop/restart the server while htdig is complaining that there is no
server it stops and starts the server just fine but htdig's message stays the same.

Where we used to have no pause whatsoever during digging, I have tried setting a pause of
2, and then 4 seconds. It seems to take longer to fail, but we still see failures. Can anyone
suggest what might be wrong? It's as if htdig can't get one page for some reason and then
never tries to get any more.

reb