|
From: Geoff H. <ghu...@ws...> - 2002-05-03 15:04:49
|
On 3 May 2002, Alain FORCIOLI wrote: > 1- default behavior is to enable persistent_connection; > 2- htdig tries to get /robots.txt on every server in start_url; > 3- In the same time, htdig detects wether the remote server support > persistent connection mechanism. Yes, this is what I see in the code. > If the previouses sentences are true then there is a problem with some > HTTP servers. A few of them add a 'Connection: close' HTTP header when > returning HTTP status code 404. So I think what you're suggesting is that we don't mark persistent connections as false until we have a *successful* query with a closed connection. This sounds like a good fix to me. -- -Geoff Hutchison Williams Students Online http://wso.williams.edu/ |