Using duplicate detection, it would be useful to
automatically determine that a server is identical to
another--perhaps this could be done by checking
checksums of the robots.txt and index.html documents?
Hmmmm. Seems fraught with difficulties. You can imagine a
staging server and a production server with identical
robots.txt and index.html, yet you certaintly wouldn't want
them to be considered the same server if for some reason
they both got into the document tree.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Logged In: YES
user_id=95470
Hmmmm. Seems fraught with difficulties. You can imagine a
staging server and a production server with identical
robots.txt and index.html, yet you certaintly wouldn't want
them to be considered the same server if for some reason
they both got into the document tree.