Using duplicate detection, it would be useful to
automatically determine that a server is identical to
another--perhaps this could be done by checking
checksums of the robots.txt and index.html documents?
Logged In: YES
Hmmmm. Seems fraught with difficulties. You can imagine a
staging server and a production server with identical
robots.txt and index.html, yet you certaintly wouldn't want
them to be considered the same server if for some reason
they both got into the document tree.