Using duplicate detection, it would be useful to
automatically determine that a server is identical to
another--perhaps this could be done by checking
checksums of the robots.txt and index.html documents?
Logged In: YES
Hmmmm. Seems fraught with difficulties. You can imagine a
staging server and a production server with identical
robots.txt and index.html, yet you certaintly wouldn't want
them to be considered the same server if for some reason
they both got into the document tree.
Log in to post a comment.