on http://www.robotstxt.org/wc/active.html there is a
complete list of active agents-names used by
webcrawlers and search engine
would it be possible to add some more of them
"SearchTone2.0 - IDEARE"
Log in to post a comment.
Sign up for the SourceForge newsletter:
You seem to have CSS turned off.
Please don't fill out this field.