#78 Add search engine of http://www.robotstxt.org/wc/active.html

open
nobody
5
2012-10-11
2002-02-26
No

on http://www.robotstxt.org/wc/active.html there is a
complete list of active agents-names used by
webcrawlers and search engine

would it be possible to add some more of them

eg
"SearchTone2.0 - IDEARE"

Discussion


Log in to post a comment.

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:

JavaScript is required for this form.





No, thanks