Nobody/Anonymous - 2006-01-31

Hi,

I wanted to suggest an addition for the robots database (robots.pm) but couldn't find a place to do this.

I have had a robot with the following User Agent spidering some websites:
"mozilla/5.0 (compatible; heritrix/1.2.0 +http://www.yell.com)"

Heritrix is the Internet Archive's web crawler software - which I found is used by a few different companies (see http://crawler.archive.org/).

Well worth adding heritrix to the robots database, oh and make it easier to suggest further additions.

Cheers,
Ross (netflare.co.uk)