Robots and crawlers showing up in Hosts instead of Robots
AWStats Log Analyzer
Brought to you by:
eldy,
manolamancha
Menu
▾
▴
#805 Robots and crawlers showing up in Hosts instead of Robots
The "Hosts" section of my awstats report contains all the spiders and crawlers. Google, MSN, Yahoo, etc are all showing up there with hostnames like b3091237.crawl.yahoo.net, crawl-66-249-65-232.googlebot.com, and msnbot-207-46-193-80.search.msn.com. I would have expected these to show up in the Robots pane. How does awstats determine whether a request is coming from a robot or not?
This behaviour come up with ActivePerl 5.14. Downgrade to 5.10 gets the panes to normal, viewing robots, spiders, searchphrases, searchengines under their appopriate pane.