Menu

#947 baiduspider not recognized as bot

7.3
open
nobody
None
5
2015-12-09
2015-12-09
No

This is for awstats 7.4: I see lots of hits like baiduspider-220-181-108-122.crawl.baidu.com in my Hosts List. At the same time (at least some) hits from baidu are also reported in the Robot/Spider Visitor List. I checked in robots.pm and baiduspider is there as 'baiduspider', - an example line from the apache logs is here:

123.125.71.55 - - [06/Dec/2015:10:03:48 +0100] "GET /en HTTP/1.1" 200 5088 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)"

Other robots e.g. google bot seem to be fine - they appear in the Robot Visitor List but not in the Host List.

Seems like not (all) hits from baidu are registerd and/or attributed to the respective categories (Hosts or Spiders) I did not double check it this also affects viewed/not-viewed traffic.

Discussion


Log in to post a comment.

MongoDB Logo MongoDB