I use the last version of Awstats (6.95) in order to generate statistics.
However, I noticed that some robots are not included in files robots.pm or
search_engines.pm with this last version.
Indeed, some robots are in the section named "Host" rather than the section
"Robots", which distorts the statistics.
It's the case for example for Google Bot (GoogleBot-Image or IP addresses
66.249.6. ), Yahoo (IP addresses 67.195.. ), Inktomi (IP 72.30.. ),
Exalead (IP 18.104.22.168 *).
I can always change these files manually, but it risks to take time, and it
can be brought to change regularly.
The last added entries to these files are in the year 2009 or 2006.
I just want to know where I could download these 2 files in order to be up to
Furthermore, I noticed by GoogleBot, that certain entries weren't taken into
section "robots", because "User Agent " doesn't match any entry of the file
www.europarl.europa.eu 22.214.171.124 126.96.36.199 200 14075 81855 GET
/sides/getDoc.do?pubRef=-//EP//TEXT+WQ+E-1998-0595+0+DOC+XML+V0//ES HTTP/1.1 -
Mozilla/5.0 (compatible; Googlebot/2.1;
=> this line appears to "Host" section rather than "Robot" section while
"googlebot" is defined in original file robots.pm.
Thanks in advance
Log in to post a comment.