I am getting many corrupted records when doing an initial import. It appears
many of the GET logs are not in the right format. I can see that the log
entries do not contain all of the expected fields, but is there some way to
use what is there for the relevant statistics? It should be able to create
unique visitors and hits information from this. The fields are not out of
order, just missing the later log information. About a 1/3 of the total log
lines are being dropped due to this.
Here is the corrupted line error:
Corrupted record line 382 (record format does not match LogFormat parameter):
58.166.74.57 - - "GET /skin/frontend/shambhala/modern/images/logo_email.gif
HTTP/1.1" 200 2699
Here is that line versus another GET log below it.
AWStats cannot process a log file that contains two different formats. You
have to preprocess the file adding fictitious referrer and user-agent fields
before you pass it to AWStats. The fictitious referrer could just be "-". The
fictitious user agent is more complicated as it has to look like a regular
browser like "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.13)
Gecko/20100914 Firefox/3.5.13 (.NET CLR 3.5.30729)" in your example. Without
real user-agent fields, AWStats will not be able to distinguish robot traffic
from human traffic.
I am getting many corrupted records when doing an initial import. It appears
many of the GET logs are not in the right format. I can see that the log
entries do not contain all of the expected fields, but is there some way to
use what is there for the relevant statistics? It should be able to create
unique visitors and hits information from this. The fields are not out of
order, just missing the later log information. About a 1/3 of the total log
lines are being dropped due to this.
Here is the corrupted line error:
Corrupted record line 382 (record format does not match LogFormat parameter):
58.166.74.57 - - "GET /skin/frontend/shambhala/modern/images/logo_email.gif
HTTP/1.1" 200 2699
Here is that line versus another GET log below it.
58.166.74.57 - - "GET /skin/frontend/shambhala/modern/images/logo_email.gif
HTTP/1.1" 200 2699
120.146.67.60 - - "GET /skin/frontend/shambhala/modern/favicon.ico HTTP/1.1"
200 395 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.13)
Gecko/20100914 Firefox/3.5.13 (.NET CLR 3.5.30729)"
Hi,
AWStats cannot process a log file that contains two different formats. You
have to preprocess the file adding fictitious referrer and user-agent fields
before you pass it to AWStats. The fictitious referrer could just be "-". The
fictitious user agent is more complicated as it has to look like a regular
browser like "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.13)
Gecko/20100914 Firefox/3.5.13 (.NET CLR 3.5.30729)" in your example. Without
real user-agent fields, AWStats will not be able to distinguish robot traffic
from human traffic.
Jean-Luc, InternetOfficer AWStats
Support
Do you know why the log file could be being written in two different formats?
Seems like an odd behavior!
Hard to say. Maybe you have several web sites/domains/subdomains (with
different log formats) sharing the same log file ?
Jean-Luc, InternetOfficer AWStats
Support