Corrupted Records in 6.95 Revision 1.943

statman14
2011-04-08
2012-10-11
  • statman14
    statman14
    2011-04-08

    Hi,

    I'm migrating an existing AWStats environment form Version 6.95 Revision 1.932
    to a different server that's using Version 6.95 Revision 1.943. The logs are
    being analyzed on both systems in parallel in order to ensure the new
    environment is working correctly. The problem is I'm seeing corrupted records
    on the new server with the new revision, but not the old. Here is the output
    from the analysis (these are old logs, so the "Found 0 new qualified records"
    is expected):

    Version 6.95 Revision 1.932

    Phase 1 : First bypass old records, searching new record...
    Searching new records from beginning of log file...
    Jumped lines in file: 0
    Parsed lines in file: 79478
     Found 0 dropped records,
     Found 0 corrupted records,
     Found 79478 old records,
     Found 0 new qualified records.
    

    Version 6.95 Revision 1.943

    Phase 1 : First bypass old records, searching new record...
    Searching new records from beginning of log file...
    Jumped lines in file: 0
    Parsed lines in file: 79478
     Found 0 dropped records,
     Found 26848 corrupted records,
     Found 52630 old records,
     Found 0 new qualified records.
    

    The analyses that produced the above results were both run on the new server
    (I copied the old revision from the old env for this test). I could proceed
    with using Revision 1.932 instead of 1.943 on the new environment, but I'm
    honestly not sure which revision is correct and I want to understand why I'm
    seeing this discrepancy. I'm using "LogFormat=1" for the configuration. I
    don't understand why one revision is reporting the logs as corrupted and the
    other one is not. Was there a known bug in 1.932 that didn't catch invalid log
    formats that was corrected in 1.943? Here is a line from the log that's marked
    as invalid in 1.943 (this is from a Domino server):

    x.x.x.x server.domain "CN=User/O=org/C=c" "GET /removed/for/obscruity&lang=EN
    HTTP/1.1" 200 7942 "" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1;
    GTB6.6; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; .NET
    CLR 1.1.4322; InfoPath.2)" 65 "" "/also/removed/for/obscurity/blah.nsf"

    Please let me know if you need any additional information in order to assist.
    Thank you!

     
  • statman14
    statman14
    2011-04-08

    I just updated to AWStats Version 7.0 Revision 1.971 and I'm getting the same
    results as I got with 6.95 Revision 1.943. I then verified that the Domino log
    entries we're analyzing don't all match up with the expected syntax for
    LogFormat 1, which explains why the entries were being reported as corrupt.
    This doesn't explain why Revision 1.932 was reporting them as valid entries.
    Based off of this post (here) I've updated the LogFormat to a custom value,
    which now shows all log entries as being processed successfully:

    LogFormat="%host %other %lognamequot %time1 %methodurl %code %bytesd
    %refererquot %uaquot %other %otherquot %otherquot"

    If anyone has anything in addition to add then please feel free to chime in.