Work at SourceForge, help us to make it a better place! We have an immediate need for a Support Technician in our San Francisco or Denver office.

Close

PerlParsingFormat mods for unique log format

Developers
2011-01-24
2012-10-11
  • Martin Bunen
    Martin Bunen
    2011-01-24

    We are using a log appliance that requires different log format than that of
    Apache NCSA.
    The example of the log entry is as following:

    11.222.333.444 - - "GET "www.ourdomain.com" "/referrer/path/to/Action.do"
    "?action=view" HTTP/1.1" 200 7488 "http://www.ourdomain.com/path/to/SomeActio
    n.do?action=view&param1=ABCD&param2=EFGH
    " "Mozilla/5.0 (Windows; U;
    Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 ( .NET CLR
    3.5.30729)" "JSESSIONID=dRRa60ZuoWymVzeI"

    I didn't see appropriate format keys provided by awstats to fit such format.
    We're using awstats v6.95.
    The config file LogFormat is as following:

    LogFormat = "%host %other %logname %time1 %methodhostrefprotquot %code %bytesd
    %urlquot %uaquot %otherquot"

    As a result, I've made some modifications to the awstats.pl and introduced the
    section:
    ...
    elsif ( $f =~ /%methodhostrefprotquot$/ ) {
    $pos_method = $i;
    $i++;
    push @fieldlib, 'method';
    $pos_vh = $i;
    $i++;
    push @fieldlib, 'vhost';
    $pos_referer = $i;
    $i++;
    push @fieldlib, 'referer';
    $PerlParsingFormat .=
    "\\"(+) \\"()\\" \\"()\\" \\"(*)\\"(?: +|)\\"";

    When I run the update the awstats doesn't complain but it also doesn't put
    anything in the resulting .txt file.
    I wonder if the regex is correct for the PerlParsingFormat is correct?

     
  • Martin Bunen
    Martin Bunen
    2011-01-24

    When I posted, I see some semicolons showed up. There are no semicolons in the
    GET string and after URL.

     
  • Jean-Luc
    Jean-Luc
    2011-01-25

    Hi,

    Yep, the text editor of this forum does some tricky things. In your example
    line, there is a double quote before GET and no double quote after it and I do
    not know if you have a referrer field. So it is hard to help this way. Do you
    have a specification of a description of your log format ?

    If some necessary fields are not directly supported by AWStats, you can try to
    use sed to filter the content of the log file before you pass it to AWStats.

    Jean-Luc, InternetOfficer AWStats
    Support

     
  • Martin Bunen
    Martin Bunen
    2011-01-25

    Hi Jean-Luc,
    The format is actually what I described - the entire "GET ... HTTP/1.1" string
    contains strings for virtual host, URL, and query.
    There is a little correction - instead of referer it's URL. The referer is the
    string that comes later.
    It seems that I fixed this problem.
    My new LogFormat string is:

    LogFormat = "%host %other %logname %time1 %methoddomainurlprotquot %code
    %bytesd %refererquot %uaquot %otherquot"

    And the new section in the awstats.pl:

    elsif ( $f =~ /%methoddomainurlprotquot$/ ) {
    $pos_method = $i;
    $i++;
    push @fieldlib, 'method';
    $pos_vh = $i;
    $i++;
    push @fieldlib, 'vhost';
    $pos_url = $i;
    $i++;
    push @fieldlib, 'url';
    $pos_query = $i;
    $i++;
    push @fieldlib, 'query';
    $PerlParsingFormat .=
    "\\"(+) \\"()\\" \\"()\\" \\"(*)\\"(?: +|)\\"";
    }

    Now I'm able to generate stats for this new log format.
    Thanks!