Learn how easy it is to sync an existing GitHub or Google Code repo to a SourceForge project! See Demo

Close

Default Install, log parse issue, any help?

DrunkMunki
2010-02-19
2013-04-24
  • DrunkMunki
    DrunkMunki
    2010-02-19

    On a default install the cron jobs are running fine, but the mysar-importer.php when executed wont insert tables as shown below;

    epservosx:bin ep$ php mysar-importer.php
    PHP Warning:  PHP Startup: gd: Unable to initialize module
    Module compiled with module API=20060613
    PHP    compiled with module API=20090626
    These options need to match
    in Unknown on line 0
    __
    ERROR on SQL query
    SQL query: INSERT INTO traffic(date,time,ip,resultCode,bytes,url,authuser) VALUES ('1970-01-01','10:03:12',INET_ATON('-'),'','http://mt0.google.com/vt/lyrs=m@118&hl=en&src=api&x=15084&y=9799&z=14&s=Gal','HTTP/1.1\"')
    Database error number: 1048
    Database error message: Column 'ip' cannot be null

    I have figured its somehow related to the ip not being parsed correctly in the logfile.

    in my squid.conf file i have:

    logformat combined %>a %ui %un  "%rm %ru HTTP/%rv" %Hs %<st "%{Referer}>h" "%{User-Agent}>h" %Ss:%Sh

    which looks like this

    192.168.2.149 - -  "GET http://www.theberesford.com.au/TheBeresfordHotel.jpg HTTP/1.1" 200 212933 "http://www.theberesford.com.au/" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.04506.30; .NET CLR 3.0.04506.648; .NET CLR 3.5.21022; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)" TCP_MISS:DIRECT

    I have included the debug output below ot help anyone deduce how i can fix this.

    epservosx:bin ep$ php mysar-importer.php
    PHP Warning:  PHP Startup: gd: Unable to initialize module
    Module compiled with module API=20060613
    PHP    compiled with module API=20090626
    These options need to match
    in Unknown on line 0

    /opt/local/www/mysar/inc/common.inc.php(38) -> Disabling PHP's execution time limit…Yes
    Initializing database connection…
    dbHost=localhost,dbUser=mysar,dbPass=*****
    Done.
    Selecting database…
    dbName=mysar
    Done.
    Start timestamp is 1266545644
    Configuration:Array
    (
         => mysar
         => emerald
         => localhost
         => mysar
    )

    (SELECT value FROM config WHERE name='mysarImporter')
    Getting last proccessed record timestamp…
    (SELECT value FROM config WHERE name='lastTimestamp')0
    (SELECT value FROM config WHERE name='squidLogPath')
    Opening log file/var/log/squid/access.log…
    OK
    Reading the last log offset…
    (SELECT value FROM config WHERE name='lastLogOffset')
    83
    Reading the first log file line…
    Array
    (
         => Feb
         => 18
         => 18:00:01
         => epservosx
         => newsyslog:
         => logfile
         => turned
         => over
         => due
         => to
         => size>10240K
         =>
    )

    (SELECT value FROM config WHERE name='firstLogTimestamp')
    This is a known log file. Skipping to offset 83…OK
    Current file offset: 83
    Now timestamp is: 1266545644. Script start was at: 1266545644
    Checking if run time exceeded 2222255 seconds…NO
    Reading one line from log file…OK
    Record received:
    Array
    (
         => 192.168.2.149
         => -
         => -
         => [18/Feb/2010:18:00:21
         => +1100]
         => "GET
         => http://mt0.google.com/vt/lyrs=m@118&hl=en&src=api&x=15084&y=9799&z=14&s=Gal
         => HTTP/1.1"
         => 200
         => 14620
         => "http://ora4syd.site.net.au/ffcc.PROD/appointment/show_recommended_route_information.php?iPlumberId=0&sScheduledDate=200&sScheduleSlot=15:00&iCustomerId=0"
         => "Mozilla/4.0
         => (compatible;
         => MSIE
         => 7.0;
         => Windows
         => NT
         => 5.1;
         => .NET
         => CLR
         => 1.1.4322;
         => .NET
         => CLR
         => 2.0.50727;
         => .NET
         => CLR
         => 3.0.04506.30;
         => .NET
         => CLR
         => 3.0.04506.648;
         => .NET
         => CLR
         => 3.5.21022;
         => .NET
         => CLR
         => 3.0.4506.2152;
         => .NET
         => CLR
         => 3.5.30729)"
         => TCP_MEM_HIT:NONE
         =>
    )

    Checking if it already processed…NO_
    Parsing record…
    Array
    (
         => 1970-01-01
         => 10:03:12
         => 10
         => -
         => [18/Feb/2010:18:00:21
         => +1100]
         => http://mt0.google.com/vt/lyrs=m@118&hl=en&src=api&x=15084&y=9799&z=14&s=Gal
         => HTTP/1.1\"
         => outCache
    )

    Checking if this is a zero-bytes record…NO
    Parsing url…Array
    (
         => http
         => mt0.google.com
         => /vt/lyrs=m@118&hl=en&src=api&x=15084&y=9799&z=14&s=Gal
    )

    Inserting raw record into the database…(INSERT INTO traffic(date,time,ip,resultCode,bytes,url,authuser) VALUES ('1970-01-01','10:03:12',INET_ATON('-'),'','http://mt0.google.com/vt/lyrs=m@118&hl=en&src=api&x=15084&y=9799&z=14&s=Gal','HTTP/1.1\"'))
    ERROR on SQL query
    SQL query: INSERT INTO traffic(date,time,ip,resultCode,bytes,url,authuser) VALUES ('1970-01-01','10:03:12',INET_ATON('-'),'','http://mt0.google.com/vt/lyrs=m@118&hl=en&src=api&x=15084&y=9799&z=14&s=Gal','HTTP/1.1\"')
    Database error number: 1048
    Database error message: Column 'ip' cannot be null
    Exiting…epservosx:bin ep$ vi mysar-importer.php

     
  • DrunkMunki
    DrunkMunki
    2010-02-24

    i take it no one actively develops this any more? shame