Re: [VuFind-Tech] Proposal: new error handling functionality

[Demian K. <dem...@vi...>]

This makes sense -- no reason to reinvent the wheel.  I'd propose the PEAR logging framework since we're already using PEAR modules for other purposes, and it seems to contain all the functionality I proposed (and then some).

We could create a web/sys/Logger.php module containing a Logger class to wrap around the PEAR code -- this would read settings from the config file and forward the log message to multiple targets (db, file, email, etc.) if necessary.  We could either create a global $logger instance or else construct a new Logger object whenever we want to log something (I'd prefer the latter approach since I generally dislike globals...  but I could be persuaded in the other direction).

We could then change my previous config file suggestion to something like this:

[Errors]
dump_backtrace              = true                    ; I still think it would be useful to have a way to turn this off

[Logging]
level                                      = 7                          ; Log level should correspond w/ PEAR constants (0-7, comment out for no logging)
database                             = table_name
file                                          = /path/to/file.log
email                                     = vuf...@my...

Does this sound reasonable?  Let me know what you think, and in the meantime I'll start playing with the framework to be sure it's as useful as it looks.

- Demian

From: Franck Borel [mailto:fra...@gb...]
Sent: Wednesday, August 05, 2009 2:09 AM
To: vuf...@li...
Subject: Re: [VuFind-Tech] Proposal: new error handling functionality

Hi all,

logging is an important thing for the development process and to help users to configure their system. If possible we should use a logging framework, for example log4php (http://logging.apache.org/log4php/) or the pear logging framework (http://pear.php.net/package/Log).

Have a nice day

-- Franck


We opted for simple file logging like Till. And built an equally simple browsing mechanism to view them. I've removed the security on our test server logs if you want to see.

Cause an error: (page/sort mangling)
http://libcat-test.usq.edu.au/Search/Home?lookfor=&type=all&submit=Find&sort=title&&sort=callnumberpage=1

Browse error logs:
http://libcat-test.usq.edu.au/error_logs.php

Find the latest one:
http://libcat-test.usq.edu.au/error_logs.php?error_id=1249427851_0

I don't clean out the logs on our test server very often, but do on prod, usually with a simple shell scripts at the back end:

======
vufind@libcat-test<mailto:vufind@libcat-test> cat find_and_delete_logs.sh
#!/bin/bash

# Quick and dirty error log purge
# delete all error logs containing a string

echo ""
echo ""

if [ $# == 0 ];
  then
    echo "No Input!"
    exit
fi
if [ $# != 1 ];
  then
    echo "Invalid Input!"
    echo "Only one search parameter is accepted, remember to quote phrases."
    exit
fi

find /error_logs/ -type f -exec grep "$1" {} \; -exec rm -f {} +
======

That way I can purge all errors from my own desktop during testing like so:

./find_and_delete_logs.sh "139.86.201.50"

We originally looked at a database table as well but abandoned the idea in the end because we couldn't see much need for it. The most annoying bugs that recur we can:

* Purge from a single IP because they are script kiddie attacks looking for phpmyadmin or such (I actually forward these to our security before purging, he likes test cases for his IDS) OR
* We eliminate straight away with a bug fix OR
* I have a wad of printouts on my desk of all the weird characters and url mangling that causes crashes. Those can be (hopefully) be swept away by a single codebase that deals with search handling/link rendering (TODO list).




Greg Pendlebury
Electronic Services Officer (Systems Team)
Division of Academic Information Services
University of Southern Queensland
Phone: +61 7 4631 1501
Fax: +61 7 4631 1841


________________________________
From: Demian Katz [mailto:dem...@vi...]
Sent: Wednesday, 5 August 2009 4:17 AM
To: vuf...@li...<mailto:vuf...@li...>
Subject: [VuFind-Tech] Proposal: new error handling functionality
I've only been working on VuFind for a few weeks, and I've already fixed a number of edge case bugs where strange records caused VuFind to display an error screen.  Without users reporting these errors, I would never have noticed that anything was wrong.

Librarians are good about reporting errors, but I'm sure that students are more inclined to ignore them, and search engine crawlers are certainly not going to tell us about them.  To allow a more active approach to dealing with problems, I propose adding a new section to the config.ini file:

; This section handles how your system deals with unexpected fatal errors.  You
; may uncomment multiple options to track errors in multiple ways.  This is
; recommended, since some error conditions (disk full, database outage) may
; prevent certain logging mechanisms from working correctly.
;
; The dump_backtrace option controls whether a backtrace is displayed as
; part of the error page displayed to the end user.  To further customize the
; way users see errors, you can edit the error.tpl file in your theme.
;
; db_log is a table in your MySQL database for logging errors.  To create
; the table, use this SQL:
; CREATE TABLE errors(id SERIAL, details TEXT, logged TIMESTAMP DEFAULT now());
;
; file_log is a file path for dumping errors.  Apache must have write access to
; this file.
;
; email is an email address to send error alerts to.  Every error will send a
; separate email, so be careful with this feature!
[Errors]
dump_backtrace  = true
;db_log         = errors
;file_log       = /usr/local/vufind/error.log
;email          = vuf...@my...<mailto:vuf...@my...>

There's a lot of room for making this fancier -- collating error emails together, filtering based on error types, etc.  For a start, though, I propose doing this the simplest possible way.  I think it will be valuable to have a better view into unexpected failures of VuFind, and once we start looking at the results of this, we may have better ideas about how to add functionality in the future.

I plan to start implementing the basics of this tomorrow -- if you have any major thoughts or concerns, please let me know before then.

thanks,
Demian
________________________________
This email (including any attached files) is confidential and is for the intended recipient(s) only. If you received this email by mistake, please, as a courtesy, tell the sender, then delete this email.

The views and opinions are the originator's and do not necessarily reflect those of the University of Southern Queensland. Although all reasonable precautions were taken to ensure that this email contained no viruses at the time it was sent we accept no liability for any losses arising from its receipt.

The University of Southern Queensland is a registered provider of education with the Australian Government (CRICOS Institution Code No's. QLD 00244B / NSW 02225M) ------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july_______________________________________________
Vufind-tech mailing list
Vuf...@li...<mailto:Vuf...@li...>
https://lists.sourceforge.net/lists/listinfo/vufind-tech