Re: [Pyblosxom-devel] error handling, logging, debugging and trouble-shooting

[will kahn-g. <wi...@bl...>]

I've replaced using cgitb with our own crash handler.  It's not
implemented as a sys.excepthook.  Instead it wraps the run_pyblosxom
method innards and thus will work in all PyBlosxom run-time contexts.

Why replace cgitb with our own crash handler?  The big reason was that I
want to enable this thing by default and the amount of information that
cgitb spills out is good for debugging, but too much for being a default
handler since it can potentially leak all kinds of information about
your setup, your configuration, what plugins you're using, ...

The new situation is a lot better and it should make it a lot easier to
trouble-shoot PyBlosxom for people setting up their blogs.  No more
commenting/uncommenting stuff in the pyblosxom.cgi file, no more
unhandled HTTP 500 messages from the web server, no more fishing through
web-server logs for the traceback, ...

I'd be interested in what people think about how it's implemented.

The one thing I'm on the fence about is whether to add a configuration
property that affects how much information shows up in the crash page.
Currently it shows a traceback and that traceback has file paths in it
which tell some information about the blog setup.  It'd be hard to scrub
that data, but it's easy to omit it if the user specifies to in the
configuration.

Is that worth doing?


On 04/02/2011 12:02 PM, will kahn-greene wrote:
> The Summary
> ===========
> 
> There have been several people hopping on IRC that have problems with
> PyBlosxom that are difficult to debug since all they see is a really
> helpful error message kind of like this one:
> 
>    A server error occurred
> 
> 
> Some of us know that that's probably an HTTP 500 error page from the
> web-server.  Other than that, we know nothing about what happened, what
> the circumstances were, or anything else that would be useful to solve
> the problem.
> 
> 
> The problem
> ===========
> 
> cgitb
> -----
> 
> First step I suggest when helping people trouble-shoot is to enable
> cgitb.  There's a line in pyblosxom.cgi that you can uncomment that
> enables this.  Enabling cgitb causes cgitb to bind to the
> sys.excepthook.  When an unhandled exception occurs, then cgitb does its
> thing and shows some useful information.  Several problems with this:
> 
> 1. since it's in pyblosxom.cgi, it's useless for people who don't run
> PyBlosxom as a CGI script
> 
> 2. the information is useful for developers and useless to everyone else
> 
> 3. there's no helpful information about what to do if you encounter a
> problem or even where the problem might lie (core?  plugins? ...)
> 
> 4. you only get the context at the point where PyBlosxom died--that may
> or may not be the origin of the actual problem
> 
> 
> logging
> -------
> 
> Moving along, PyBlosxom also has some logging infrastructure.  Neil has
> been kind enough to go look at this in detail because, as many have
> pointed out, it doesn't actually seem to log anything.  There's a bunch
> of logging infrastructure code in Pyblosxom/tools.py, but neither
> PyBlosxom nor the plugins actually do any logging.  So problems with this:
> 
> 1. nothing seems to do any actual logging
> 
> 2. the logging code doesn't look threadsafe because it's using a global
> cache
> 
> 
> verification
> ------------
> 
> We added some configuration verification stuff to PyBlosxom which you
> can run on the command line.  This enables PyBlosxom to tell you what's
> horribly wrong with your config.py file and also ask plugins to do the
> same.  The problem here is two-fold:
> 
> 1. since it's run from the command line and not the web-server, this
> can't point out any permissions issues
> 
> 2. most plugins don't implement verify_installation
> 
> 
> Solutions?
> ==========
> 
> So, I think we can all read that and come to the same conclusion: the
> situation sucks.  Amongst other things, it makes it really difficult to
> write an effective troubleshooting chapter for the PyBlosxom User Manual.
> 
> I'm tossing around the following things:
> 
> 1. Write our own exception hook that we can bind at the relevant entry
> points.  By doing this, we can be clever about the running context, we
> make the output PyBlosxom-specific, and we can direct people to the
> forums where we can help them.
> 
> 2. Change logging so that it:
> 
> 2.1. always logs to stderr
> 2.2. optionally logs to a log file
> 2.3. has less "clever" infrastructure overhead
> 2.4. keeps track of the last x logged items to display in the except hook
> 
> 3. Somehow rework the verify_installation stuff to be easier.  At a
> minimum, it should verify that required configuration parameters exist
> and this should be easy to add for a plugin developer.
> 
> 
> I bring this up now because I'm increasingly of the mind that problems
> with troubleshooting are awful enough that we should alleviate some of
> them for 1.5.  Amongst other things, it helps us solve a bunch of other
> problems more easily.  Also, I think solution 1 is easy to implement (an
> hour or two) and 2 might be, too.
> 
> Any thoughts on these?  Do these solutions seem like a good idea?
> 
> /will