error handling, logging, debugging and trouble-shooting

The Summary
===========

There have been several people hopping on IRC that have problems with
PyBlosxom that are difficult to debug since all they see is a really
helpful error message kind of like this one:

   A server error occurred


Some of us know that that's probably an HTTP 500 error page from the
web-server.  Other than that, we know nothing about what happened, what
the circumstances were, or anything else that would be useful to solve
the problem.


The problem
===========

cgitb
-----

First step I suggest when helping people trouble-shoot is to enable
cgitb.  There's a line in pyblosxom.cgi that you can uncomment that
enables this.  Enabling cgitb causes cgitb to bind to the
sys.excepthook.  When an unhandled exception occurs, then cgitb does its
thing and shows some useful information.  Several problems with this:

1. since it's in pyblosxom.cgi, it's useless for people who don't run
PyBlosxom as a CGI script

2. the information is useful for developers and useless to everyone else

3. there's no helpful information about what to do if you encounter a
problem or even where the problem might lie (core?  plugins? ...)

4. you only get the context at the point where PyBlosxom died--that may
or may not be the origin of the actual problem


logging
-------

Moving along, PyBlosxom also has some logging infrastructure.  Neil has
been kind enough to go look at this in detail because, as many have
pointed out, it doesn't actually seem to log anything.  There's a bunch
of logging infrastructure code in Pyblosxom/tools.py, but neither
PyBlosxom nor the plugins actually do any logging.  So problems with this:

1. nothing seems to do any actual logging

2. the logging code doesn't look threadsafe because it's using a global
cache


verification
------------

We added some configuration verification stuff to PyBlosxom which you
can run on the command line.  This enables PyBlosxom to tell you what's
horribly wrong with your config.py file and also ask plugins to do the
same.  The problem here is two-fold:

1. since it's run from the command line and not the web-server, this
can't point out any permissions issues

2. most plugins don't implement verify_installation


Solutions?
==========

So, I think we can all read that and come to the same conclusion: the
situation sucks.  Amongst other things, it makes it really difficult to
write an effective troubleshooting chapter for the PyBlosxom User Manual.

I'm tossing around the following things:

1. Write our own exception hook that we can bind at the relevant entry
points.  By doing this, we can be clever about the running context, we
make the output PyBlosxom-specific, and we can direct people to the
forums where we can help them.

2. Change logging so that it:

2.1. always logs to stderr
2.2. optionally logs to a log file
2.3. has less "clever" infrastructure overhead
2.4. keeps track of the last x logged items to display in the except hook

3. Somehow rework the verify_installation stuff to be easier.  At a
minimum, it should verify that required configuration parameters exist
and this should be easy to add for a plugin developer.


I bring this up now because I'm increasingly of the mind that problems
with troubleshooting are awful enough that we should alleviate some of
them for 1.5.  Amongst other things, it helps us solve a bunch of other
problems more easily.  Also, I think solution 1 is easy to implement (an
hour or two) and 2 might be, too.

Any thoughts on these?  Do these solutions seem like a good idea?

/will

------------------------------------------------------------------------------
Create and publish websites with WebMatrix
Use the most popular FREE web apps or write code yourself; 
WebMatrix provides all the features you need to develop and 
publish your website. http://p.sf.net/sfu/ms-webmatrix-sf