Merge lp://staging/~sinzui/charms/precise/charmworld/error-notifications into lp://staging/~juju-jitsu/charms/precise/charmworld/trunk

Email exceptions to a list

This branch adds an SMTP logging handler to the production ini/config.
When set the error-email setting. When error-email is not set, the
NullHandler is used to avoid "missing handler" warnings from the logger.

RULES

    Pre-implementation: abentley
    * Staging only send ingest error email to Aaron. Only he can see that
      there is a defect.
    * Devs cannot see any exceptions from the staging app. Aaron's ssh
      key is required to visit the server and read the error logs.
    * We do not want Aaron two be a liability. We do not want to interrupt
      Aaron to diagnose an error
    * Rick suggests using the SMPT handler via the exc_logger that is already
      used by the production ini config.
    * We can extend scripts/write_config to add the SMTP handler when the
      error-email setting is define, and send exceptions to that email
      address.
    * We can create a private team with a mailing list, and use the list
      address as the error-email in the config.

QA

    * Deploy this charm or upgrade a deployment.
    * Instrument an exception --
      Hiding removing, or remapping search's path to /var so that it
      cannot access the index with. For example.
      cd charmworld; HOME=/bogus/path make run
    * Verify that no notification was sent for that time in
      /var/log/mail.log or /var/log/mail.err
    * Configure error-email to test that you receive
      juju set charmworld <email address hidden>
    * Instrument an exception.
    * Verify you get an email with the exception's traceback.

IMPLEMENTATION

There are several points in this implementation that are not ideal. I welcome
advise.

1.Bogus args are set in production_overrides.ini. I chose to use something
that looks like the intent, but the real default is a NullHandler.

2. The from address must be from a legitimate domain; canonical's SMTP doesn't
like instance names. The email contains the instance in the exception, so
I hard coded <email address hidden> as the from address.