Merge lp://staging/~barry/launchpad/325962-makestart into lp://staging/launchpad

= Summary =

Bug 325962 originally described what was thought to be a problem with Mailman
starting up after a stale pid file was left, following an unexpected crash.
After diagnosing the issue with the LOSAs though, it was learned that the
traceback is actually caused by the appserver. The interesting problem there
is that when Mailman is started up on forster, the appserver should /not/ be
starting. The root cause then is that there is no way to inhibit the
appserver from starting.

Note that we still want to use 'make start' to control this because the
initscripts are set up that way, and we may want to start other services.

== Proposed fix ==

Add a 'launch: <bool>' key to the [launchpad] section of the config file, and
read this key to decide whether to start the appserver or not.

== Pre-implementation notes ==

I had a conversation with Curtis and Gary about some related issues, and of
course with Steve who helped diagnose the problem, and describe what is
happening in the production system.

== Implementation details ==

Really, there are only two tricky parts to this branch. The first is the more
important one: what do we do if Zope doesn't start up? runlaunchpad.py acts
like a 'master process'; as long as it's running, all the other processes that
get started continue to run in the background. As soon as runlaunchpad.py
exits though, all the other processes are torn down. Once started, Zope runs
forever until it receives an interrupting signal. If we don't start Zope,
then runlaunchpad.py exits immediately, and immediately tears down all the
other child processes, which is not what we want!

Clearly we don't want to busy-loop when Zope doesn't start up, so I chose a
very simple implementation that infinitely loops in a time.sleep(3600). The
foreground process takes up no CPU for an hour at a time, then it wakes up
only to turn right around and sleep again. This seems like a fine trade-off,
especially because control-c continues to work.

The other tricky problem is that there are no good tests for whether processes
actually start up or not, based on the config file. I opted to test that the
config option works correctly. I didn't add any tests to make sure that
Zope doesn't start up. That kind of negative test is quite problematic
because it would require us to poll check for the non-existence of something
(e.g. a pid file or log entry). It's much harder to test for the
non-existence of something, and poll loops can the cause of spurious
failures. So I opted for manual testing of this aspect.

Note that there will be a parallel change to the production configs to take
advantage of this new option.

== Tests ==

bin/test -vv -t runlaunchpad

== Demo and Q/A ==

 * Edit configs/development/launchpad-lazr.conf
 * Find the [launchpad] section
 * Add 'launch: False'
 * Run 'make run_all'
 * Notice that Zope does not start, but Mailman does
 * Hit C-c
 * Notice that Mailman properly shuts down.

= Launchpad lint =

Checking for conflicts. and issues in doctests and templates.
Running jslint, xmllint, pyflakes, and pylint.
Using normal rules.

Linting changed files:
  lib/canonical/config/schema-lazr.conf
  lib/canonic...

Hi Barry,

Wow, I'm glad I didn't have to debug this. I just have one minor suggestion.

merge-conditional

>=== modified file 'lib/canonical/launchpad/scripts/runlaunchpad.py'
>--- lib/canonical/launchpad/scripts/runlaunchpad.py>---2009-07-23 06:57:17 +0000
>+++ lib/canonical/launchpad/scripts/runlaunchpad.py>---2009-08-12 21:00:49 +0000
>@@ -12,6 +12,7 @@
> import atexit
> import signal
> import subprocess
>+import time
>
> from canonical.config import config
> from canonical.lazr.pidfile import make_pidfile, pidfile_path
>@@ -265,7 +266,17 @@
> # Create the ZCML override file based on the instance.
> config.generate_overrides()
>
>- main(argv)
>+ if config.launchpad.launch:
>+ main(argv)
>+ else:
>+ # We just need the foreground process to sit around forever waiting
>+ # for the signal to shut everything down. Normally, Zope itself would
>+ # be this master process, but we're not starting that up, so we need
>+ # to do something else. An infinite looping sleep of 1h seems fine
>+ # since it won't busyloop and it will exit when a C-c is received.
>+ while True:
>+ time.sleep(3600)

I think that signal.pause() would accomplish the exact same thing and be slightly cleaner.

-Edwin

On Aug 12, 2009, at 09:14 PM, Edwin Grubbs wrote:

>Wow, I'm glad I didn't have to debug this. I just have one minor suggestion.
>
>merge-conditional

Thanks!

>>=== modified file 'lib/canonical/launchpad/scripts/runlaunchpad.py'
>>--- lib/canonical/launchpad/scripts/runlaunchpad.py>---2009-07-23 06:57:17 +0000
>>+++ lib/canonical/launchpad/scripts/runlaunchpad.py>---2009-08-12 21:00:49 +0000
>>@@ -12,6 +12,7 @@
>> import atexit
>> import signal
>> import subprocess
>>+import time
>>
>> from canonical.config import config
>> from canonical.lazr.pidfile import make_pidfile, pidfile_path
>>@@ -265,7 +266,17 @@
>> # Create the ZCML override file based on the instance.
>> config.generate_overrides()
>>
>>- main(argv)
>>+ if config.launchpad.launch:
>>+ main(argv)
>>+ else:
>>+ # We just need the foreground process to sit around forever waiting
>>+ # for the signal to shut everything down. Normally, Zope itself would
>>+ # be this master process, but we're not starting that up, so we need
>>+ # to do something else. An infinite looping sleep of 1h seems fine
>>+ # since it won't busyloop and it will exit when a C-c is received.
>>+ while True:
>>+ time.sleep(3600)
>
>
>I think that signal.pause() would accomplish the exact same thing and be slightly cleaner.

<palm slaps forehead>

Yes of course, how did I forget that!? Thanks. Here's the incremental diff.

=== modified file 'lib/canonical/launchpad/scripts/runlaunchpad.py'
--- lib/canonical/launchpad/scripts/runlaunchpad.py 2009-08-12 19:05:17 +0000
+++ lib/canonical/launchpad/scripts/runlaunchpad.py 2009-08-12 21:39:59 +0000
@@ -12,7 +12,6 @@
 import atexit
 import signal
 import subprocess
-import time

 from canonical.config import config
 from canonical.lazr.pidfile import make_pidfile, pidfile_path
@@ -272,10 +271,11 @@
         # We just need the foreground process to sit around forever waiting
         # for the signal to shut everything down. Normally, Zope itself would
         # be this master process, but we're not starting that up, so we need
- # to do something else. An infinite looping sleep of 1h seems fine
- # since it won't busyloop and it will exit when a C-c is received.
- while True:
- time.sleep(3600)
+ # to do something else.
+ try:
+ signal.pause()
+ except KeyboardInterrupt:
+ pass

 def start_librarian():