Code review comment for lp://staging/~gary/zc.buildout/support-system-python

I have made a number of changes to zc.buildout in the gary-support-system-python branch in zope.org's svn repository. They all support Launchpad or Landscape's use of Buildout with a system Python, directly or indirectly.

The following is a list of changes taken from my CHANGES.txt additions:

- Support and bugfixes for using a system Python with buildout.

  In all of these descriptions, "site-packages" is an imprecise term for a
  precise definition: the packages that are added by the Python's site.py.
  Practically, this is the difference of the set of paths normally used by
  Python minus those used when Python is started with the -S flag.

  * A new boolean option, 'include-site-packages', includes or excludes site
    packages from finding requirements, and from generated scripts.
    zc.buildout's own buildout.cfg dogfoods this option.

  * Script generation pushes dependency paths that are in site-packages to
    the end of the dependency paths in sys.path (but, as before, these are
    still before extra paths, the standard library, and the rest of the
    site-package paths).

  * Fix an error when at least two dependencies were in a shared location like
    site-packages, and the first one met the "versions" setting. The first
    dependency would be added, but subsequent dependencies from the same
    location (e.g., site-packages) would use the version of the package found
    in the shared location, ignoring the version setting.

- The generated interpreter scripts now have a few more similarities to a
  real Python interpreter.

  * __file__ is set correctly for executed files.

  * -c incorporates all subsequent arguments as part of the command.

  * -V returns the version.

  * -S causes the script to not modify the standard path (for tests)

- Improve bootstrap.

  * New options let you specify where to find ez_setup.py and where to find
    a download cache. These options can keep bootstrap from going over the
    network.

  * Another new option lets you specify where to put generated eggs.

  * The buildout script generated by bootstrap honors more of the settings
    in the designated configuration file (e.g., buildout.cfg).

I failed to implement one feature I had wanted: a way to cherrypick parts of a system Python's site-packages. I proposed an approach, but it ugly, fragile, and risky; and Jim proposed an approach, but it does not work for a modern Ubuntu.

Review requests for small changes to zc.recipe.testrunner and z3c.recipe.filetemplate, based on some of these changes, are also forthcoming.