Merge lp://staging/~gary/launchpad/lazr_uri into lp://staging/launchpad

This seemingly innocuous branch hides a fair amount of code churn under the surface.

The point of this branch is to eliminate the build problems on Karmic--or more specifically, when lazr.uri was installed by apt.

The problem was caused by the lazr.uri setup.py doing a sys.path hack in the setup.py so that it could import __version__ from its own __init__.py. In the case of lazr.uri, that caused it to import apt's version of the __version__, and so misreport its own version, which halted the build.

The right fix is to make buildout not allow this confusion, if possible. However, a quick fix was to use a newer, already existing version of lazr.uri (1.0.1 -> 1.0.2) because the newer one did not do the sys.path-in-setup.py trick. I tried that first. That did fix the problem for lazr.uri, but then lazr.smtptest, which does a similar trick, fell over for a related reason that I won't get into unless asked.

At that point, I decided to try to change buildout to do the right thing.

This coincided in timing with my trying to bring my zc.buildout branch back up to trunk so that it could be reviewed and merged upstream. It was a good time to work on the two problems, because the merge was going to make me rewrite a part of the code that fixing the sys.path-in-setup.py problem would touch.

Therefore, I merged zc.buildout's trunk with my branch, and in the course of resolving conflicts, rewrote one part of it to accommodate both concerns. The selected versions of zc.buildout, zc.recipe.egg, and z3c.recipe.filetemplate come from this effort. The diff of this effort is thousands of lines, and in zc.buildout's case necessarily merged with the changes from upstream's trunk.

For lack of an idea of what better to do, I include the test for the specific problem from zc.buildout below.

Thank you!

Gary

If you use a Python that has a different version of one of your
dependencies, and the new package tries to do sys.path tricks in the
setup.py to get a __version__, and it uses namespace packages, the older
package will be loaded first, making the setup version the wrong number.
While very arguably packages simply shouldn't do this, some do, and we
don't want buildout to fall over when they do.

To demonstrate this, we will need to create a distribution that has one of
these unpleasant tricks, and a Python that has an older version installed.

    >>> for i in (0, 1):
    ... tmp = tempfile.mkdtemp()
    ... try:
    ... write(tmp, 'README.txt', '')
    ... mkdir(tmp, 'src')
    ... mkdir(tmp, 'src', 'tellmy')
    ... write(tmp, 'src', 'tellmy', '__init__.py',
    ... "__import__("
    ... "'pkg_resources').declare_namespace(__name__)\n")
    ... mkdir(tmp, 'src', 'tellmy', 'version')
    ... write(tmp, 'src', 'tellmy', 'version',
    ... '__init__.py', '__version__="1.%d"\n' % i)
    ... write(
    ... tmp, 'setup.py',
    ... "from setuptools import setup\n"
    ... "import sys\n"
    ... "sys.path.insert(0, 'src')\n"
    ... "from tellmy.version import __version__\n"
    ... ...

On IRC, I only asked that the for i in (0,1): part be changed to be more clear about the intent.

Otherwise, r=me.