Merge lp://staging/~benji/python-pgbouncer/buildout-fixes-and-tweaks into lp://staging/python-pgbouncer

Looks good!

[1]

-# Copyright 2011 Canonical Ltd. This software is licensed under the
+

Was this change an accident?

[2]

I've filed bug 1036876 to get tests working with other versions of PostgreSQL.

*ALL* our projects use those shared cache directories. What benefit does making it harder for the 99% of contributors that also work on other LP stuff serve?

> *ALL* our projects use those shared cache directories. What benefit does
> making it harder for the 99% of contributors that also work on other LP stuff
> serve?

We don't specify them in MAAS. Instead we expect people to create
their own in ~/.buildout (see HACKING.txt). It's less surprising to
omit them from buildout.cfg, because then anyone can build the project
without wondering what's going on:

{{{
While:
  Initializing.
Error: The specified download cache:
'/home/gavin/Launchpad/python-pgbouncer/trunk/download-cache'
Doesn't exist.
}}}

Specifying them makes the project harder to start with for casual
contributors, and probably for some not-so-casual contributors.

In places where we must use the cache, the following would have the
desired effect:

  $ ./bootstrap.py buildout:download-cache=download-cache buildout:eggs=eggs
  $ ./bin/buildout buildout:download-cache=download-cache buildout:eggs=eggs

On Wed, Aug 15, 2012 at 2:02 PM, Gavin Panella
<email address hidden> wrote:
>> *ALL* our projects use those shared cache directories. What benefit does
>> making it harder for the 99% of contributors that also work on other LP stuff
>> serve?
>
> We don't specify them in MAAS. Instead we expect people to create
> their own in ~/.buildout (see HACKING.txt). It's less surprising to
> omit them from buildout.cfg, because then anyone can build the project
> without wondering what's going on:

Right. Buildout has a standard mechanism for shared downloads and eggs
which lets the user opt in or out, designate the locations to their
liking, doesn't require a particular directory structure (the current
pattern in pgbouncer does not work well with the way I structure my
working directories, for example), and, most importantly, doesn't
require inter-project coordination as to where the shared directories
live.

> {{{
> While:
> Initializing.
> Error: The specified download cache:
> '/home/gavin/Launchpad/python-pgbouncer/trunk/download-cache'
> Doesn't exist.
> }}}
>
> Specifying them makes the project harder to start with for casual
> contributors, and probably for some not-so-casual contributors.

A very good point.

> In places where we must use the cache, the following would have the
> desired effect:
>
> $ ./bootstrap.py buildout:download-cache=download-cache buildout:eggs=eggs
> $ ./bin/buildout buildout:download-cache=download-cache buildout:eggs=eggs

When, for deployment reasons, we need a branch for all a project's
dependencies, I tend to favor using a makefile to check out the branch
into the project directory and using a README/HACKING file to direct
developers to use make instead of calling buildout directly.
--
Benji York

On Tue, Aug 14, 2012 at 10:02 PM, Gavin Panella
<email address hidden> wrote:
> -# Copyright 2011 Canonical Ltd. This software is licensed under the
> +
>
> Was this change an accident?

Yep. I think I meant to update the date.

So, I don't follow this argument. What problems does the LP idiom
cause, and why aren't they affecting the 20 other projects we have?

> So, I don't follow this argument. What problems does the LP idiom
> cause, and why aren't they affecting the 20 other projects we have?

Someone checking out lp:python-pgbouncer and running ./bootstrap.py
will get a weird error about download-cache. Same goes for all the
other projects.

We /could/ add download-cache and eggs directories and .bzrignore
their contents, but that doesn't strike me as much of an advantage.

Those people who want a cache can be directed, in a README, to set one
up in ~/.buildout, and those that want to use a specific cache can set
command-line parameters.

I *want* all buildout projects to use my cache in ~/.buildout because
it contains lots of already downloaded and built stuff. I think it's
normally bad form to override these in buildout.cfg.

For larger projects, where deps are specific, patched, or otherwise
not available on PyPI, I can see that it's worth telling people they
must use a specific cache. Likewise when working with release branches
that may be old, where the deps are no longer available from PyPI. But
overriding download-cache in buildout.cfg does not address that,
unless the deps are bundled with the branch.

For small and focused projects like python-pgbouncer, I don't see a
need to bundle dependencies, or provide a separate cache. If a
dependency is no longer available from PyPI that means the package
needs updating. Unless we bundle all its deps, python-pgbouncer is not
very useful when its deps are missing from PyPI. It's the inexorable
march of progress; standing still is falling behind. Again, overriding
download-cache in buildout.cfg does not address this either.

tl;dr
Most of the time download-cache and eggs should be left up to the
user. If a specific cache is needed, bundle the cache in the branch
and point to it in buildout.cfg. Otherwise download-cache is not
intrinsic to the piece of software and should be left as a build-time
option.

On Thursday 16 August 2012 10:08:29 you wrote:
> I want all buildout projects to use my cache in ~/.buildout because
> it contains lots of already downloaded and built stuff.

A thousand times yes.

This is still marked as needing fixing. I would like us to either discuss it more or for it to be approved.

Thanks.

On Tue, Aug 21, 2012 at 1:45 AM, Benji York <email address hidden> wrote:
> This is still marked as needing fixing. I would like us to either discuss it more or for it to be approved.

I don't like the idea of this one project being different. Lets take
this to the LP development list for discussion.

-Rob

Benji, do you want to split that change out into a separate branch, land this, then we can thrash it out on the list?

> Benji, do you want to split that change out into a separate branch, land this,
> then we can thrash it out on the list?

Good idea. Here it is:
https://code.launchpad.net/~benji/python-pgbouncer/tidy-up-dependencies/+merge/120426

This branch has been rejected. See launchpad-dev discussion starting with https://lists.launchpad.net/launchpad-dev/msg09576.html.