Merge lp://staging/~vila/u1-test-utils/lxc into lp://staging/u1-test-utils

I think LXC would be a good default for vm.class.

29 + help='''Where lxc definitions are stored.'''

Why does it have three quotes?

94 + :return: False if the file is in the download cache, True if a download
95 + occurred.

This is more like: True if a download ocurred. False if the file is in the download cache and we are not forcing the download.

137 + def download(self):
138 + raise NotImplementedError(self.download)

Shouldn't this be the same call to wget you do on _download_in_cache?
If this should be implemented on the children classes, I think that would be a better errror message for the exception raised.

526 + # Create an lxc, relying on cloud-init to customize the base image.

This would be better as a docstring.

951 + tests.requires_feature(self, tests.sudo_feature)

This is already on the setUp.

1395 +[lxc1]

What's this lxc machine for? If you are using it for something, it should have a better name.

The branch looks really good. There are things I don't yet understand, from the libraries you are using or from cloud-init. But you have my +1. Next week I hope I will be able to give it a try.

There are additional revisions which have not been approved in review. Please seek review and approval of these new revisions.

> I think LXC would be a good default for vm.class.
>
> 29 + help='''Where lxc definitions are stored.'''
>
> Why does it have three quotes?

So that all helps have 3 quotes.

>
> 94 + :return: False if the file is in the download cache, True if a
> download
> 95 + occurred.
>
> This is more like: True if a download ocurred. False if the file is in the
> download cache and we are not forcing the download.

Well, if we are forcing the download the file can't be in the cache, that's what 'force' do, it clears the cache.

>
> 137 + def download(self):
> 138 + raise NotImplementedError(self.download)
>
> Shouldn't this be the same call to wget you do on _download_in_cache?
> If this should be implemented on the children classes, I think that would be a
> better errror message for the exception raised.

Well, implementing lxc I discovered that lxc-create provides its own caching mechanism that cannot be bypassed.

So the API provided by setup_vm doesn't fit: the download will occur during lxc-create (called by --install) and not during --download.

We probably want to tweak setup_vm instead as I think lxc-create provides a better user experience:
- download if needed when creating a vm and requires it,
- find a way to force a new download if the user requires it (I've yet to see how to get that for lxc)

I filed hhtp://pad.lv/1210428 to track that.

>
> 526 + # Create an lxc, relying on cloud-init to customize the base image.
>
> This would be better as a docstring.

Done.
>
> 951 + tests.requires_feature(self, tests.sudo_feature)
>
> This is already on the setUp.

Fixed. Thanks for catching that one, these tests were changed multiple times
in this proposal and I lost track ;)

>
> 1395 +[lxc1]
>
> What's this lxc machine for? If you are using it for something, it should have
> a better name.

Sorry, that was for manual tests and I should have deleted it before
submitting (or even better, defined it in my ~/vms.conf...). Thanks for the
heads-up.

I'll leave lxc-precise-server-pristine in place for now as an example or
base for future uses but I don't need it myself (comments added to clarify).

>
> The branch looks really good. There are things I don't yet understand, from
> the libraries you are using or from cloud-init. But you have my +1. Next week
> I hope I will be able to give it a try.

Feel free to continue the discussion about the things you don't understand,
that may reveal bugs or at least lack of documentation.

Thanks for the review !

The attempt to merge lp:~vila/u1-test-utils/lxc into lp:u1-test-utils failed. Below is the output from the failed tests.

Setting up the virtual environment.
[localhost] local: which virtualenv
[localhost] local: /usr/bin/python /usr/bin/virtualenv --version
[localhost] local: /usr/bin/python /usr/bin/virtualenv --distribute --clear .env
Not deleting .env/bin
New python executable in .env/bin/python
Installing distribute.............................................................................................................................................................................................done.
Installing pip...............done.
[localhost] local: . /mnt/tarmac/cache/u1-test-utils/trunk/.env/bin/activate && pip install -U -r requirements.txt
Downloading/unpacking bzr+ssh://bazaar.launchpad.net/~bloodearnest/localmail/trunk@36 (from -r requirements.txt (line 9))
  Checking out bzr+ssh://bazaar.launchpad.net/~bloodearnest/localmail/trunk (to revision 36) to /tmp/pip-pJTY95-build
  Running setup.py egg_info for package from bzr+ssh://bazaar.launchpad.net/~bloodearnest/localmail/trunk@36

Downloading/unpacking bzr+http://bazaar.launchpad.net/~ubuntuone-hackers/payclient/trunk@4 (from -r requirements.txt (line 10))
  Checking out http://bazaar.launchpad.net/~ubuntuone-hackers/payclient/trunk (to revision 4) to /tmp/pip-DUKT4G-build
  Running setup.py egg_info for package from bzr+http://bazaar.launchpad.net/~ubuntuone-hackers/payclient/trunk@4

    warning: no previously-included files matching '*.pyc' found anywhere in distribution
Downloading/unpacking bzr+http://bazaar.launchpad.net/~canonical-isd-qa/selenium-simple-test/trunk@tag:sst-0.2.4 (from -r requirements.txt (line 11))
  Checking out http://bazaar.launchpad.net/~canonical-isd-qa/selenium-simple-test/trunk (to revision tag:sst-0.2.4) to /tmp/pip-UsMJJ9-build
  Running setup.py egg_info for package from bzr+http://bazaar.launchpad.net/~canonical-isd-qa/selenium-simple-test/trunk@tag:sst-0.2.4

    warning: no files found matching 'testproj.db.original' under directory 'src/testproject'
Downloading/unpacking bzr+http://bazaar.launchpad.net/~canonical-isd-hackers/canonical-identity-provider/ssoclient@3 (from -r requirements.txt (line 12))
  Checking out http://bazaar.launchpad.net/~canonical-isd-hackers/canonical-identity-provider/ssoclient (to revision 3) to /tmp/pip-HSVF6F-build
  Running setup.py egg_info for package from bzr+http://bazaar.launchpad.net/~canonical-isd-hackers/canonical-identity-provider/ssoclient@3

Downloading/unpacking beautifulsoup4 (from -r requirements.txt (line 2))
  Running setup.py egg_info for package beautifulsoup4

Downloading/unpacking django==1.4 (from -r requirements.txt (line 3))
  Running setup.py egg_info for package django

Downloading/unpacking mock (from -r requirements.txt (line 4))
  Running setup.py egg_info for package mock

    warning: no files found matching '*.png' under directory 'docs'
    warning: no files found matching '*.css' under directory 'docs'
    warning: no files found matching '*.html' under directory 'docs'
    warning: no files found matching '*.js' under directory 'docs'
Downloading...

The attempt to merge lp:~vila/u1-test-utils/lxc into lp:u1-test-utils failed. Below is the output from the failed tests.

Setting up the virtual environment.
[localhost] local: which virtualenv
[localhost] local: /usr/bin/python /usr/bin/virtualenv --version
[localhost] local: /usr/bin/python /usr/bin/virtualenv --distribute --clear .env
Not deleting .env/bin
New python executable in .env/bin/python
Installing distribute.............................................................................................................................................................................................done.
Installing pip...............done.
[localhost] local: . /mnt/tarmac/cache/u1-test-utils/trunk/.env/bin/activate && pip install -U -r requirements.txt
Downloading/unpacking bzr+ssh://bazaar.launchpad.net/~bloodearnest/localmail/trunk@36 (from -r requirements.txt (line 9))
  Checking out bzr+ssh://bazaar.launchpad.net/~bloodearnest/localmail/trunk (to revision 36) to /tmp/pip-oDnmSZ-build
  Running setup.py egg_info for package from bzr+ssh://bazaar.launchpad.net/~bloodearnest/localmail/trunk@36

Downloading/unpacking bzr+http://bazaar.launchpad.net/~ubuntuone-hackers/payclient/trunk@4 (from -r requirements.txt (line 10))
  Checking out http://bazaar.launchpad.net/~ubuntuone-hackers/payclient/trunk (to revision 4) to /tmp/pip-lArRRG-build
  Running setup.py egg_info for package from bzr+http://bazaar.launchpad.net/~ubuntuone-hackers/payclient/trunk@4

    warning: no previously-included files matching '*.pyc' found anywhere in distribution
Downloading/unpacking bzr+http://bazaar.launchpad.net/~canonical-isd-qa/selenium-simple-test/trunk@tag:sst-0.2.4 (from -r requirements.txt (line 11))
  Checking out http://bazaar.launchpad.net/~canonical-isd-qa/selenium-simple-test/trunk (to revision tag:sst-0.2.4) to /tmp/pip-bwzQrD-build
  Running setup.py egg_info for package from bzr+http://bazaar.launchpad.net/~canonical-isd-qa/selenium-simple-test/trunk@tag:sst-0.2.4

    warning: no files found matching 'testproj.db.original' under directory 'src/testproject'
Downloading/unpacking bzr+http://bazaar.launchpad.net/~canonical-isd-hackers/canonical-identity-provider/ssoclient@3 (from -r requirements.txt (line 12))
  Checking out http://bazaar.launchpad.net/~canonical-isd-hackers/canonical-identity-provider/ssoclient (to revision 3) to /tmp/pip-88gtUw-build
  Running setup.py egg_info for package from bzr+http://bazaar.launchpad.net/~canonical-isd-hackers/canonical-identity-provider/ssoclient@3

Downloading/unpacking beautifulsoup4 (from -r requirements.txt (line 2))
  Running setup.py egg_info for package beautifulsoup4

Downloading/unpacking django==1.4 (from -r requirements.txt (line 3))
  Running setup.py egg_info for package django

Downloading/unpacking mock (from -r requirements.txt (line 4))
  Running setup.py egg_info for package mock

    warning: no files found matching '*.png' under directory 'docs'
    warning: no files found matching '*.css' under directory 'docs'
    warning: no files found matching '*.html' under directory 'docs'
    warning: no files found matching '*.js' under directory 'docs'
Downloading...

The attempt to merge lp:~vila/u1-test-utils/lxc into lp:u1-test-utils failed. Below is the output from the failed tests.

Setting up the virtual environment.
[localhost] local: which virtualenv
[localhost] local: /usr/bin/python /usr/bin/virtualenv --version
[localhost] local: /usr/bin/python /usr/bin/virtualenv --distribute --clear .env
Not deleting .env/bin
New python executable in .env/bin/python
Installing distribute.............................................................................................................................................................................................done.
Installing pip...............done.
[localhost] local: . /mnt/tarmac/cache/u1-test-utils/trunk/.env/bin/activate && pip install -U -r requirements.txt
Downloading/unpacking bzr+ssh://bazaar.launchpad.net/~bloodearnest/localmail/trunk@36 (from -r requirements.txt (line 9))
  Checking out bzr+ssh://bazaar.launchpad.net/~bloodearnest/localmail/trunk (to revision 36) to /tmp/pip-KtJOxw-build
  Running setup.py egg_info for package from bzr+ssh://bazaar.launchpad.net/~bloodearnest/localmail/trunk@36

Downloading/unpacking bzr+http://bazaar.launchpad.net/~ubuntuone-hackers/payclient/trunk@4 (from -r requirements.txt (line 10))
  Checking out http://bazaar.launchpad.net/~ubuntuone-hackers/payclient/trunk (to revision 4) to /tmp/pip-l1NQrT-build
  Running setup.py egg_info for package from bzr+http://bazaar.launchpad.net/~ubuntuone-hackers/payclient/trunk@4

    warning: no previously-included files matching '*.pyc' found anywhere in distribution
Downloading/unpacking bzr+http://bazaar.launchpad.net/~canonical-isd-qa/selenium-simple-test/trunk@tag:sst-0.2.4 (from -r requirements.txt (line 11))
  Checking out http://bazaar.launchpad.net/~canonical-isd-qa/selenium-simple-test/trunk (to revision tag:sst-0.2.4) to /tmp/pip-0aXiDS-build
  Running setup.py egg_info for package from bzr+http://bazaar.launchpad.net/~canonical-isd-qa/selenium-simple-test/trunk@tag:sst-0.2.4

    warning: no files found matching 'testproj.db.original' under directory 'src/testproject'
Downloading/unpacking bzr+http://bazaar.launchpad.net/~canonical-isd-hackers/canonical-identity-provider/ssoclient@3 (from -r requirements.txt (line 12))
  Checking out http://bazaar.launchpad.net/~canonical-isd-hackers/canonical-identity-provider/ssoclient (to revision 3) to /tmp/pip-RZdPEc-build
  Running setup.py egg_info for package from bzr+http://bazaar.launchpad.net/~canonical-isd-hackers/canonical-identity-provider/ssoclient@3

Downloading/unpacking beautifulsoup4 (from -r requirements.txt (line 2))
  Running setup.py egg_info for package beautifulsoup4

Downloading/unpacking django==1.4 (from -r requirements.txt (line 3))
  Running setup.py egg_info for package django

Downloading/unpacking mock (from -r requirements.txt (line 4))
  Running setup.py egg_info for package mock

    warning: no files found matching '*.png' under directory 'docs'
    warning: no files found matching '*.css' under directory 'docs'
    warning: no files found matching '*.html' under directory 'docs'
    warning: no files found matching '*.js' under directory 'docs'
Downloading...