Merge into trunk : releasing-timeout-bug-1390144 : Code : MAAS

Status:

Merged

Approved by:

Julian Edwards on 2014-11-11

Approved revision:

no longer in the source branch.

Merged at revision:

3359

Proposed branch:

lp://staging/~julian-edwards/maas/releasing-timeout-bug-1390144

Merge into:

lp://staging/~maas-committers/maas/trunk

Diff against target:

182 lines (+42/-0)

4 files modified

src/maasserver/api/tests/test_node.py (+4/-0)
src/maasserver/models/node.py (+9/-0)
src/maasserver/models/tests/test_node.py (+27/-0)
src/maasserver/tests/test_node_action.py (+2/-0)

To merge this branch:

bzr merge lp://staging/~julian-edwards/maas/releasing-timeout-bug-1390144

High

Fix Released

Link a bug report

Reviewer	Review Type	Date Requested	Status
Graham Binns (community)		2014-11-10	Approve on 2014-11-10
Review via email: mp+241231@code.staging.launchpad.net

Commit message

Use the transition monitor to put a timeout on RELEASING.

Description of the change

There's a lot of collateral test damage fixed here, as was the case with the similar change at r3332. I am deliberately not adding more of the *_CONNECT stuff because I want us to discuss in more detail a coherent testing strategy for anything involving cluster comms, because it's just too easy to lose test isolation as things stand.

Revision history for this message

Graham Binns (gmb) on 2014-11-10:

#

review: Approve

Revision history for this message

MAAS Lander (maas-lander) wrote on 2014-11-10:

#

Download full text (32.4 KiB)

The attempt to merge lp:~julian-edwards/maas/releasing-timeout-bug-1390144 into lp:maas failed. Below is the output from the failed tests.

Ign http://security.ubuntu.com trusty-security InRelease
Ign http://nova.clouds.archive.ubuntu.com trusty InRelease
Hit http://security.ubuntu.com trusty-security Release.gpg
Ign http://nova.clouds.archive.ubuntu.com trusty-updates InRelease
Hit http://security.ubuntu.com trusty-security Release
Hit http://nova.clouds.archive.ubuntu.com trusty Release.gpg
Get:1 http://nova.clouds.archive.ubuntu.com trusty-updates Release.gpg [933 B]
Hit http://nova.clouds.archive.ubuntu.com trusty Release
Get:2 http://nova.clouds.archive.ubuntu.com trusty-updates Release [62.0 kB]
Hit http://security.ubuntu.com trusty-security/main Sources
Hit http://security.ubuntu.com trusty-security/universe Sources
Hit http://nova.clouds.archive.ubuntu.com trusty/main Sources
Hit http://security.ubuntu.com trusty-security/main amd64 Packages
Hit http://nova.clouds.archive.ubuntu.com trusty/universe Sources
Hit http://security.ubuntu.com trusty-security/universe amd64 Packages
Hit http://nova.clouds.archive.ubuntu.com trusty/main amd64 Packages
Hit http://security.ubuntu.com trusty-security/main Translation-en
Hit http://nova.clouds.archive.ubuntu.com trusty/universe amd64 Packages
Hit http://security.ubuntu.com trusty-security/universe Translation-en
Hit http://nova.clouds.archive.ubuntu.com trusty/main Translation-en
Hit http://nova.clouds.archive.ubuntu.com trusty/universe Translation-en
Ign http://nova.clouds.archive.ubuntu.com trusty/main Translation-en_US
Ign http://nova.clouds.archive.ubuntu.com trusty/universe Translation-en_US
Get:3 http://nova.clouds.archive.ubuntu.com trusty-updates/main Sources [136 kB]
Get:4 http://nova.clouds.archive.ubuntu.com trusty-updates/universe Sources [89.3 kB]
Get:5 http://nova.clouds.archive.ubuntu.com trusty-updates/main amd64 Packages [356 kB]
Get:6 http://nova.clouds.archive.ubuntu.com trusty-updates/universe amd64 Packages [217 kB]
Hit http://nova.clouds.archive.ubuntu.com trusty-updates/main Translation-en
Hit http://nova.clouds.archive.ubuntu.com trusty-updates/universe Translation-en
Fetched 861 kB in 2s (307 kB/s)
Reading package lists...
sudo DEBIAN_FRONTEND=noninteractive apt-get -y \
--no-install-recommends install apache2 authbind bind9 bind9utils build-essential bzr-builddeb curl daemontools debhelper dh-apport distro-info dnsutils firefox freeipmi-tools gjs ipython isc-dhcp-common libjs-raphael libjs-yui3-full libjs-yui3-min libpq-dev make pep8 postgresql pyflakes python-amqplib python-bzrlib python-celery python-convoy python-crochet python-cssselect python-curtin python-dev python-distro-info python-django python-django-piston python-django-south python-djorm-ext-pgarray python-docutils python-extras python-fixtures python-flake8 python-formencode python-hivex python-httplib2 python-jinja2 python-jsonschema python-lockfile python-lxml python-mimeparse python-mock python-netaddr python-netifaces python-nose python-oauth python-oops python-oops-amqp python-oops-datedir-repo python-oops-twisted python-oops-wsgi python-openssl python-paramiko python-pexpect python-pip python-poc...

The attempt to merge lp:~julian-edwards/maas/releasing-timeout-bug-1390144 into lp:maas failed. Below is the output from the failed tests.

Ign http://security.ubuntu.com trusty-security InRelease
Ign http://nova.clouds.archive.ubuntu.com trusty InRelease
Hit http://security.ubuntu.com trusty-security Release.gpg
Ign http://nova.clouds.archive.ubuntu.com trusty-updates InRelease
Hit http://security.ubuntu.com trusty-security Release
Hit http://nova.clouds.archive.ubuntu.com trusty Release.gpg
Get:1 http://nova.clouds.archive.ubuntu.com trusty-updates Release.gpg [933 B]
Hit http://nova.clouds.archive.ubuntu.com trusty Release
Get:2 http://nova.clouds.archive.ubuntu.com trusty-updates Release [62.0 kB]
Hit http://security.ubuntu.com trusty-security/main Sources
Hit http://security.ubuntu.com trusty-security/universe Sources
Hit http://nova.clouds.archive.ubuntu.com trusty/main Sources
Hit http://security.ubuntu.com trusty-security/main amd64 Packages
Hit http://nova.clouds.archive.ubuntu.com trusty/universe Sources
Hit http://security.ubuntu.com trusty-security/universe amd64 Packages
Hit http://nova.clouds.archive.ubuntu.com trusty/main amd64 Packages
Hit http://security.ubuntu.com trusty-security/main Translation-en
Hit http://nova.clouds.archive.ubuntu.com trusty/universe amd64 Packages
Hit http://security.ubuntu.com trusty-security/universe Translation-en
Hit http://nova.clouds.archive.ubuntu.com trusty/main Translation-en
Hit http://nova.clouds.archive.ubuntu.com trusty/universe Translation-en
Ign http://nova.clouds.archive.ubuntu.com trusty/main Translation-en_US
Ign http://nova.clouds.archive.ubuntu.com trusty/universe Translation-en_US
Get:3 http://nova.clouds.archive.ubuntu.com trusty-updates/main Sources [136 kB]
Get:4 http://nova.clouds.archive.ubuntu.com trusty-updates/universe Sources [89.3 kB]
Get:5 http://nova.clouds.archive.ubuntu.com trusty-updates/main amd64 Packages [356 kB]
Get:6 http://nova.clouds.archive.ubuntu.com trusty-updates/universe amd64 Packages [217 kB]
Hit http://nova.clouds.archive.ubuntu.com trusty-updates/main Translation-en
Hit http://nova.clouds.archive.ubuntu.com trusty-updates/universe Translation-en
Fetched 861 kB in 2s (307 kB/s)
Reading package lists...
sudo DEBIAN_FRONTEND=noninteractive apt-get -y \
	    --no-install-recommends install apache2 authbind bind9 bind9utils build-essential bzr-builddeb curl daemontools debhelper dh-apport distro-info dnsutils firefox freeipmi-tools gjs ipython isc-dhcp-common libjs-raphael libjs-yui3-full libjs-yui3-min libpq-dev make pep8 postgresql pyflakes python-amqplib python-bzrlib python-celery python-convoy python-crochet python-cssselect python-curtin python-dev python-distro-info python-django python-django-piston python-django-south python-djorm-ext-pgarray python-docutils python-extras python-fixtures python-flake8 python-formencode python-hivex python-httplib2 python-jinja2 python-jsonschema python-lockfile python-lxml python-mimeparse python-mock python-netaddr python-netifaces python-nose python-oauth python-oops python-oops-amqp python-oops-datedir-repo python-oops-twisted python-oops-wsgi python-openssl python-paramiko python-pexpect python-pip python-pocket-lint python-psycopg2 python-pyinotify python-seamicroclient python-simplejson python-simplestreams python-sphinx python-subunit python-tempita python-testresources python-testscenarios python-testtools python-twisted python-txamqp python-txtftp python-tz python-virtualenv python-yaml python-zope.interface syslinux-common tgt ubuntu-cloudimage-keyring wget xvfb
Reading package lists...
Building dependency tree...
Reading state information...
authbind is already the newest version.
bind9 is already the newest version.
bind9utils is already the newest version.
build-essential is already the newest version.
debhelper is already the newest version.
distro-info is already the newest version.
dnsutils is already the newest version.
isc-dhcp-common is already the newest version.
libjs-raphael is already the newest version.
libjs-yui3-full is already the newest version.
libjs-yui3-min is already the newest version.
make is already the newest version.
pep8 is already the newest version.
postgresql is already the newest version.
pyflakes is already the newest version.
python-amqplib is already the newest version.
python-celery is already the newest version.
python-convoy is already the newest version.
python-crochet is already the newest version.
python-curtin is already the newest version.
python-dev is already the newest version.
python-distro-info is already the newest version.
python-django-piston is already the newest version.
python-django-south is already the newest version.
python-djorm-ext-pgarray is already the newest version.
python-docutils is already the newest version.
python-extras is already the newest version.
python-fixtures is already the newest version.
python-formencode is already the newest version.
python-httplib2 is already the newest version.
python-jinja2 is already the newest version.
python-jsonschema is already the newest version.
python-lockfile is already the newest version.
python-mimeparse is already the newest version.
python-mock is already the newest version.
python-netaddr is already the newest version.
python-netifaces is already the newest version.
python-nose is already the newest version.
python-oauth is already the newest version.
python-oops is already the newest version.
python-oops-amqp is already the newest version.
python-oops-datedir-repo is already the newest version.
python-oops-twisted is already the newest version.
python-oops-wsgi is already the newest version.
python-openssl is already the newest version.
python-paramiko is already the newest version.
python-psycopg2 is already the newest version.
python-pyinotify is already the newest version.
python-simplejson is already the newest version.
python-subunit is already the newest version.
python-tempita is already the newest version.
python-testresources is already the newest version.
python-testscenarios is already the newest version.
python-testtools is already the newest version.
python-twisted is already the newest version.
python-txamqp is already the newest version.
python-txtftp is already the newest version.
python-tz is already the newest version.
python-yaml is already the newest version.
python-zope.interface is already the newest version.
syslinux-common is already the newest version.
tgt is already the newest version.
ubuntu-cloudimage-keyring is already the newest version.
bzr-builddeb is already the newest version.
daemontools is already the newest version.
ipython is already the newest version.
python-cssselect is already the newest version.
python-flake8 is already the newest version.
python-hivex is already the newest version.
python-pip is already the newest version.
python-pocket-lint is already the newest version.
python-virtualenv is already the newest version.
apache2 is already the newest version.
curl is already the newest version.
dh-apport is already the newest version.
firefox is already the newest version.
freeipmi-tools is already the newest version.
libpq-dev is already the newest version.
python-bzrlib is already the newest version.
python-django is already the newest version.
python-lxml is already the newest version.
python-pexpect is already the newest version.
python-seamicroclient is already the newest version.
python-simplestreams is already the newest version.
python-sphinx is already the newest version.
wget is already the newest version.
xvfb is already the newest version.
gjs is already the newest version.
0 upgraded, 0 newly installed, 0 to remove and 18 not upgraded.
sudo DEBIAN_FRONTEND=noninteractive apt-get -y \
	    purge 
Reading package lists...
Building dependency tree...
Reading state information...
0 upgraded, 0 newly installed, 0 to remove and 18 not upgraded.
make -C acceptance clean
make[1]: Entering directory `/tmp/tarmac/branch.fa9NZm/acceptance'
rm -f -r source build *.build *.changes *.deb
make[1]: Leaving directory `/tmp/tarmac/branch.fa9NZm/acceptance'
find . -type f -name '*.py[co]' -print0 | xargs -r0 rm -f
find . -type f -name '*~' -print0 | xargs -r0 rm -f
find . -type f -name dropin.cache -print0 | xargs -r0 rm -f
rm -f -r media/demo/* media/development
rm -f src/maasserver/static/js/enums.js
rm -f *.log
rm -f docs/api.rst
rm -f -r docs/_autosummary docs/_build
rm -f -r man/.doctrees
rm -f -r bin include lib local
rm -f -r eggs develop-eggs
rm -f -r build dist logs/* parts
rm -f tags TAGS .installed.cfg
rm -f -r *.egg *.egg-info src/*.egg-info
rm -f -r run/* services/*/supervise
virtualenv --python=python2.7 --system-site-packages /tmp/tarmac/branch.fa9NZm
New python executable in /tmp/tarmac/branch.fa9NZm/bin/python2.7
Also creating executable in /tmp/tarmac/branch.fa9NZm/bin/python
Installing setuptools, pip...done.
Running virtualenv with interpreter /usr/bin/python2.7
bin/python -m pip --quiet install --ignore-installed \
	    --no-dependencies bootstrap/zc.buildout-1.5.2.tar.gz
rm -f -f README.txt  # zc.buildout installs an annoying README.txt.
bin/buildout install database
Creating directory '/tmp/tarmac/branch.fa9NZm/parts'.
Creating directory '/tmp/tarmac/branch.fa9NZm/develop-eggs'.
Installing database.
Generated script '/tmp/tarmac/branch.fa9NZm/bin/database'.
bin/buildout install repl
Installing repl.
Generated script '/tmp/tarmac/branch.fa9NZm/bin/ipy'.
Generated interpreter '/tmp/tarmac/branch.fa9NZm/bin/py'.
bin/py -m src/maasserver/utils/jsenums src/maasserver/enum.py src/metadataserver/enum.py > src/maasserver/static/js/enums.js
bin/buildout install maas
Installing maas.
Generated script '/tmp/tarmac/branch.fa9NZm/bin/maas-region-admin'.
bin/buildout install maas-test
Installing maas-test.
Generated script '/tmp/tarmac/branch.fa9NZm/bin/test.maas'.
bin/buildout install maascli
Installing maascli.
Generated script '/tmp/tarmac/branch.fa9NZm/bin/maas'.
bin/buildout install maascli-test
Installing maascli-test.
Generated script '/tmp/tarmac/branch.fa9NZm/bin/test.maascli'.
bin/buildout install maastesting-test
Installing maastesting-test.
Generated script '/tmp/tarmac/branch.fa9NZm/bin/test.maastesting'.
bin/buildout install pserv
Installing pserv.
Generated script '/tmp/tarmac/branch.fa9NZm/bin/maas-provision'.
Generated script '/tmp/tarmac/branch.fa9NZm/bin/twistd.pserv'.
bin/buildout install pserv-test
Installing pserv-test.
Generated script '/tmp/tarmac/branch.fa9NZm/bin/test.pserv'.
bin/buildout install config-test
Installing config-test.
Generated script '/tmp/tarmac/branch.fa9NZm/bin/test.config'.
bin/buildout install maas-probe-dhcp
Installing maas-probe-dhcp.
Generated script '/tmp/tarmac/branch.fa9NZm/bin/maas-probe-dhcp'.
make -C acceptance clean
make[1]: Entering directory `/tmp/tarmac/branch.fa9NZm/acceptance'
rm -f -r source build *.build *.changes *.deb
make[1]: Leaving directory `/tmp/tarmac/branch.fa9NZm/acceptance'
find . -type f -name '*.py[co]' -print0 | xargs -r0 rm -f
find . -type f -name '*~' -print0 | xargs -r0 rm -f
find . -type f -name dropin.cache -print0 | xargs -r0 rm -f
rm -f -r media/demo/* media/development
rm -f src/maasserver/static/js/enums.js
rm -f *.log
rm -f docs/api.rst
rm -f -r docs/_autosummary docs/_build
rm -f -r man/.doctrees
bin/py -m src/maasserver/utils/jsenums src/maasserver/enum.py src/metadataserver/enum.py > src/maasserver/static/js/enums.js
bin/buildout install maas
Updating maas.
bin/buildout install maas-test
Updating maas-test.
echo bin/test.config bin/test.maas bin/test.maascli bin/test.maastesting bin/test.pserv | xargs -n1 env
nosetests --exclude=provisioningserver --exclude=maastesting --exclude=maascli --logging-level=INFO --verbosity=1
Creating test database for alias 'default'...
Destroying test database for alias 'default'...

..............................................
----------------------------------------------------------------------
Ran 46 tests in 0.327s

OK
..S.............................................................................................................................................................................................................................................................................................................................................................................................................................FF.....FF...............................................................................................................................................................S..........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................S.............................................................................................................................................................S.................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................SS........................................................................................................................................................................................
======================================================================
FAIL: maasserver.api.tests.test_node.TestNodeAPI.test_POST_release_allows_admin_to_release_anyones_node
----------------------------------------------------------------------
_StringException: Traceback (most recent call last):
  File "/tmp/tarmac/branch.fa9NZm/src/maasserver/api/tests/test_node.py", line 511, in test_POST_release_allows_admin_to_release_anyones_node
    self.assertEqual(httplib.OK, response.status_code, response.content)
  File "/usr/lib/python2.7/dist-packages/testtools/testcase.py", line 321, in assertEqual
    self.assertThat(observed, matcher, message)
  File "/usr/lib/python2.7/dist-packages/testtools/testcase.py", line 406, in assertThat
    raise mismatch_error
MismatchError: 200 != 503: Unable to connect to cluster 'cluster-pepU9F' (88f78ec0-68af-11e4-89e4-fa163e47b3b7); no connections available.

-------------------- >> begin captured logging << --------------------
maas.tftp: WARNING: No boot images have been imported from the region.
maas.node: INFO: 6a4XFejJNWsSzT29wlSA: Releasing node
root: ERROR: Unable to connect to cluster 88f78ec0-68af-11e4-89e4-fa163e47b3b7; no connections available.
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/django/core/handlers/base.py", line 112, in get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/usr/lib/python2.7/dist-packages/django/views/decorators/vary.py", line 19, in inner_func
    response = func(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/piston/resource.py", line 167, in __call__
    result = self.error_handler(e, request, meth, em_format)
  File "/usr/lib/python2.7/dist-packages/piston/resource.py", line 165, in __call__
    result = meth(request, *args, **kwargs)
  File "/tmp/tarmac/branch.fa9NZm/src/maasserver/api/support.py", line 189, in dispatch
    return function(self, request, *args, **kwargs)
  File "/tmp/tarmac/branch.fa9NZm/src/maasserver/api/nodes.py", line 345, in release
    node.release_or_erase()
  File "/tmp/tarmac/branch.fa9NZm/src/maasserver/models/node.py", line 1264, in release_or_erase
    self.release()
  File "/tmp/tarmac/branch.fa9NZm/src/maasserver/models/node.py", line 1225, in release
    self.start_transition_monitor(self.get_releasing_time())
  File "/tmp/tarmac/branch.fa9NZm/src/maasserver/models/node.py", line 593, in start_transition_monitor
    client = getClientFor(self.nodegroup.uuid)
  File "/tmp/tarmac/branch.fa9NZm/src/provisioningserver/utils/twisted.py", line 111, in wrapper
    return func_in_reactor(*args, **kwargs).wait()
  File "/usr/lib/python2.7/dist-packages/crochet/_eventloop.py", line 219, in wait
    result.raiseException()
  File "<string>", line 2, in raiseException
NoConnectionsAvailable: Unable to connect to cluster 88f78ec0-68af-11e4-89e4-fa163e47b3b7; no connections available.
--------------------- >> end captured logging << ---------------------

======================================================================
FAIL: maasserver.api.tests.test_node.TestNodeAPI.test_POST_release_combines_with_acquire
----------------------------------------------------------------------
_StringException: Traceback (most recent call last):
  File "/tmp/tarmac/branch.fa9NZm/src/maasserver/api/tests/test_node.py", line 524, in test_POST_release_combines_with_acquire
    self.assertEqual(httplib.OK, response.status_code, response.content)
  File "/usr/lib/python2.7/dist-packages/testtools/testcase.py", line 321, in assertEqual
    self.assertThat(observed, matcher, message)
  File "/usr/lib/python2.7/dist-packages/testtools/testcase.py", line 406, in assertThat
    raise mismatch_error
MismatchError: 200 != 503: Unable to connect to cluster 'cluster-lqDA5e' (89027308-68af-11e4-89e4-fa163e47b3b7); no connections available.

-------------------- >> begin captured logging << --------------------
maas.tftp: WARNING: No boot images have been imported from the region.
maas.api: INFO: Request from user test to acquire a node with constraints <QueryDict: {u'op': [u'acquire']}>
maas.node: INFO: 3tVpjZHlukUfptyQ4wZ1 allocated to user test
maas.node: INFO: 3tVpjZHlukUfptyQ4wZ1: Releasing node
twisted: ERROR: Unhandled error in Deferred:
twisted: ERROR: Unhandled Error
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 139, in maybeDeferred
    result = f(*args, **kw)
  File "/tmp/tarmac/branch.fa9NZm/src/maasserver/clusterrpc/power.py", line 56, in power_nodes
    deferreds[system_id] = getClientFor(cluster_uuid).addCallback(
  File "/tmp/tarmac/branch.fa9NZm/src/provisioningserver/utils/twisted.py", line 107, in wrapper
    return func(*args, **kwargs)
  File "/tmp/tarmac/branch.fa9NZm/src/provisioningserver/utils/twisted.py", line 62, in wrapper
    return maybeDeferred(func, *args, **kwargs)
--- <exception caught here> ---
  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 139, in maybeDeferred
    result = f(*args, **kw)
  File "/tmp/tarmac/branch.fa9NZm/src/maasserver/rpc/__init__.py", line 44, in getClientFor
    uuid, uuid=uuid)
provisioningserver.rpc.exceptions.NoConnectionsAvailable: Unable to connect to cluster 88f78ec0-68af-11e4-89e4-fa163e47b3b7; no connections available.

root: ERROR: Unable to connect to cluster 89027308-68af-11e4-89e4-fa163e47b3b7; no connections available.
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/django/core/handlers/base.py", line 112, in get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/usr/lib/python2.7/dist-packages/django/views/decorators/vary.py", line 19, in inner_func
    response = func(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/piston/resource.py", line 167, in __call__
    result = self.error_handler(e, request, meth, em_format)
  File "/usr/lib/python2.7/dist-packages/piston/resource.py", line 165, in __call__
    result = meth(request, *args, **kwargs)
  File "/tmp/tarmac/branch.fa9NZm/src/maasserver/api/support.py", line 189, in dispatch
    return function(self, request, *args, **kwargs)
  File "/tmp/tarmac/branch.fa9NZm/src/maasserver/api/nodes.py", line 345, in release
    node.release_or_erase()
  File "/tmp/tarmac/branch.fa9NZm/src/maasserver/models/node.py", line 1264, in release_or_erase
    self.release()
  File "/tmp/tarmac/branch.fa9NZm/src/maasserver/models/node.py", line 1225, in release
    self.start_transition_monitor(self.get_releasing_time())
  File "/tmp/tarmac/branch.fa9NZm/src/maasserver/models/node.py", line 593, in start_transition_monitor
    client = getClientFor(self.nodegroup.uuid)
  File "/tmp/tarmac/branch.fa9NZm/src/provisioningserver/utils/twisted.py", line 111, in wrapper
    return func_in_reactor(*args, **kwargs).wait()
  File "/usr/lib/python2.7/dist-packages/crochet/_eventloop.py", line 219, in wait
    result.raiseException()
  File "<string>", line 2, in raiseException
NoConnectionsAvailable: Unable to connect to cluster 89027308-68af-11e4-89e4-fa163e47b3b7; no connections available.
--------------------- >> end captured logging << ---------------------

======================================================================
FAIL: maasserver.api.tests.test_node.TestNodeAPI.test_POST_release_releases_failed_node
----------------------------------------------------------------------
_StringException: Traceback (most recent call last):
  File "/tmp/tarmac/branch.fa9NZm/src/maasserver/api/tests/test_node.py", line 443, in test_POST_release_releases_failed_node
    httplib.OK, response.status_code, response.content)
  File "/usr/lib/python2.7/dist-packages/testtools/testcase.py", line 321, in assertEqual
    self.assertThat(observed, matcher, message)
  File "/usr/lib/python2.7/dist-packages/testtools/testcase.py", line 406, in assertThat
    raise mismatch_error
MismatchError: 200 != 503: Unable to connect to cluster 'cluster-a3P5mu' (896cbff6-68af-11e4-89e4-fa163e47b3b7); no connections available.

-------------------- >> begin captured logging << --------------------
maas.tftp: WARNING: No boot images have been imported from the region.
maas.node: INFO: rUMapliPH0QJCnQYWHD8: Releasing node
root: ERROR: Unable to connect to cluster 896cbff6-68af-11e4-89e4-fa163e47b3b7; no connections available.
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/django/core/handlers/base.py", line 112, in get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/usr/lib/python2.7/dist-packages/django/views/decorators/vary.py", line 19, in inner_func
    response = func(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/piston/resource.py", line 167, in __call__
    result = self.error_handler(e, request, meth, em_format)
  File "/usr/lib/python2.7/dist-packages/piston/resource.py", line 165, in __call__
    result = meth(request, *args, **kwargs)
  File "/tmp/tarmac/branch.fa9NZm/src/maasserver/api/support.py", line 189, in dispatch
    return function(self, request, *args, **kwargs)
  File "/tmp/tarmac/branch.fa9NZm/src/maasserver/api/nodes.py", line 345, in release
    node.release_or_erase()
  File "/tmp/tarmac/branch.fa9NZm/src/maasserver/models/node.py", line 1264, in release_or_erase
    self.release()
  File "/tmp/tarmac/branch.fa9NZm/src/maasserver/models/node.py", line 1225, in release
    self.start_transition_monitor(self.get_releasing_time())
  File "/tmp/tarmac/branch.fa9NZm/src/maasserver/models/node.py", line 593, in start_transition_monitor
    client = getClientFor(self.nodegroup.uuid)
  File "/tmp/tarmac/branch.fa9NZm/src/provisioningserver/utils/twisted.py", line 111, in wrapper
    return func_in_reactor(*args, **kwargs).wait()
  File "/usr/lib/python2.7/dist-packages/crochet/_eventloop.py", line 219, in wait
    result.raiseException()
  File "<string>", line 2, in raiseException
NoConnectionsAvailable: Unable to connect to cluster 896cbff6-68af-11e4-89e4-fa163e47b3b7; no connections available.
--------------------- >> end captured logging << ---------------------

======================================================================
FAIL: maasserver.api.tests.test_node.TestNodeAPI.test_POST_release_releases_owned_node
----------------------------------------------------------------------
_StringException: Traceback (most recent call last):
  File "/tmp/tarmac/branch.fa9NZm/src/maasserver/api/tests/test_node.py", line 429, in test_POST_release_releases_owned_node
    [response.status_code for response in responses])
  File "/usr/lib/python2.7/dist-packages/testtools/testcase.py", line 321, in assertEqual
    self.assertThat(observed, matcher, message)
  File "/usr/lib/python2.7/dist-packages/testtools/testcase.py", line 406, in assertThat
    raise mismatch_error
MismatchError: [200, 200] != [503, 503]

-------------------- >> begin captured logging << --------------------
maas.tftp: WARNING: No boot images have been imported from the region.
maas.node: INFO: bXbIFArPbNq0PY8zBsox: Releasing node
root: ERROR: Unable to connect to cluster 89767776-68af-11e4-89e4-fa163e47b3b7; no connections available.
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/django/core/handlers/base.py", line 112, in get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/usr/lib/python2.7/dist-packages/django/views/decorators/vary.py", line 19, in inner_func
    response = func(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/piston/resource.py", line 167, in __call__
    result = self.error_handler(e, request, meth, em_format)
  File "/usr/lib/python2.7/dist-packages/piston/resource.py", line 165, in __call__
    result = meth(request, *args, **kwargs)
  File "/tmp/tarmac/branch.fa9NZm/src/maasserver/api/support.py", line 189, in dispatch
    return function(self, request, *args, **kwargs)
  File "/tmp/tarmac/branch.fa9NZm/src/maasserver/api/nodes.py", line 345, in release
    node.release_or_erase()
  File "/tmp/tarmac/branch.fa9NZm/src/maasserver/models/node.py", line 1264, in release_or_erase
    self.release()
  File "/tmp/tarmac/branch.fa9NZm/src/maasserver/models/node.py", line 1225, in release
    self.start_transition_monitor(self.get_releasing_time())
  File "/tmp/tarmac/branch.fa9NZm/src/maasserver/models/node.py", line 593, in start_transition_monitor
    client = getClientFor(self.nodegroup.uuid)
  File "/tmp/tarmac/branch.fa9NZm/src/provisioningserver/utils/twisted.py", line 111, in wrapper
    return func_in_reactor(*args, **kwargs).wait()
  File "/usr/lib/python2.7/dist-packages/crochet/_eventloop.py", line 219, in wait
    result.raiseException()
  File "<string>", line 2, in raiseException
NoConnectionsAvailable: Unable to connect to cluster 89767776-68af-11e4-89e4-fa163e47b3b7; no connections available.
maas.node: INFO: 2QVeP4y9fNKcdXxbYIdU: Releasing node
root: ERROR: Unable to connect to cluster 89793fec-68af-11e4-89e4-fa163e47b3b7; no connections available.
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/django/core/handlers/base.py", line 112, in get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/usr/lib/python2.7/dist-packages/django/views/decorators/vary.py", line 19, in inner_func
    response = func(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/piston/resource.py", line 167, in __call__
    result = self.error_handler(e, request, meth, em_format)
  File "/usr/lib/python2.7/dist-packages/piston/resource.py", line 165, in __call__
    result = meth(request, *args, **kwargs)
  File "/tmp/tarmac/branch.fa9NZm/src/maasserver/api/support.py", line 189, in dispatch
    return function(self, request, *args, **kwargs)
  File "/tmp/tarmac/branch.fa9NZm/src/maasserver/api/nodes.py", line 345, in release
    node.release_or_erase()
  File "/tmp/tarmac/branch.fa9NZm/src/maasserver/models/node.py", line 1264, in release_or_erase
    self.release()
  File "/tmp/tarmac/branch.fa9NZm/src/maasserver/models/node.py", line 1225, in release
    self.start_transition_monitor(self.get_releasing_time())
  File "/tmp/tarmac/branch.fa9NZm/src/maasserver/models/node.py", line 593, in start_transition_monitor
    client = getClientFor(self.nodegroup.uuid)
  File "/tmp/tarmac/branch.fa9NZm/src/provisioningserver/utils/twisted.py", line 111, in wrapper
    return func_in_reactor(*args, **kwargs).wait()
  File "/usr/lib/python2.7/dist-packages/crochet/_eventloop.py", line 219, in wait
    result.raiseException()
  File "<string>", line 2, in raiseException
NoConnectionsAvailable: Unable to connect to cluster 89793fec-68af-11e4-89e4-fa163e47b3b7; no connections available.
--------------------- >> end captured logging << ---------------------

----------------------------------------------------------------------
Ran 3885 tests in 297.407s

FAILED (SKIP=6, failures=4)
...............................................................................................................................
----------------------------------------------------------------------
Ran 127 tests in 1.879s

OK
.............................................................S........................................................................................
----------------------------------------------------------------------
Ran 150 tests in 28.156s

OK (SKIP=1)
....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
----------------------------------------------------------------------
Ran 1716 tests in 21.060s

OK
make: *** [test] Error 123

MAAS

Merge lp://staging/~julian-edwards/maas/releasing-timeout-bug-1390144 into lp://staging/~maas-committers/maas/trunk

Commit message

Description of the change

Preview Diff

Subscribers