Ubuntu CI Dashboard

Merge lp://staging/~doanac/qa-dashboard/live-status into lp://staging/qa-dashboard

live-status
Merge into dev

Proposed by Andy Doan on 2013-12-11

Status:	Merged
Approved by:	Andy Doan on 2013-12-13
Approved revision:	694
Merged at revision:	703
Proposed branch:	lp://staging/~doanac/qa-dashboard/live-status
Merge into:	lp://staging/qa-dashboard
Diff against target:	392 lines (+328/-1) 5 files modified qa_dashboard/settings.py (+13/-0) requirements.txt (+3/-0) smokeng/api.py (+119/-0) smokeng/tests.py (+188/-0) smokeng/urls.py (+5/-1)
To merge this branch:	bzr merge lp://staging/~doanac/qa-dashboard/live-status
Related bugs:	Link a bug report

Reviewer	Review Type	Date Requested	Status
Joe Talbott		2013-12-11	Approve on 2013-12-13
PS Jenkins bot	continuous-integration		Approve on 2013-12-12
Chris Johnston			Approve on 2013-12-11
Andy Doan			Pending
Review via email: mp+198602@code.staging.launchpad.net

This proposal supersedes a proposal from 2013-11-21.

Commit message

Add a REST API to Smoke objects

Description of the change

Add a REST API to Smoke objects

This gives us the ability to have our jenkins jobs notify the
dashboard in realtime about smoke status and not have to wait
until the job has been pushed to the public server and then
polled by the pull-script.

In addition to a fairly thorough set of unit-tests, I've also
created a pretty easy CLI to do some exploratory testing with:

http://paste.ubuntu.com/6454113/

That script will probably land in the lp:ubuntu-test-cases/touch repo
in some form or another.

Some notes to keep in mind:

* The API does not allow for delete operations (just to help decrease
the amount of damage someone might accidentally do).

* It doesn't require auth[entication|orization] for GETS, but
use's Django users + Tastypie ApiKey for PUT and PATCH operations.

* I had to use two hacks with Tastypie:
1) 0.9.15 because 0.10 migrations don't work with python 2.7 and
django 1.5

2) settings.US_TZ had to be enabled as noted in the settings.py
due to a tastypie bug.

I'm not really excited about #2, but given the limited use of "writes"
we have in the dashboard, I think this may be fairly safe/isolated.

Revision history for this message

Andy Doan (doanac) wrote on 2013-11-21: Posted in a previous version of this proposal

wanted to share the branch, but I'd like to try and get one more unit test written to help make sure it plays nicely with the pull-script

Revision history for this message

Andy Doan (doanac) wrote on 2013-11-25: Posted in a previous version of this proposal

we are blocked on:

https://rt.admin.canonical.com/Ticket/Display.html?id=66058

Revision history for this message

Andy Doan (doanac) wrote on 2013-12-02: Posted in a previous version of this proposal

tastypie 0.9.15 should be ready, so this can be reviewed now.

Revision history for this message

Andy Doan (doanac) wrote on 2013-12-03: Posted in a previous version of this proposal

here's a branch to give you guys some insight into how i use this API:

http://bazaar.launchpad.net/~doanac/ubuntu-test-cases/live-status/revision/128

this is all working at home now. The remaining work I need to look at is getting our smoke pull script to handle multiple results from a single job.

Revision history for this message

Paul Larson (pwlars) wrote on 2013-12-06: Posted in a previous version of this proposal

> here's a branch to give you guys some insight into how i use this API:
>
> http://bazaar.launchpad.net/~doanac/ubuntu-test-cases/live-
> status/revision/128
>
> this is all working at home now. The remaining work I need to look at is
> getting our smoke pull script to handle multiple results from a single job.
Really cool! Should we have to care about the pull script though? It should just replace the current result with what it pulls. So if it was in progress before, or even complete before, it's only going to get the "DONE" status for a fully complete job on a pull. At worst, it should be the same as what it has.

Or do you mean that pulling a job that was in a still running state before would be marked as still running?

Revision history for this message

Andy Doan (doanac) wrote on 2013-12-06: Posted in a previous version of this proposal

On 12/06/2013 11:51 AM, Paul Larson wrote:
> Really cool! Should we have to care about the pull script though? It should just replace the current result with what it pulls. So if it was in progress before, or even complete before, it's only going to get the "DONE" status for a fully complete job on a pull. At worst, it should be the same as what it has.
>
> Or do you mean that pulling a job that was in a still running state before would be marked as still running?

The pulls script also learns about all the job artifacts. This is the
key thing we need before marking a result as "COMPLETE".

Revision history for this message

Joe Talbott (joetalbott) wrote on 2013-12-10: Posted in a previous version of this proposal

Only thing I see is the 2012 in the copyright.

Otherwise +1.

review: Approve

Revision history for this message

Joe Talbott (joetalbott) on 2013-12-11:

review: Approve

Revision history for this message

Chris Johnston (cjohnston) on 2013-12-11:

review: Approve

Revision history for this message

Chris Johnston (cjohnston) wrote on 2013-12-11:

The attempt to merge lp:~doanac/qa-dashboard/live-status into lp:qa-dashboard failed. Below is the output from the failed tests.

Traceback (most recent call last):
  File "./manage.py", line 11, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/lib/python2.7/dist-packages/django/core/management/__init__.py", line 453, in execute_from_command_line
    utility.execute()
  File "/usr/lib/python2.7/dist-packages/django/core/management/__init__.py", line 392, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/lib/python2.7/dist-packages/django/core/management/__init__.py", line 272, in fetch_command
    klass = load_command_class(app_name, subcommand)
  File "/usr/lib/python2.7/dist-packages/django/core/management/__init__.py", line 77, in load_command_class
    module = import_module('%s.management.commands.%s' % (app_name, name))
  File "/usr/lib/python2.7/dist-packages/django/utils/importlib.py", line 35, in import_module
    __import__(name)
  File "/usr/lib/python2.7/dist-packages/south/management/commands/__init__.py", line 10, in <module>
    import django.template.loaders.app_directories
  File "/usr/lib/python2.7/dist-packages/django/template/loaders/app_directories.py", line 25, in <module>
    raise ImproperlyConfigured('ImportError %s: %s' % (app, e.args[0]))
django.core.exceptions.ImproperlyConfigured: ImportError tastypie: No module named tastypie

Revision history for this message

PS Jenkins bot (ps-jenkins) wrote on 2013-12-11:

FAILED: Continuous integration, rev:692
http://s-jenkins.ubuntu-ci:8080/job/dashboard-ci/274/
Executed test runs:

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/dashboard-ci/274/rebuild

review: Needs Fixing (continuous-integration)

Revision history for this message

PS Jenkins bot (ps-jenkins) wrote on 2013-12-11:

FAILED: Continuous integration, rev:692
http://s-jenkins.ubuntu-ci:8080/job/dashboard-ci/275/
Executed test runs:

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/dashboard-ci/275/rebuild

review: Needs Fixing (continuous-integration)

lp://staging/~doanac/qa-dashboard/live-status updated on 2013-12-12

693. By Andy Doan on 2013-12-12

a hack to deal with tastypie bug

setting USE_TZ=True is pain. It causes lots of django warnings and
broke some of our exsiting test case logic. This backs out the
settings change for USE_TZ and monkey-patches the actual buggy spot
in code so that we can use the admin panel properly

Revision history for this message

PS Jenkins bot (ps-jenkins) wrote on 2013-12-12:

FAILED: Continuous integration, rev:693
http://s-jenkins.ubuntu-ci:8080/job/dashboard-ci/276/
Executed test runs:

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/dashboard-ci/276/rebuild