Ubuntu CI Services

Merge lp://staging/~doanac/ubuntu-ci-services-itself/bsbuilder into lp://staging/ubuntu-ci-services-itself

bsbuilder
Merge into trunk

Proposed by Andy Doan on 2013-12-09

Status:	Merged
Approved by:	Francis Ginther on 2013-12-12
Approved revision:	25
Merged at revision:	28
Proposed branch:	lp://staging/~doanac/ubuntu-ci-services-itself/bsbuilder
Merge into:	lp://staging/ubuntu-ci-services-itself
Diff against target:	800 lines (+720/-2) 12 files modified README (+31/-2) branch-source-builder/bsbuilder/__init__.py (+17/-0) branch-source-builder/bsbuilder/amqp_utils.py (+108/-0) branch-source-builder/bsbuilder/resources/root.py (+30/-0) branch-source-builder/bsbuilder/resources/v1.py (+53/-0) branch-source-builder/bsbuilder/tests/test_utils.py (+165/-0) branch-source-builder/bsbuilder/tests/test_v1.py (+70/-0) branch-source-builder/bsbuilder/utils.py (+78/-0) branch-source-builder/bsbuilder/wsgi.py (+48/-0) branch-source-builder/run_worker (+47/-0) branch-source-builder/setup.py (+44/-0) juju-deployer/branch-source-builder.yaml (+29/-0)
To merge this branch:	bzr merge lp://staging/~doanac/ubuntu-ci-services-itself/bsbuilder
Related bugs:	Link a bug report

Reviewer	Review Type	Date Requested	Status
Francis Ginther		2013-12-09	Approve on 2013-12-12
Review via email: mp+198308@code.staging.launchpad.net

Commit message

Implements the webservice portion of the branch source builder

This creates a simple app using restish that takes a build_source request and sends it to rabbitmq. On the other end we have a simple worker script, branch-source-builder/run_worker that will wait for messages and print a TODO.

The MP itself is bigger than the service because of the groundwork I did for the restish service.

Description of the change

Implements the webservice portion of the branch source builder

The MP itself is bigger than the service because of the groundwork I did for the restish service.

Revision history for this message

Andy Doan (doanac) wrote on 2013-12-09:

There are some caveats to getting the juju-deployer portion of this working:

1) the rabbitmq-server charm is broke. It seems to be the .deb itself, but you have to edit /etc/hosts and add the host name to the 127.0.0.1 entry so the install will work. If you are fast and do that before juju-deployer gets to the bsb-rabbit unit the deployer will work. Otherwise you have to make that change, run "juju resolved --retry bsb-rabbit/0". then re-run the deployer script.

2) the bsb-worker script doesn't work out the first time. For some reason even with upstart respawing it, it won't start until you run: sudo stop bsb_worker; sudo start bsb_worker

At this point the service should be running and you can test with:

curl --dump-header - -H "Content-Type: application/json" -X POST --data '{"source_packages": "todo_source", "ppa": "todo_ppa", "progress_trigger": "todo_trigger"}' http://<BSB RESTISH IP>:8080/api/v1/build_source

then on the bsb_worker node, you'll see a TODO printed in /var/log/upstart/bsb_worker.log

lp://staging/~doanac/ubuntu-ci-services-itself/bsbuilder updated on 2013-12-10

23. By Andy Doan on 2013-12-10: work around a canonistack bug with seen with rabbitmq-server

Revision history for this message

Andy Doan (doanac) wrote on 2013-12-10:

I've fixed the rabbitmq-server issue mentioned above by creating our own rabbitmq-server charm with a fix.

I've identified the solution to the 2nd problem and should have a minor fix for that ready soon.

lp://staging/~doanac/ubuntu-ci-services-itself/bsbuilder updated on 2013-12-10

24. By Andy Doan on 2013-12-10

move queue helpers to their own module

This makes the queue helpers more accessible by the run_worker
script w/o making it have to pull in python-restish dependencies.

It also uses the new common configuration file mechanism thats
been changed in the rabbitmq-worker charm to be like the
restish charm.

Revision history for this message

Andy Doan (doanac) wrote on 2013-12-10:

as of revno 24, things "just work".

Revision history for this message

Francis Ginther (fginther) wrote on 2013-12-10:

In addition to eventually refactoring the queue helpers into a common utils directory as you mentioned:

- when refactoring, be sure to split the tests in branch-source-builder/bsbuilder/tests/test_utils.py.
- branch-source-builder/setup.py is not executable
- wsgi.py starts the server on port 8080, but prints '8000'. Is there a guideline for choosing ports? My local jenkins runs on 8080 also so I can't start this locally as is.

Not having done a REST app before, I don't have a lot of input. I'm in favor of how you're doing this, but not familiar with any alternatives. I did spend extra time reviewing the queue setup and like what's going on there.

Want to do some local testing before approving.

lp://staging/~doanac/ubuntu-ci-services-itself/bsbuilder updated on 2013-12-11

25. By Andy Doan on 2013-12-11

address comments from fginther

make setup.py executable
use argparse for local wsgi server

Revision history for this message

Francis Ginther (fginther) wrote on 2013-12-12:

Was able to deploy successfully after adjusting the juju-deployer file to point to this branch.

Let's go with this and tweak it over time.

review: Approve