Merge lp://staging/~doanac/ubuntu-ci-services-itself/source-builder-design into lp://staging/ubuntu-ci-services-itself

Excellent, just one update. Initially this is only going to operate on source packages, so the purpose becomes:

Accepts a source package and dispatches it to a PPA for building and monitors for completion and collection of results.

good catch. its been updated now

Such a ridiculously minor comment, but you could also use the status URL as a Nagios check of the service. I did something along those lines here:
http://bazaar.launchpad.net/~daisy-pluckers/errors/trunk/view/head:/errors/views.py#L89

The Online Services team have lots of examples of this sort of thing:
https://bazaar.launchpad.net/~ubuntuone-pqm-team/ubuntuone-servers/trunk-2a/view/head:/lib/ubuntuone/wsgi/twisted.py#L848

Looks good. Out of curiosity, did you have a quick look at Celery?

I ask because William had that comment in the standup about using Celery as a starting point and moving onto python-amqplib for the tasks that don't fit the Celery model.

I haven't used it myself, so if you're confident that python-amqplib does the job simply, please proceed :)

On 12/06/2013 07:02 AM, Evan Dandrea wrote:
> Such a ridiculously minor comment, but you could also use the status URL as a Nagios check of the service. I did something along those lines here:
> http://bazaar.launchpad.net/~daisy-pluckers/errors/trunk/view/head:/errors/views.py#L89

That's interesting. I've been wondering if we should add some sort of
URL resource to all our services like:

  curl http://service/api/v1/health_check

This could return some json array of components with pass/fail. Does
this make sense in that context? My thinking was around something like
rabbit where its hard to know if its working until its online. ie, one
part of the health check could be to send a test message and ensure a
worker responds. This could serve as the basis for some sort of
diagnostic view in the WebUI some day. Does this make sense, or am I
trying to solve something that's already been solved in a more generic way?

> The Online Services team have lots of examples of this sort of thing:
> https://bazaar.launchpad.net/~ubuntuone-pqm-team/ubuntuone-servers/trunk-2a/view/head:/lib/ubuntuone/wsgi/twisted.py#L848
>

On 12/06/2013 07:07 AM, Evan Dandrea wrote:
> Review: Approve
>
> Looks good. Out of curiosity, did you have a quick look at Celery?

I had done this MP before talking to William.

> I ask because William had that comment in the standup about using Celery as a starting point and moving onto python-amqplib for the tasks that don't fit the Celery model.
>
> I haven't used it myself, so if you're confident that python-amqplib does the job simply, please proceed :)

I've used Celery before, but TBH I hit problems where it couldn't seem
to route things properly. My reasons for leaning towards amqplib would be:

1) one less dependency to pull in (not a good enough reason though)
2) we have some code from daisy that fits our usage model

So my gut is telling me to do amqplib for now, but I don't really have
any hard supporting data to back it up.

Checking rabbit queues is a solved problem :)

https://bazaar.launchpad.net/~canonical-is-robot/canonical-is-puppet/golive/view/head:/modules/rabbitmq/files/check_rabbitmq.py
https://bazaar.launchpad.net/~canonical-is-robot/canonical-is-puppet/golive/view/head:/modules/rabbitmq/files/check_rabbitmq_queues.py

That's run as a standard nrpe (Nagios) check. I suspect we want to
isolate that check to be directly run off the Rabbit server, rather
than going through one of our other services. If you have a more
specific check for the Source Builder, then I'd agree that putting
that behind the heath_check API call makes a lot of sense.

On 6 December 2013 15:21, Andy Doan <email address hidden> wrote:
> I've used Celery before, but TBH I hit problems where it couldn't seem
> to route things properly. My reasons for leaning towards amqplib would be:
>
> 1) one less dependency to pull in (not a good enough reason though)
> 2) we have some code from daisy that fits our usage model
>
> So my gut is telling me to do amqplib for now, but I don't really have
> any hard supporting data to back it up.

Go with your gut. I'd agree that python-amqplib is perfectly workable,
and appreciate the points about it bringing less complexity than
Celery.

Approve.