Merge lp://staging/~vila/ubuntu-ci-services-itself/testy into lp://staging/ubuntu-ci-services-itself

Proposed by Vincent Ladeuil
Status: Merged
Approved by: Francis Ginther
Approved revision: no longer in the source branch.
Merged at revision: 14
Proposed branch: lp://staging/~vila/ubuntu-ci-services-itself/testy
Merge into: lp://staging/ubuntu-ci-services-itself
Diff against target: 402 lines (+112/-56)
8 files modified
docs/components/existing-pieces.rst (+1/-1)
docs/components/landing-manager.rst (+4/-4)
docs/components/planned.rst (+1/-1)
docs/components/test-runner.rst (+75/-19)
docs/components/ticket-manager.rst (+3/-3)
docs/components/ticket-system.rst (+1/-1)
docs/sequence.rst (+23/-23)
docs/timeline.rst (+4/-4)
To merge this branch: bzr merge lp://staging/~vila/ubuntu-ci-services-itself/testy
Reviewer Review Type Date Requested Status
Francis Ginther Approve
Review via email: mp+198083@code.staging.launchpad.net

Commit message

Update the Test Runner component design and provide the initial API.

Description of the change

Main changes from the trunk version:

* 'test_image' request

  s/test_image(image_url, packages, archive_id, progress_trigger)
   /test_image(test_request_id, image_url, package)

  The rationale is that the caller should do multiple requests if it needs
  multiple test results (otherwise the 'done' request will have to aggregate
  the results and the artifacts and the caller will have to sort them out =>
  unneeded complexity).

* handles a single DEP8 package which produce a single test result and a
  list of artifacts (phase0). For following phases, I'd like to embed the
  articats in the test result itself as discussed during the 'subunit'
  hangout. For now, the request will contain them all, leaving the caller
  store them as it sees fit.

  The rationale is that asking this component to know about the data store
  is coupling them for no good reason.

* new requests: 'done' and 'progress'

Questions:

* messages exchanged between components

We talked about REST APIs and about task queues. I more or less assumed that
a request is added to a queue as a task and the response is sent back when
the task is done (from the queue POV). Feedback welcome.

For the 'done' request, the above works ok, but if we want a REST API and no
queue, I need to know *where* to send that request (some url provided as
part of the 'test_package' request parameters ?).

* progress

Emitting progress and providing an ETA or some estimate of the work already
done raise two issues:

- how many messages should be sent or alternatively at which rate ? Emitting
  the same number of messages for a 1 minute job and a several hours one
  doesn't make much sense. It seems to me that one message very minute or
  every 30 seconds should be enough. But at scale, that may still be far too
  much messages. Thoughts ?

- ETA is a pipe dream so let's settle for estimates to start with. This can
  be be based on the number of tests as long as each test is shorter than
  the interval between two messages. If it's not, we risk misreadings
  between a long test and a hanging test.

If we don't get good ideas there, I'd settle with some arbitrary interval
for phase 0 and plan to acquire the number of tests to initialize 'total'.

To post a comment you must log in.
Revision history for this message
Andy Doan (doanac) wrote :

On 12/06/2013 09:34 AM, Vincent Ladeuil wrote:
> + test_image(image_url, package_name) -> test_request_id

> +This ends the processing of a 'done' request and is sent to the 'Landing
> +Manager'.

> + done(test_request_id, status=[FAIL, SUCCESS], test_result, artifacts)

This introduces a new decision for us to make. In the
branch-source-builder we have something called a "progress_trigger". It
appears to be the same concept as your "test_request_id". So the first
decision is simply agreeing on a common name for this thing.

The thing where the branch-source-builder really differs is on who
creates progress_trigger. In its design, the caller passes this value.
In your design the implementation creates and returns the value. I don't
have a strong opinion. However, I suspect the branch-source-builder's
approach will be more HA friendly. eg, it can create this object and
persist it somehow for the service. If the calling service dies while
the request is being processed, its okay. The messages will get queued
up somewhere to be determined, and when the back-up service comes back
online it will know about this progress_trigger and be able to catch
back up with the queued responses.

Revision history for this message
Francis Ginther (fginther) wrote :

Thanks for agreeing on the caller/callee semantics for passing of the test_request_id as the queue handle from the Lander to the Test Runner.

This MP looks good, I'll approve.

The questions you raise above merit their own MP. I'll hopefully get to a proposal by Monday morning. The general idea is to:
 - define the calling convention to pass the progress queue handle from the caller to the callee (as Andy described)
 - define progress semantics (what is the progress interval, what is considered a timeout, role of caller and callee)
 - define task queue semantics (role of the producer and consumer)

review: Approve
11. By Andy Doan

[r=Francis Ginther, Evan Dandrea] updated branch/source builder design from Andy Doan

12. By Andy Doan

[r=Francis Ginther] adds first pass of ppa-assigner

more changes to come following design updates. from Andy Doan

13. By Andy Doan

[r=Francis Ginther, Evan Dandrea] updated branch/source builder design from Andy Doan

14. By Vincent Ladeuil

[r=Francis Ginther] Update the Test Runner component design and provide the initial API. from Vincent Ladeuil

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
The diff is not available at this time. You can reload the page or download it.

Subscribers

People subscribed via source and target branches