Merge lp://staging/~gmb/maas-test/dont-report-successes into lp://staging/maas-test

Looks good.

On the one hand, I think that the code is better structured this way (having the control of whether or not things are logged in the main execution flow and all the actual work done in helper methods) but on the other hand, it means part of the code escaped in non-tested territory. Oh well, this is still probably for the best.

[0]

96 + "Visit https://bugs.launchpad.net/maas/+filebug/%s "
97 + "to file a bug and complete the maas-test reporting process."

Btw, this is unrelated to this change but I was wondering if we could pre-fill the title of the bug somehow. Do you know if it's possible? If it is, it would be nice because —unless the user changes it of course— it would enable us to quickly spot if a bug has been submitted because of a failed maas-test run; it would be even better if we could attach a tag to it.

On 9 January 2014 15:27, Raphaël Badin <email address hidden> wrote:
> Btw, this is unrelated to this change but I was wondering if we could pre-fill the title of the bug somehow. Do you know if it's possible? If it is, it would be nice because —unless the user changes it of course— it would enable us to quickly spot if a bug has been submitted because of a failed maas-test run; it would be even better if we could attach a tag to it.

We can — and we did, in my initial branch, but we dropped it IIRC. We
already tag it with "maas-test-results". Do you think just pre-filling
it with "maas-test failure" would be sufficient? We can always update
the title to be more specific later.

> We can — and we did, in my initial branch, but we dropped it IIRC. We
> already tag it with "maas-test-results". Do you think just pre-filling
> it with "maas-test failure" would be sufficient? We can always update
> the title to be more specific later.

Yeah, "maas-test failure" sounds like a good start.

While I appreciate we don't necessarily want to file bugs for successes, the point of maas-test is that we DO report successes *somewhere*. Since we don't have somewhere else, I am afraid this branch should be reverted.

Warning: early-morning shambolic rambling ahead:

On 10 January 2014 00:57, Julian Edwards <email address hidden> wrote:
> Review: Disapprove
>
> While I appreciate we don't necessarily want to file bugs for
> successes, the point of maas-test is that we DO report successes
> *somewhere*. Since we don't have somewhere else, I am afraid this
> branch should be reverted.

Okay, I can see that recording successes has some value — maybe more
value than I realise beyond the "look at everything that works with
MAAS" marketing standpoint. But do we _really_ want to have every
success captured?

I suppose we could file them as "Fix Released" :). (Assuming LP lets you
do that).

My argument's a little fuzzy here — hey, it's 07:24 and I haven't
finished my first cup of tea yet — but here goes:

 1. We want to record all test runs, be they pass or failure.
 2. However, passes all look the same
 3. That means that we'd end up with a lot of duplicate bugs
 4. Worse, de-duping them will be really hard because there's nothing
    unique about the reports. (In fact this applies to failures as well
    as successes — imagine if 10 companies try maas-test on similar but
    subtly different hardware).
 5. Actually, then, my argument isn't "we shouldn't report successes" at
    all; in fact, my argument is "maas-test reports should carry some
    identifying information as standard so that de-duping is easier".
 6. If that were true, I'd be fine with reverting this; at the moment
    I'm -0 at the most.

We do need to report success, so some way of IDing the test run on a
hardware basis would be the best way forwards here (and then dupe on that).

On 10 January 2014 09:27, Julian Edwards <email address hidden> wrote:
> We do need to report success, so some way of IDing the test run on a
> hardware basis would be the best way forwards here (and then dupe on that).

+1. I'll revert this branch for starters.

On 10/01/14 19:42, Graham Binns wrote:
> On 10 January 2014 09:27, Julian Edwards <email address hidden> wrote:
>> We do need to report success, so some way of IDing the test run on a
>> hardware basis would be the best way forwards here (and then dupe on that).
>
> +1. I'll revert this branch for starters.
>

Thanks. Sorry I wasn't around to put you off in the first place, I keep
assuming everyone knows everything... and you know what they say about
assumptions.