Merge lp://staging/~vila/ols-store-tests/ols-run-tests into lp://staging/~ubuntuone-pqm-team/ols-store-tests/store-acceptance-tests

Proposed by Vincent Ladeuil
Status: Merged
Approved by: Vincent Ladeuil
Approved revision: 43
Merged at revision: 40
Proposed branch: lp://staging/~vila/ols-store-tests/ols-run-tests
Merge into: lp://staging/~ubuntuone-pqm-team/ols-store-tests/store-acceptance-tests
Diff against target: 305 lines (+156/-11)
8 files modified
Makefile (+3/-3)
README (+1/-1)
dependencies.txt (+1/-0)
ols-vms.conf (+2/-1)
run-landing-tests.sh (+1/-2)
tests/api/snap/helpers.py (+68/-0)
tests/api/snap/test_helpers.py (+70/-0)
tests/api/snap/test_snap_release.py (+10/-4)
To merge this branch: bzr merge lp://staging/~vila/ols-store-tests/ols-run-tests
Reviewer Review Type Date Requested Status
Fabián Ezequiel Gallina (community) Approve
Review via email: mp+318929@code.staging.launchpad.net

Commit message

Display log when test_release_snap_successful fails.

Use ols-run-tests to display test names and timings when running on jenkaas.

Properly isolate pip3 by using the virtualenv one rather than the system one.

Description of the change

Display log when test_release_snap_successful fails.

Use ols-run-tests to display test names and timings when running on jenkaas.

Properly isolate pip3 by using the virtualenv one rather than the system one.

To post a comment you must log in.
Revision history for this message
Fabián Ezequiel Gallina (fgallina) wrote :

I like where this is going, in fact I like so much that I'm proposing we have the log on failure for all tests ;)

Revision history for this message
Vincent Ladeuil (vila) wrote :

> What if we made this part of the setUp and tearDown process and put it in the base TestCase.

The thing is, this trick requires wrapping the test method itself.

There is no good way (at least I haven't found any so far) that can be achieved "transparently" for all test methods.

If you look at the unittest implementation, you'll see that __call__() calls run() which provides no way to wrap the test method call (search for testMethod there) itself.

Alternatively, this may be be implemented on the TestResult object but that looks brittle (and would require implementing it for all TestResult objects).

This code originated in ols-vms (https://bazaar.launchpad.net/~ubuntuone-hackers/ols-vms/trunk/view/head:/olsvms/tests/__init__.py#L55 but is not used there anymore) and worked well, the constraint of explicitly decorating each test may seem a bit tedious but was worth it (not all tests have a use for that).

While borrowing it I attempted to do just what you suggest but fail.

Come to think of it that should allow the implementation to be simplified (there is no need for a class since there are no level and fmt parameters anymore).

Also note that here, we're not calling logging from any code under test, we only use it as part of the test code.

So this proposal is to activate it where we need it. As such, it's not exposed either where it's not needed.

I'll keep thinking about generalizing it though.

41. By Vincent Ladeuil

Display log on failure for all APITestCase.

Revision history for this message
Vincent Ladeuil (vila) wrote :

> What if we made this part of the setUp and tearDown process and put it in the base TestCase.

Take two.

setUp() is too late, this has to happen in __init__.

Last revision does that.

I still think it's a bit too magic (I have a gut feeling this may break in an obscure way in edge cases (parametrized tests for example)) and prefer to decorate the tests that really needs that but let's see how this goes.

42. By Vincent Ladeuil

Hacking from __init__ is too brittle, come back to simpler decorator.

43. By Vincent Ladeuil

Remove useless code.

Revision history for this message
Fabián Ezequiel Gallina (fgallina) wrote :

LGTM

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
The diff is not available at this time. You can reload the page or download it.

Subscribers

People subscribed via source and target branches