Merge lp://staging/~clint-fewbar/pyjuju/charm-tests-spec into lp://staging/~juju/pyjuju/docs

Proposed by Clint Byrum
Status: Merged
Approved by: Mark Mims
Approved revision: 15
Merge reported by: Mark Mims
Merged at revision: not available
Proposed branch: lp://staging/~clint-fewbar/pyjuju/charm-tests-spec
Merge into: lp://staging/~juju/pyjuju/docs
Diff against target: 293 lines (+289/-0)
1 file modified
source/charm-tests.rst (+289/-0)
To merge this branch: bzr merge lp://staging/~clint-fewbar/pyjuju/charm-tests-spec
Reviewer Review Type Date Requested Status
Mark Mims (community) Approve
Review via email: mp+91503@code.staging.launchpad.net

Description of the change

This is a draft specification for implementing large scale charm tests.

This is a draft specification for implementing large scale charm tests.

https://codereview.appspot.com/5624044/

To post a comment you must log in.
Revision history for this message
Gustavo Niemeyer (niemeyer) wrote :
Download full text (3.6 KiB)

Thanks Clint. Here are some comments:

https://codereview.appspot.com/5624044/diff/1/source/charm-tests.rst
File source/charm-tests.rst (right):

https://codereview.appspot.com/5624044/diff/1/source/charm-tests.rst#newcode51
source/charm-tests.rst:51: utilized to attach tests to charms. Under the
charm root directory,
"To facilitate tests attached to charms .. to attach tests to charms."

The sentence is a bit self-referential. Would be nice to reword

https://codereview.appspot.com/5624044/diff/1/source/charm-tests.rst#newcode60
source/charm-tests.rst:60: * the default environment is bootstrapped
To be more clear, we might say something like:

* A testing environment is already bootstrapped and is made the default
for command line usage.

https://codereview.appspot.com/5624044/diff/1/source/charm-tests.rst#newcode61
source/charm-tests.rst:61: * The CWD is the charm root directory
I think this should be the "tests/" directory itself. Note that the test
doesn't actually have a strong relation to the content of the charm. The
content is deployed into the remote machine, and the test is sitting
locally. Its interaction will be mainly remote.

https://codereview.appspot.com/5624044/diff/1/source/charm-tests.rst#newcode64
source/charm-tests.rst:64: charm url and/or repository arguments of the
test runner's choice. This
We have to qualify a bit better what the "runner's choice" mean, since a
charmer reading that will feel like "Okay, but _what is the version_?".

Maybe something like "the current version at the moment the test is
run"? This gives us some freedom to move back and forth, and gives the
charmer a vague feeling of what to expect.

https://codereview.appspot.com/5624044/diff/1/source/charm-tests.rst#newcode69
source/charm-tests.rst:69: * a special sub-command of juju,
``deploy-previous``, will deploy the
This command doesn't exist. Would you mind to drop the upgrade aspect
from the specification entirely? I'd be happy to debate about a follow
up improvement once we have the basics in place.

https://codereview.appspot.com/5624044/diff/1/source/charm-tests.rst#newcode76
source/charm-tests.rst:76: * ``~/.juju`` will not be accessible to the
tests
Read-only, maybe? It needs to be accessible for reading purposes or the
command doesn't work.

https://codereview.appspot.com/5624044/diff/1/source/charm-tests.rst#newcode82
source/charm-tests.rst:82: If present, tests/requirements.yaml will be
read to determine packages
Looks good, but can we please name that as "tests/tests.yaml", so that
this may be used for other needs too, when/if they show up?

https://codereview.appspot.com/5624044/diff/1/source/charm-tests.rst#newcode90
source/charm-tests.rst:90: If a tool is needed to perform a test and not
available in the Ubuntu
s/and not/and is not/?

https://codereview.appspot.com/5624044/diff/1/source/charm-tests.rst#newcode111
source/charm-tests.rst:111: running charms, and so may affect the
outcome.
Section looks great.

https://codereview.appspot.com/5624044/diff/1/source/charm-tests.rst#newcode141
source/charm-tests.rst:141: trap teardown EXIT
Destroying the temporary state manually should not be necessary. The
harness should be able to put the...

Read more...

6. By Clint Byrum

Rewording paragraph to be more succinct.

7. By Clint Byrum

clarifying default environment

8. By Clint Byrum

moving CWD to tests directory to encourage better encapsulation of test code

9. By Clint Byrum

using strong language in explanation of charm resolver

10. By Clint Byrum

dropping deploy-previous pending some discussion of how best to test upgrade-charm

11. By Clint Byrum

No need to state this, as it complicates things unnecessarily and is implied by not guaranteeing access to it

12. By Clint Byrum

renaming requirements.yaml to tests.yaml to allow broader usage in the future

13. By Clint Byrum

grammar is our friend

14. By Clint Byrum

Re-merging fixes from email discussion. Moves test runner to the end, adds another example, clarifies exit codes and intended test output.

15. By Clint Byrum

Removing suggestions to cleanup services. Also clarifying output suggestions.

Revision history for this message
Mark Mims (mark-mims) :
review: Approve
Revision history for this message
Mark Mims (mark-mims) wrote :

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
The diff is not available at this time. You can reload the page or download it.

Subscribers

People subscribed via source and target branches