Merge ~pwlars/checkbox/+git/support:test-in-lxd into ~checkbox-dev/checkbox/+git/support:master

Proposed by Paul Larson
Status: Merged
Approved by: Paul Larson
Approved revision: 0ed63d9089e02cd953ae8f8ed2d2746cdf562c0a
Merged at revision: 360166f31d4267ffa4383e5105a2a51030f94b3a
Proposed branch: ~pwlars/checkbox/+git/support:test-in-lxd
Merge into: ~checkbox-dev/checkbox/+git/support:master
Diff against target: 346 lines (+89/-149)
2 files modified
provision-testing-environment (+0/-34)
test-in-lxc.sh (+89/-115)
Reviewer Review Type Date Requested Status
Maciej Kisielewski (community) Approve
Paul Larson Needs Resubmitting
Review via email: mp+359567@code.staging.launchpad.net

Description of the change

This adds support for testing in lxd. I chose to keep it as a shell script *for now*, because it is so intertwined with other shell scripts from other checkbox projects still. Consolidating this will be a little tricky and require some more thought, but the bigger goal for right now was to move to lxd, and also to get some better speed. So not only does this move to lxd, but it also speeds things up a bit by updating the pristine container each time. That way the diff between it and the clean copy is not so large. We also now add the non-ppa debian dependencies directly to the pristine container. I don't *think* there's a downside to this, but let me know if you see one. The end result is much faster, down to about 3min9s for bionic.

Additionally, the output is improved. I combine stdout/stderr into one stream so that it's not so much to dig through when looking for problems, and add timestamps in place of timings around specific operations. I tested this with checkbox-ng test suite and here's what it looks like:
[bionic] [22:51:16] starting container
Device project added to bionic-testing
[bionic] [22:51:35] provisioning container
[bionic] [22:52:36] Starting tests...
Found a test script: ./requirements/001-container-tests-plainbox-egg-info
[bionic] [22:52:51] 001-container-tests-plainbox-egg-info: PASS
Found a test script: ./requirements/container-tests-checkbox-documentation
[bionic] [22:52:53] container-tests-checkbox-documentation: PASS
Found a test script: ./requirements/container-tests-checkbox-ng-unit
[bionic] [22:53:11] container-tests-checkbox-ng-unit: PASS
Found a test script: ./requirements/container-tests-providers-internal
[bionic] [22:53:43] container-tests-providers-internal: PASS
[bionic] [22:53:43] Fixing file permissions in source directory
[bionic] [22:53:44] Destroying container

To actually activate this, we would need to change the jenkins script to run it on a different system, and also change all the checkbox projects that use test-in-lxc to use test-in-lxd instead

To post a comment you must log in.
Revision history for this message
Paul Larson (pwlars) wrote :

The second commit I just added replaces test-in-lxc.sh with the new test-in-lxd.sh. So anything using test-in-lxc should automatically convert over to it. Landing this, and the corresponding patch to the pmr jobs at https://code.launchpad.net/~pwlars/hwcert-jenkins-jobs/+git/hwcert-jenkins-jobs/+merge/359641 may require some manual help, but we'll see.

review: Needs Resubmitting
Revision history for this message
Paul Larson (pwlars) wrote :

New improvements!
- More consistent output, container names on all lines of output
- No more random sleeps, now we just wait for network activation in the containers
- Return of kissiel's parallel execution patch, which makes a much bigger difference now that io overhead is reduced

Total overall testing time on checkbox-ng is now down to about 3m30s

review: Needs Resubmitting
Revision history for this message
Maciej Kisielewski (kissiel) wrote :

Makes sense and looks good! +1

review: Approve

There was an error fetching revisions from git servers. Please try again in a few minutes. If the problem persists, contact Launchpad support.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
The diff is not available at this time. You can reload the page or download it.

Subscribers

People subscribed via source and target branches