Merge lp://staging/~toykeeper/ubuntu-power-tests/python-refactor-tk into lp://staging/ubuntu-power-tests
- python-refactor-tk
- Merge into trunk
Status: | Merged |
---|---|
Approved by: | Selene ToyKeeper |
Approved revision: | 76 |
Merged at revision: | 15 |
Proposed branch: | lp://staging/~toykeeper/ubuntu-power-tests/python-refactor-tk |
Merge into: | lp://staging/ubuntu-power-tests |
Diff against target: |
3006 lines (+2655/-144) 21 files modified
Makefile (+2/-0) README (+141/-66) channels.lst (+4/-0) debian/tests/power_usage_bt_on_wifi_disconnected (+13/-0) debian/tests/power_usage_display_on (+10/-6) debian/tests/power_usage_flight_mode_on (+12/-0) debian/tests/power_usage_gps_app_backgrounded (+8/-0) debian/tests/power_usage_gps_app_closed (+9/-0) debian/tests/power_usage_idle (+12/-0) debian/tests/prepare-device (+17/-0) devices.lst (+2/-0) dut-monitor.sh (+34/-0) flash/arale/urls (+1/-0) flash/flash.sh (+305/-0) flash/krillin/urls (+2/-0) power-graph.py (+180/-0) power.cfg.example (+27/-0) power.py (+1137/-72) run-in-background.sh (+23/-0) test-new-builds.sh (+168/-0) tests/unit/test_power.py (+548/-0) |
To merge this branch: | bzr merge lp://staging/~toykeeper/ubuntu-power-tests/python-refactor-tk |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
PS Jenkins bot | continuous-integration | Approve | |
Selene ToyKeeper (community) | Approve | ||
Max Brustkern (community) | Approve | ||
Review via email: mp+263325@code.staging.launchpad.net |
Commit message
Update the tests to run in python using a class to manage many aspects of the testing.
Description of the change
Early preview of the python rewrite+refactor. Still changing rapidly.
PS Jenkins bot (ps-jenkins) wrote : | # |
- 53. By Selene ToyKeeper
-
Fixed the power graph size / DPI.
Moved the time origin to the test begin tag. - 54. By Selene ToyKeeper
-
Fixed incorrect argument order for a non-refactored function.
Added a wait_after test option, to keep measuring for a bit before rebooting.
Tried to calibrate display_on timing for both arale and krillin,
but they seem to need different timings.
Added my krillin to the device list.
PS Jenkins bot (ps-jenkins) wrote : | # |
FAILED: Continuous integration, rev:54
No commit message was specified in the merge proposal. Click on the following link and set the commit message (if you want a jenkins rebuild you need to trigger it yourself):
https:/
http://
Executed test runs:
SUCCESS: http://
SUCCESS: http://
SUCCESS: http://
Click here to trigger a rebuild:
http://
- 55. By Selene ToyKeeper
-
Refactor milestone: Uses TestResult object and keeps adt logs.
Moved a variety of things to TestResult class...
Keep entire result set in a dir, with a subdir per run.
Made partial() arguments explicit to avoid ordering issues.
Renamed run() to avoid name collisions.
(still need to remove it entirely and merge the two sets of run()s)
PS Jenkins bot (ps-jenkins) wrote : | # |
FAILED: Continuous integration, rev:55
No commit message was specified in the merge proposal. Click on the following link and set the commit message (if you want a jenkins rebuild you need to trigger it yourself):
https:/
http://
Executed test runs:
SUCCESS: http://
SUCCESS: http://
SUCCESS: http://
Click here to trigger a rebuild:
http://
- 56. By Selene ToyKeeper
-
Added logging to file(s) during test runs.
Fixed code which tried to directly access good_runs.
(which is intended to be true/false, not a list of run objects) - 57. By Selene ToyKeeper
-
Extended timeout so krillin tests will work. (setup on it is slow!)
Forced a visible axis at power=0 on graphs, in case it goes negative.
Made graph bounds smarter, based mostly on the samples where the
device wasn't plugged in at the time. - 58. By Selene ToyKeeper
-
Added adt log verification, so it requires adt to pass or it will ignore
that test run and try again (up to the max tries). - 59. By Selene ToyKeeper
-
Fixed the case where a graph has no "unplugged" data.
Made it easier to switch between devices in new-build test script. - 60. By Selene ToyKeeper
-
Fixed graph layout.
- 61. By Selene ToyKeeper
-
Finally got adt-run --keep-
screen- active to work.
display_on test is finally working.
Uses value from test file instead of test filename to control behavior.
PS Jenkins bot (ps-jenkins) wrote : | # |
FAILED: Continuous integration, rev:61
No commit message was specified in the merge proposal. Click on the following link and set the commit message (if you want a jenkins rebuild you need to trigger it yourself):
https:/
http://
Executed test runs:
SUCCESS: http://
SUCCESS: http://
SUCCESS: http://
Click here to trigger a rebuild:
http://
- 62. By Selene ToyKeeper
-
Made grapher work again with legacy graphs.
(let it use autoscale if no USB tags were found) - 63. By Selene ToyKeeper
-
power.py now collects crash dumps and screenshots and the date.
Adjusted test parameters to allow for more failures
(since the tests now fail a lot more often).
Cleaned up some power test code.
PS Jenkins bot (ps-jenkins) wrote : | # |
FAILED: Continuous integration, rev:63
No commit message was specified in the merge proposal. Click on the following link and set the commit message (if you want a jenkins rebuild you need to trigger it yourself):
https:/
http://
Executed test runs:
SUCCESS: http://
SUCCESS: http://
SUCCESS: http://
Click here to trigger a rebuild:
http://
- 64. By Selene ToyKeeper
-
Added graphs for bad runs too.
Might help reveal info for fixing the bugs which cause failures. - 65. By Selene ToyKeeper
-
Added my flash script and minimal supporting files.
(includes recovery images)
Cleaned up the flash script a little, but it still has some options
we don't actually use for power testing. - 66. By Selene ToyKeeper
-
Added date sanity checking. Phone must be within 24 hours of host.
This catches the boots where the phone thinks it's January 1 2010. - 67. By Selene ToyKeeper
-
Added on-device monitoring, at least the basics.
(took a while to find a way to make it run in the background
and not die as soon as USB gets unplugged)
Made power.py log full tracebacks instead of just the error string.
Now ensures the clock is correct before starting a test
(though it might skew again on pre-test reboot, d'oh).
Moved adt verify check lower so it won't prevent collecting logs
when adt fails.
Added an attempt to collect logs after failure, just in case it
wasn't done before.
Added a reboot after failure to avoid cases where the error also
breaks the next test run.
Removed the need to pass anything to usb_connect / usb_disconnect.
Fixed the case where a bad run has no log to graph.
Now collects syslog too.
Refactored a little to allow starting on-device monitoring scripts
at the appropriate point in the code.
Replaced asserts with exceptions which provide more human-friendly errors.Made test-new-builds accept a device name/list on the command line.
Also made it not wait after running tests, but instead poll again immediately.Increased display_on test time to 30 seconds again
(since the --keep-screen- active thing is working). - 68. By Selene ToyKeeper
-
Er, stop trying to grab a file which never exists. :)
(I forgot I moved it)
PS Jenkins bot (ps-jenkins) wrote : | # |
FAILED: Continuous integration, rev:68
No commit message was specified in the merge proposal. Click on the following link and set the commit message (if you want a jenkins rebuild you need to trigger it yourself):
https:/
http://
Executed test runs:
SUCCESS: http://
SUCCESS: http://
SUCCESS: http://
Click here to trigger a rebuild:
http://
- 69. By Selene ToyKeeper
-
Moved fix_clock() and date-pre to adt's prepare-device script.
Added pretty-printed stats to the 'stats' result file,
since this is an important part of the results and should be visible.
Took all the detail out of graph filenames, since the parent dir has it.
PS Jenkins bot (ps-jenkins) wrote : | # |
FAILED: Continuous integration, rev:69
No commit message was specified in the merge proposal. Click on the following link and set the commit message (if you want a jenkins rebuild you need to trigger it yourself):
https:/
http://
Executed test runs:
SUCCESS: http://
SUCCESS: http://
SUCCESS: http://
Click here to trigger a rebuild:
http://
- 70. By Selene ToyKeeper
-
Hah, make sure the DUT is plugged in before trying to collect logs post-fail.
Also log more detail about post-fail exceptions.
Grab dmesg after each test run.
PS Jenkins bot (ps-jenkins) wrote : | # |
FAILED: Continuous integration, rev:70
No commit message was specified in the merge proposal. Click on the following link and set the commit message (if you want a jenkins rebuild you need to trigger it yourself):
https:/
http://
Executed test runs:
SUCCESS: http://
SUCCESS: http://
SUCCESS: http://
Click here to trigger a rebuild:
http://
Max Brustkern (nuclearbob) wrote : | # |
I've got a few comments, but mostly this is working when I run jobs with it, so I think we should be on track to land it soon.
- 71. By Selene ToyKeeper
-
Check the date later, because if it fails it prevents collecting other logs.
PS Jenkins bot (ps-jenkins) wrote : | # |
FAILED: Continuous integration, rev:71
No commit message was specified in the merge proposal. Click on the following link and set the commit message (if you want a jenkins rebuild you need to trigger it yourself):
https:/
http://
Executed test runs:
SUCCESS: http://
SUCCESS: http://
SUCCESS: http://
Click here to trigger a rebuild:
http://
Selene ToyKeeper (toykeeper) wrote : | # |
I started to fix each item as it came up, then found some which seemed better to do in a later merge... but then found a bunch more to do now. I think I'll have to finish this in the morning.
- 72. By Selene ToyKeeper
-
Oops, tell phablet-screenshot which device to use.
Also, show output from screenshot command for debugging.
Added -h / --help to usage docs. - 73. By Selene ToyKeeper
-
Changed channel order so it'll test rc-proposed last.
This helps reduce how much time the power supply fan spends spinning
between tests, since rc-proposed behaves better and is less likely to
leave the device in a high-power state. - 74. By Selene ToyKeeper
-
Updated the README quite a bit.
Removed outdated info and merged my notes into the main document. - 75. By Selene ToyKeeper
-
Addressed remaining review comments.
Removed a variety of dead code.
Replaced bash with sh.
Pep8'd the graph script.
Renamed the good_runs variables to make the purpose more clear.
Removed good_count.
Switched run_cmd to use .communicate(); should be safer.
Ran a single test to make sure nothing huge broke.
Selene ToyKeeper (toykeeper) wrote : | # |
Okay, nearly all comments addressed in the code, one smoke test executed, and a new rev pushed.
PS Jenkins bot (ps-jenkins) wrote : | # |
FAILED: Continuous integration, rev:75
No commit message was specified in the merge proposal. Click on the following link and set the commit message (if you want a jenkins rebuild you need to trigger it yourself):
https:/
http://
Executed test runs:
SUCCESS: http://
SUCCESS: http://
SUCCESS: http://
Click here to trigger a rebuild:
http://
Max Brustkern (nuclearbob) wrote : | # |
My automated jobs have been running with this branch and succeeding last night and this morning, so I think we should be ready to merge.
- 76. By Selene ToyKeeper
-
Save a little extra network info before each test.
Selene ToyKeeper (toykeeper) wrote : | # |
I pushed a tiny change to make our bots try again.
PS Jenkins bot (ps-jenkins) wrote : | # |
PASSED: Continuous integration, rev:76
http://
Executed test runs:
SUCCESS: http://
SUCCESS: http://
SUCCESS: http://
Click here to trigger a rebuild:
http://
FAILED: Continuous integration, rev:52 /code.launchpad .net/~toykeeper /ubuntu- power-tests/ python- refactor- tk/+merge/ 263325/ +edit-commit- message
No commit message was specified in the merge proposal. Click on the following link and set the commit message (if you want a jenkins rebuild you need to trigger it yourself):
https:/
http:// jenkins. qa.ubuntu. com/job/ ubuntu- power-tests- ci/29/ jenkins. qa.ubuntu. com/job/ ubuntu- power-tests- vivid-amd64- ci/29 jenkins. qa.ubuntu. com/job/ ubuntu- power-tests- vivid-armhf- ci/29 jenkins. qa.ubuntu. com/job/ ubuntu- power-tests- vivid-i386- ci/29
Executed test runs:
SUCCESS: http://
SUCCESS: http://
SUCCESS: http://
Click here to trigger a rebuild: s-jenkins. ubuntu- ci:8080/ job/ubuntu- power-tests- ci/29/rebuild
http://