Merge lp://staging/~nskaggs/ubuntu-test-cases/increase-ap-timeout into lp://staging/ubuntu-test-cases
Status: | Superseded | ||||
---|---|---|---|---|---|
Proposed branch: | lp://staging/~nskaggs/ubuntu-test-cases/increase-ap-timeout | ||||
Merge into: | lp://staging/ubuntu-test-cases | ||||
Diff against target: |
20582 lines (+19068/-0) (has conflicts) 293 files modified
README-cli.rst (+80/-0) jenkins/custom-demo.py (+32/-0) jenkins/production.py (+92/-0) jenkins/setup_jenkins.py (+222/-0) jenkins/staging.py (+42/-0) jenkins/templates/touch-base.xml.jinja2 (+97/-0) jenkins/templates/touch-master.xml.jinja2 (+80/-0) jenkins/templates/touch-smoke.xml.jinja2 (+178/-0) jenkins/testconfig.py (+208/-0) scripts/assert-image (+30/-0) scripts/combine_results (+124/-0) scripts/dashboard.py (+305/-0) scripts/device_info.py (+100/-0) scripts/get-adb-id (+13/-0) scripts/get-device-info (+37/-0) scripts/jenkins.sh (+189/-0) scripts/junit2utah.py (+70/-0) scripts/ncd_usb.py (+48/-0) scripts/provision.sh (+201/-0) scripts/reboot-and-wait (+54/-0) scripts/recover.py (+124/-0) scripts/run-autopilot-tests.sh (+255/-0) scripts/run-emulator (+97/-0) scripts/run-smoke (+395/-0) scripts/run-touch-upgrade.sh (+46/-0) scripts/statsd.py (+35/-0) selftests/test_junit2utah.py (+49/-0) selftests/test_reboot_and_wait.py (+66/-0) selftests/test_run_smoke.py (+143/-0) selftests/test_statsd.py (+64/-0) tests/bootspeed/bootchart/run.py (+73/-0) tests/bootspeed/bootchart/setup.sh (+4/-0) tests/bootspeed/bootchart/tc_control (+12/-0) tests/bootspeed/master.run (+5/-0) tests/bootspeed/tslist.run (+1/-0) tests/click_image_tests/check_preinstalled_list/check_preinstalled_list.py (+61/-0) tests/click_image_tests/check_preinstalled_list/tc_control (+10/-0) tests/click_image_tests/master.run (+5/-0) tests/click_image_tests/tslist.run (+1/-0) tests/customizations/master.run (+5/-0) tests/customizations/setup.sh (+10/-0) tests/customizations/ts_control (+1/-0) tests/customizations/tslist.auto (+4/-0) tests/default/apport/tc_control (+9/-0) tests/default/ifconfig/tc_control (+13/-0) tests/default/install/tc_control (+12/-0) tests/default/master.run (+5/-0) tests/default/netstat/tc_control (+12/-0) tests/default/ping/pingtest.sh (+12/-0) tests/default/ping/tc_control (+12/-0) tests/default/pwd/tc_control (+9/-0) tests/default/pwd/test.sh (+18/-0) tests/default/route/tc_control (+12/-0) tests/default/systemsettle/systemsettle.sh (+127/-0) tests/default/systemsettle/tc_control (+9/-0) tests/default/ts_control (+3/-0) tests/default/tslist.run (+12/-0) tests/default/uname/tc_control (+12/-0) tests/default/unity8/tc_control (+9/-0) tests/default/vmstat/tc_control (+12/-0) tests/default/whoopsie/tc_control (+9/-0) tests/default/whoopsie/whoopsie-test.sh (+86/-0) tests/eventstat/eventstat/eventstat.sh (+5/-0) tests/eventstat/eventstat/setup.sh (+7/-0) tests/eventstat/eventstat/tc_control (+12/-0) tests/eventstat/master.run (+5/-0) tests/eventstat/tslist.run (+1/-0) tests/health-check/health-check-test-get-pids.py (+38/-0) tests/health-check/health-check-test-pid.py (+198/-0) tests/health-check/master.run (+5/-0) tests/health-check/setup.sh (+10/-0) tests/health-check/test.sh (+15/-0) tests/health-check/thresholds/armv7l/NetworkManager.threshold (+73/-0) tests/health-check/thresholds/armv7l/accounts-daemon.threshold (+73/-0) tests/health-check/thresholds/armv7l/address-book-service.threshold (+73/-0) tests/health-check/thresholds/armv7l/bluetoothd.threshold (+73/-0) tests/health-check/thresholds/armv7l/bridgemhrd.threshold (+73/-0) tests/health-check/thresholds/armv7l/dconf-service.threshold (+73/-0) tests/health-check/thresholds/armv7l/default.threshold (+73/-0) tests/health-check/thresholds/armv7l/dhclient.threshold (+73/-0) tests/health-check/thresholds/armv7l/dnsmasq.threshold (+73/-0) tests/health-check/thresholds/armv7l/evolution-calendar-factory.threshold (+73/-0) tests/health-check/thresholds/armv7l/evolution-source-registry.threshold (+73/-0) tests/health-check/thresholds/armv7l/hud-service.threshold (+73/-0) tests/health-check/thresholds/armv7l/indicator-bluetooth-service.threshold (+73/-0) tests/health-check/thresholds/armv7l/indicator-datetime-service.threshold (+74/-0) tests/health-check/thresholds/armv7l/indicator-location-service.threshold (+74/-0) tests/health-check/thresholds/armv7l/indicator-messages-service.threshold (+73/-0) tests/health-check/thresholds/armv7l/indicator-network-service.threshold (+73/-0) tests/health-check/thresholds/armv7l/indicator-power-service.threshold (+73/-0) tests/health-check/thresholds/armv7l/indicator-secret-agent.threshold (+73/-0) tests/health-check/thresholds/armv7l/indicator-sound-service.threshold (+73/-0) tests/health-check/thresholds/armv7l/maliit-server.threshold (+73/-0) tests/health-check/thresholds/armv7l/mediascanner-service.threshold (+73/-0) tests/health-check/thresholds/armv7l/mission-control-5.threshold (+73/-0) tests/health-check/thresholds/armv7l/mpdecision.threshold (+73/-0) tests/health-check/thresholds/armv7l/mtp-server.threshold (+73/-0) tests/health-check/thresholds/armv7l/ofonod.threshold (+73/-0) tests/health-check/thresholds/armv7l/polkitd.threshold (+73/-0) tests/health-check/thresholds/armv7l/powerd.threshold (+73/-0) tests/health-check/thresholds/armv7l/pulseaudio.threshold (+73/-0) tests/health-check/thresholds/armv7l/qmuxd.threshold (+73/-0) tests/health-check/thresholds/armv7l/rsyslogd.threshold (+73/-0) tests/health-check/thresholds/armv7l/rtkit-daemon.threshold (+73/-0) tests/health-check/thresholds/armv7l/systemd-logind.threshold (+73/-0) tests/health-check/thresholds/armv7l/systemd-udevd.threshold (+73/-0) tests/health-check/thresholds/armv7l/telepathy-ofono.threshold (+73/-0) tests/health-check/thresholds/armv7l/thermald.threshold (+73/-0) tests/health-check/thresholds/armv7l/ubuntu-location-serviced.threshold (+74/-0) tests/health-check/thresholds/armv7l/ueventd.theshold (+73/-0) tests/health-check/thresholds/armv7l/unity-scope-home.threshold (+73/-0) tests/health-check/thresholds/armv7l/unity-scope-loader.threshold (+73/-0) tests/health-check/thresholds/armv7l/upowerd.threshold (+73/-0) tests/health-check/thresholds/armv7l/upstart-dbus-bridge.threshold (+73/-0) tests/health-check/thresholds/armv7l/upstart-event-bridge.threshold (+73/-0) tests/health-check/thresholds/armv7l/upstart-file-bridge.threshold (+73/-0) tests/health-check/thresholds/armv7l/upstart-local-bridge.threshold (+73/-0) tests/health-check/thresholds/armv7l/upstart-property-watcher.threshold (+73/-0) tests/health-check/thresholds/armv7l/upstart-socket-bridge.threshold (+73/-0) tests/health-check/thresholds/armv7l/upstart-udev-bridge.threshold (+73/-0) tests/health-check/thresholds/armv7l/url-dispatcher.threshold (+73/-0) tests/health-check/thresholds/armv7l/whoopsie.threshold (+73/-0) tests/health-check/thresholds/armv7l/window-stack-bridge.threshold (+73/-0) tests/health-check/thresholds/armv7l/wpa_supplicant.threshold (+73/-0) tests/health-check/thresholds/i686/NetworkManager.threshold (+73/-0) tests/health-check/thresholds/i686/accounts-daemon.threshold (+73/-0) tests/health-check/thresholds/i686/acpid.threshold (+73/-0) tests/health-check/thresholds/i686/avahi-daemon:.threshold (+74/-0) tests/health-check/thresholds/i686/colord.threshold (+73/-0) tests/health-check/thresholds/i686/cupsd.threshold (+73/-0) tests/health-check/thresholds/i686/dbus-daemon.threshold.blacklist (+73/-0) tests/health-check/thresholds/i686/dconf-service.threshold (+73/-0) tests/health-check/thresholds/i686/default.threshold (+73/-0) tests/health-check/thresholds/i686/deja-dup-monitor.threshold (+73/-0) tests/health-check/thresholds/i686/dhclient.threshold (+73/-0) tests/health-check/thresholds/i686/dnsmasq.threshold (+73/-0) tests/health-check/thresholds/i686/evolution-calendar-factory.threshold (+73/-0) tests/health-check/thresholds/i686/evolution-source-registry.threshold (+73/-0) tests/health-check/thresholds/i686/gnome-fallback-mount-helper.threshold (+73/-0) tests/health-check/thresholds/i686/gnome-keyring-daemon.threshold.blacklist (+73/-0) tests/health-check/thresholds/i686/gnome-settings-daemon.threshold (+73/-0) tests/health-check/thresholds/i686/gvfs-afc-volume-monitor.threshold (+73/-0) tests/health-check/thresholds/i686/gvfs-gphoto2-volume-monitor.threshold (+73/-0) tests/health-check/thresholds/i686/gvfs-mtp-volume-monitor.threshold (+73/-0) tests/health-check/thresholds/i686/gvfs-udisks2-volume-monitor.threshold (+73/-0) tests/health-check/thresholds/i686/gvfsd-burn.threshold (+73/-0) tests/health-check/thresholds/i686/gvfsd-fuse.threshold (+73/-0) tests/health-check/thresholds/i686/gvfsd.threshold (+73/-0) tests/health-check/thresholds/i686/hud-service.threshold (+73/-0) tests/health-check/thresholds/i686/ibus-dconf.threshold (+73/-0) tests/health-check/thresholds/i686/ibus-engine-simple.threshold (+73/-0) tests/health-check/thresholds/i686/indicator-application-service.threshold (+73/-0) tests/health-check/thresholds/i686/indicator-bluetooth-service.threshold (+73/-0) tests/health-check/thresholds/i686/indicator-datetime-service.threshold (+74/-0) tests/health-check/thresholds/i686/indicator-keyboard-service.threshold (+73/-0) tests/health-check/thresholds/i686/indicator-messages-service.threshold (+73/-0) tests/health-check/thresholds/i686/indicator-power-service.threshold (+73/-0) tests/health-check/thresholds/i686/indicator-printers-service.threshold (+73/-0) tests/health-check/thresholds/i686/indicator-session-service.threshold (+73/-0) tests/health-check/thresholds/i686/indicator-sound-service.threshold (+73/-0) tests/health-check/thresholds/i686/indicator-sync-service.threshold (+73/-0) tests/health-check/thresholds/i686/kerneloops.threshold (+73/-0) tests/health-check/thresholds/i686/modem-manager.threshold (+73/-0) tests/health-check/thresholds/i686/nm-applet.threshold (+73/-0) tests/health-check/thresholds/i686/polkitd.threshold (+73/-0) tests/health-check/thresholds/i686/rtkit-daemon.threshold (+73/-0) tests/health-check/thresholds/i686/systemd-logind.threshold (+73/-0) tests/health-check/thresholds/i686/systemd-udevd.threshold (+73/-0) tests/health-check/thresholds/i686/telepathy-indicator.threshold (+73/-0) tests/health-check/thresholds/i686/udisksd.threshold (+73/-0) tests/health-check/thresholds/i686/unity-panel-service.threshold (+73/-0) tests/health-check/thresholds/i686/update-notifier.threshold (+73/-0) tests/health-check/thresholds/i686/upowerd.threshold (+73/-0) tests/health-check/thresholds/i686/upstart-dbus-bridge.threshold (+73/-0) tests/health-check/thresholds/i686/upstart-event-bridge.threshold (+73/-0) tests/health-check/thresholds/i686/upstart-file-bridge.threshold (+73/-0) tests/health-check/thresholds/i686/upstart-socket-bridge.threshold (+73/-0) tests/health-check/thresholds/i686/upstart-udev-bridge.threshold (+73/-0) tests/health-check/thresholds/i686/whoopsie.threshold (+73/-0) tests/health-check/thresholds/i686/zeitgeist-daemon.threshold (+73/-0) tests/health-check/thresholds/i686/zeitgeist-datahub.threshold (+73/-0) tests/health-check/thresholds/i686/zeitgeist-fts.threshold (+73/-0) tests/health-check/thresholds/x86_64/NetworkManager.threshold (+73/-0) tests/health-check/thresholds/x86_64/accounts-daemon.threshold (+73/-0) tests/health-check/thresholds/x86_64/acpid.threshold (+73/-0) tests/health-check/thresholds/x86_64/avahi-daemon:.threshold (+74/-0) tests/health-check/thresholds/x86_64/colord.threshold (+73/-0) tests/health-check/thresholds/x86_64/cupsd.threshold (+73/-0) tests/health-check/thresholds/x86_64/dbus-daemon.threshold.blacklist (+73/-0) tests/health-check/thresholds/x86_64/dconf-service.threshold (+73/-0) tests/health-check/thresholds/x86_64/default.threshold (+73/-0) tests/health-check/thresholds/x86_64/deja-dup-monitor.threshold (+73/-0) tests/health-check/thresholds/x86_64/dhclient.threshold (+73/-0) tests/health-check/thresholds/x86_64/dnsmasq.threshold (+73/-0) tests/health-check/thresholds/x86_64/evolution-calendar-factory.threshold (+73/-0) tests/health-check/thresholds/x86_64/evolution-source-registry.threshold (+73/-0) tests/health-check/thresholds/x86_64/gnome-fallback-mount-helper.threshold (+73/-0) tests/health-check/thresholds/x86_64/gnome-keyring-daemon.threshold.blacklist (+73/-0) tests/health-check/thresholds/x86_64/gnome-settings-daemon.threshold (+73/-0) tests/health-check/thresholds/x86_64/gvfs-afc-volume-monitor.threshold (+73/-0) tests/health-check/thresholds/x86_64/gvfs-gphoto2-volume-monitor.threshold (+73/-0) tests/health-check/thresholds/x86_64/gvfs-mtp-volume-monitor.threshold (+73/-0) tests/health-check/thresholds/x86_64/gvfs-udisks2-volume-monitor.threshold (+73/-0) tests/health-check/thresholds/x86_64/gvfsd-burn.threshold (+73/-0) tests/health-check/thresholds/x86_64/gvfsd-fuse.threshold (+73/-0) tests/health-check/thresholds/x86_64/gvfsd.threshold (+73/-0) tests/health-check/thresholds/x86_64/hud-service.threshold (+73/-0) tests/health-check/thresholds/x86_64/ibus-dconf.threshold (+73/-0) tests/health-check/thresholds/x86_64/ibus-engine-simple.threshold (+73/-0) tests/health-check/thresholds/x86_64/indicator-application-service.threshold (+73/-0) tests/health-check/thresholds/x86_64/indicator-bluetooth-service.threshold (+73/-0) tests/health-check/thresholds/x86_64/indicator-datetime-service.threshold (+74/-0) tests/health-check/thresholds/x86_64/indicator-keyboard-service.threshold (+73/-0) tests/health-check/thresholds/x86_64/indicator-messages-service.threshold (+73/-0) tests/health-check/thresholds/x86_64/indicator-power-service.threshold (+73/-0) tests/health-check/thresholds/x86_64/indicator-printers-service.threshold (+73/-0) tests/health-check/thresholds/x86_64/indicator-session-service.threshold (+73/-0) tests/health-check/thresholds/x86_64/indicator-sound-service.threshold (+73/-0) tests/health-check/thresholds/x86_64/indicator-sync-service.threshold (+73/-0) tests/health-check/thresholds/x86_64/kerneloops.threshold (+73/-0) tests/health-check/thresholds/x86_64/modem-manager.threshold (+73/-0) tests/health-check/thresholds/x86_64/nm-applet.threshold (+73/-0) tests/health-check/thresholds/x86_64/polkitd.threshold (+73/-0) tests/health-check/thresholds/x86_64/rtkit-daemon.threshold (+73/-0) tests/health-check/thresholds/x86_64/systemd-logind.threshold (+73/-0) tests/health-check/thresholds/x86_64/systemd-udevd.threshold (+73/-0) tests/health-check/thresholds/x86_64/telepathy-indicator.threshold (+73/-0) tests/health-check/thresholds/x86_64/udisksd.threshold (+73/-0) tests/health-check/thresholds/x86_64/unity-panel-service.threshold (+73/-0) tests/health-check/thresholds/x86_64/update-notifier.threshold (+73/-0) tests/health-check/thresholds/x86_64/upowerd.threshold (+73/-0) tests/health-check/thresholds/x86_64/upstart-dbus-bridge.threshold (+73/-0) tests/health-check/thresholds/x86_64/upstart-event-bridge.threshold (+73/-0) tests/health-check/thresholds/x86_64/upstart-file-bridge.threshold (+73/-0) tests/health-check/thresholds/x86_64/upstart-socket-bridge.threshold (+73/-0) tests/health-check/thresholds/x86_64/upstart-udev-bridge.threshold (+73/-0) tests/health-check/thresholds/x86_64/whoopsie.threshold (+73/-0) tests/health-check/thresholds/x86_64/zeitgeist-daemon.threshold (+73/-0) tests/health-check/thresholds/x86_64/zeitgeist-datahub.threshold (+73/-0) tests/health-check/thresholds/x86_64/zeitgeist-fts.threshold (+73/-0) tests/health-check/ts_control (+2/-0) tests/health-check/tslist.auto (+3/-0) tests/memevent/master.run (+5/-0) tests/memevent/setup.sh (+8/-0) tests/memevent/ts_control (+1/-0) tests/memevent/tslist.auto (+7/-0) tests/memevent/ubuntu_test_cases/__init__.py (+6/-0) tests/memevent/ubuntu_test_cases/memory_usage_measurement/__init__.py (+1/-0) tests/memevent/ubuntu_test_cases/memory_usage_measurement/apps/__init__.py (+21/-0) tests/memevent/ubuntu_test_cases/memory_usage_measurement/apps/browser.py (+103/-0) tests/memevent/ubuntu_test_cases/memory_usage_measurement/apps/camera.py (+58/-0) tests/memevent/ubuntu_test_cases/memory_usage_measurement/apps/gallery.py (+41/-0) tests/memevent/ubuntu_test_cases/memory_usage_measurement/apps/media_player.py (+53/-0) tests/memevent/ubuntu_test_cases/memory_usage_measurement/matchers.py (+37/-0) tests/memevent/ubuntu_test_cases/memory_usage_measurement/probes.py (+169/-0) tests/memevent/ubuntu_test_cases/memory_usage_measurement/smem-tabs (+687/-0) tests/memevent/ubuntu_test_cases/memory_usage_measurement/tests/__init__.py (+1/-0) tests/memevent/ubuntu_test_cases/memory_usage_measurement/tests/test_memory_usage.py (+79/-0) tests/sdk/master.run (+5/-0) tests/sdk/run_test.sh (+20/-0) tests/sdk/setup.sh (+12/-0) tests/sdk/ts_control (+1/-0) tests/sdk/tslist.auto (+3/-0) tests/security/master.run (+5/-0) tests/security/run_test.sh (+26/-0) tests/security/setup.sh (+12/-0) tests/security/ts_control (+1/-0) tests/security/tslist.auto (+6/-0) tests/smem/master.run (+15/-0) tests/smem/smem/smem-tabs (+687/-0) tests/smem/smem/smem.sh (+12/-0) tests/smem/smem/tc_control (+11/-0) tests/smem/tslist.run (+1/-0) tests/suspend-blocker/master.run (+5/-0) tests/suspend-blocker/suspend-blocker/README (+7/-0) tests/suspend-blocker/suspend-blocker/setup.sh (+10/-0) tests/suspend-blocker/suspend-blocker/susblock.conf (+5/-0) tests/suspend-blocker/suspend-blocker/suspend-blocker.sh (+25/-0) tests/suspend-blocker/suspend-blocker/tc_control (+16/-0) tests/suspend-blocker/tslist.run (+1/-0) tests/systemsettle/systemsettle-after/tc_control (+9/-0) tests/systemsettle/systemsettle-before/tc_control (+9/-0) tests/systemsettle/systemsettle.sh (+127/-0) tests/systemsettle/tslist.run (+2/-0) utils/host/adb-shell (+23/-0) utils/host/autopilot-list (+23/-0) utils/host/autopilot-run (+12/-0) utils/host/prepare-autopilot-test.sh (+7/-0) utils/host/reboot-and-unlock.sh (+15/-0) utils/target/autopilot-list (+13/-0) utils/target/autopilot-run (+6/-0) utils/target/check-clickhook-rules (+10/-0) utils/target/prepare-autopilot-test.sh (+16/-0) Conflict adding file scripts. Moved existing file to scripts.moved. |
||||
To merge this branch: | bzr merge lp://staging/~nskaggs/ubuntu-test-cases/increase-ap-timeout | ||||
Related bugs: |
|
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Ubuntu Test Case Developers | Pending | ||
Review via email: mp+235705@code.staging.launchpad.net |
This proposal has been superseded by a proposal from 2014-09-23.
Commit message
This increases the default timeout for autopilot test runs to 'long', which is currently defined as 30 seconds.
Description of the change
This increases the default timeout for autopilot test runs to 'long', which is currently defined as 30 seconds. This is intended to prevent bug 369990, but it will affect all timeouts for autopilot, including wait_select's, which are currently timing out after the default timeout of 10 seconds. This could cause tests to take an additional 20 seconds to fail (when they do).
Test case authors that require or expect the default timeout would also be affected. In general waiting longer shouldn't impose any additional risk beyond taking longer to fail (any assertions made on a specific time should already be defined inside a test). This is somewhat a heavy handed approach towards fixing the bug, and if possible perhaps it could be applied on a individual testsuite basis.
The underlying causes for why applications don't launch within 10 seconds is concerning and worthy of study, but that is outside of the scope of running these tests. I would rather see more reflective results of the application than using it as a proxy to potential app startup performance concerns. As to tests not reflecting reality and hiding long startup issues, I would only point out the apps incur some additional overhead to load with autopilot (including memory), and thus take longer to launch than launching an application without introspection. Thus this is not reflective of a real world user experience from the start.
Unmerged revisions
- 301. By Nicholas Skaggs
-
fix unintended whitespace changes
- 300. By Nicholas Skaggs
-
change default AP timeout to long (30 seconds), fix bug 1369990
- 299. By Paul Larson
-
remove calendar app
- 298. By Paul Larson
-
run health check with sudo
- 297. By Paul Larson
-
Add python3-dateutil for calendar tests
- 296. By Paul Larson
-
Apply the same workaround to phablet-network that we previously used for phablet-
click-test- setup - 295. By Paul Larson
-
match a bit more of the upstart session env variable
- 294. By Paul Larson
-
correct for some oddness that seems to only happen when running it from a script
- 293. By Paul Larson
-
Don't run phablet-
click-test- setup until we're sure it should work - 292. By Paul Larson
-
retry phablet-
click-test- setup if the session is not yet available