Merge lp://staging/~nskaggs/ubuntu-test-cases/increase-ap-timeout into lp://staging/ubuntu-test-cases

Proposed by Nicholas Skaggs
Status: Superseded
Proposed branch: lp://staging/~nskaggs/ubuntu-test-cases/increase-ap-timeout
Merge into: lp://staging/ubuntu-test-cases
Diff against target: 20582 lines (+19068/-0) (has conflicts)
293 files modified
README-cli.rst (+80/-0)
jenkins/custom-demo.py (+32/-0)
jenkins/production.py (+92/-0)
jenkins/setup_jenkins.py (+222/-0)
jenkins/staging.py (+42/-0)
jenkins/templates/touch-base.xml.jinja2 (+97/-0)
jenkins/templates/touch-master.xml.jinja2 (+80/-0)
jenkins/templates/touch-smoke.xml.jinja2 (+178/-0)
jenkins/testconfig.py (+208/-0)
scripts/assert-image (+30/-0)
scripts/combine_results (+124/-0)
scripts/dashboard.py (+305/-0)
scripts/device_info.py (+100/-0)
scripts/get-adb-id (+13/-0)
scripts/get-device-info (+37/-0)
scripts/jenkins.sh (+189/-0)
scripts/junit2utah.py (+70/-0)
scripts/ncd_usb.py (+48/-0)
scripts/provision.sh (+201/-0)
scripts/reboot-and-wait (+54/-0)
scripts/recover.py (+124/-0)
scripts/run-autopilot-tests.sh (+255/-0)
scripts/run-emulator (+97/-0)
scripts/run-smoke (+395/-0)
scripts/run-touch-upgrade.sh (+46/-0)
scripts/statsd.py (+35/-0)
selftests/test_junit2utah.py (+49/-0)
selftests/test_reboot_and_wait.py (+66/-0)
selftests/test_run_smoke.py (+143/-0)
selftests/test_statsd.py (+64/-0)
tests/bootspeed/bootchart/run.py (+73/-0)
tests/bootspeed/bootchart/setup.sh (+4/-0)
tests/bootspeed/bootchart/tc_control (+12/-0)
tests/bootspeed/master.run (+5/-0)
tests/bootspeed/tslist.run (+1/-0)
tests/click_image_tests/check_preinstalled_list/check_preinstalled_list.py (+61/-0)
tests/click_image_tests/check_preinstalled_list/tc_control (+10/-0)
tests/click_image_tests/master.run (+5/-0)
tests/click_image_tests/tslist.run (+1/-0)
tests/customizations/master.run (+5/-0)
tests/customizations/setup.sh (+10/-0)
tests/customizations/ts_control (+1/-0)
tests/customizations/tslist.auto (+4/-0)
tests/default/apport/tc_control (+9/-0)
tests/default/ifconfig/tc_control (+13/-0)
tests/default/install/tc_control (+12/-0)
tests/default/master.run (+5/-0)
tests/default/netstat/tc_control (+12/-0)
tests/default/ping/pingtest.sh (+12/-0)
tests/default/ping/tc_control (+12/-0)
tests/default/pwd/tc_control (+9/-0)
tests/default/pwd/test.sh (+18/-0)
tests/default/route/tc_control (+12/-0)
tests/default/systemsettle/systemsettle.sh (+127/-0)
tests/default/systemsettle/tc_control (+9/-0)
tests/default/ts_control (+3/-0)
tests/default/tslist.run (+12/-0)
tests/default/uname/tc_control (+12/-0)
tests/default/unity8/tc_control (+9/-0)
tests/default/vmstat/tc_control (+12/-0)
tests/default/whoopsie/tc_control (+9/-0)
tests/default/whoopsie/whoopsie-test.sh (+86/-0)
tests/eventstat/eventstat/eventstat.sh (+5/-0)
tests/eventstat/eventstat/setup.sh (+7/-0)
tests/eventstat/eventstat/tc_control (+12/-0)
tests/eventstat/master.run (+5/-0)
tests/eventstat/tslist.run (+1/-0)
tests/health-check/health-check-test-get-pids.py (+38/-0)
tests/health-check/health-check-test-pid.py (+198/-0)
tests/health-check/master.run (+5/-0)
tests/health-check/setup.sh (+10/-0)
tests/health-check/test.sh (+15/-0)
tests/health-check/thresholds/armv7l/NetworkManager.threshold (+73/-0)
tests/health-check/thresholds/armv7l/accounts-daemon.threshold (+73/-0)
tests/health-check/thresholds/armv7l/address-book-service.threshold (+73/-0)
tests/health-check/thresholds/armv7l/bluetoothd.threshold (+73/-0)
tests/health-check/thresholds/armv7l/bridgemhrd.threshold (+73/-0)
tests/health-check/thresholds/armv7l/dconf-service.threshold (+73/-0)
tests/health-check/thresholds/armv7l/default.threshold (+73/-0)
tests/health-check/thresholds/armv7l/dhclient.threshold (+73/-0)
tests/health-check/thresholds/armv7l/dnsmasq.threshold (+73/-0)
tests/health-check/thresholds/armv7l/evolution-calendar-factory.threshold (+73/-0)
tests/health-check/thresholds/armv7l/evolution-source-registry.threshold (+73/-0)
tests/health-check/thresholds/armv7l/hud-service.threshold (+73/-0)
tests/health-check/thresholds/armv7l/indicator-bluetooth-service.threshold (+73/-0)
tests/health-check/thresholds/armv7l/indicator-datetime-service.threshold (+74/-0)
tests/health-check/thresholds/armv7l/indicator-location-service.threshold (+74/-0)
tests/health-check/thresholds/armv7l/indicator-messages-service.threshold (+73/-0)
tests/health-check/thresholds/armv7l/indicator-network-service.threshold (+73/-0)
tests/health-check/thresholds/armv7l/indicator-power-service.threshold (+73/-0)
tests/health-check/thresholds/armv7l/indicator-secret-agent.threshold (+73/-0)
tests/health-check/thresholds/armv7l/indicator-sound-service.threshold (+73/-0)
tests/health-check/thresholds/armv7l/maliit-server.threshold (+73/-0)
tests/health-check/thresholds/armv7l/mediascanner-service.threshold (+73/-0)
tests/health-check/thresholds/armv7l/mission-control-5.threshold (+73/-0)
tests/health-check/thresholds/armv7l/mpdecision.threshold (+73/-0)
tests/health-check/thresholds/armv7l/mtp-server.threshold (+73/-0)
tests/health-check/thresholds/armv7l/ofonod.threshold (+73/-0)
tests/health-check/thresholds/armv7l/polkitd.threshold (+73/-0)
tests/health-check/thresholds/armv7l/powerd.threshold (+73/-0)
tests/health-check/thresholds/armv7l/pulseaudio.threshold (+73/-0)
tests/health-check/thresholds/armv7l/qmuxd.threshold (+73/-0)
tests/health-check/thresholds/armv7l/rsyslogd.threshold (+73/-0)
tests/health-check/thresholds/armv7l/rtkit-daemon.threshold (+73/-0)
tests/health-check/thresholds/armv7l/systemd-logind.threshold (+73/-0)
tests/health-check/thresholds/armv7l/systemd-udevd.threshold (+73/-0)
tests/health-check/thresholds/armv7l/telepathy-ofono.threshold (+73/-0)
tests/health-check/thresholds/armv7l/thermald.threshold (+73/-0)
tests/health-check/thresholds/armv7l/ubuntu-location-serviced.threshold (+74/-0)
tests/health-check/thresholds/armv7l/ueventd.theshold (+73/-0)
tests/health-check/thresholds/armv7l/unity-scope-home.threshold (+73/-0)
tests/health-check/thresholds/armv7l/unity-scope-loader.threshold (+73/-0)
tests/health-check/thresholds/armv7l/upowerd.threshold (+73/-0)
tests/health-check/thresholds/armv7l/upstart-dbus-bridge.threshold (+73/-0)
tests/health-check/thresholds/armv7l/upstart-event-bridge.threshold (+73/-0)
tests/health-check/thresholds/armv7l/upstart-file-bridge.threshold (+73/-0)
tests/health-check/thresholds/armv7l/upstart-local-bridge.threshold (+73/-0)
tests/health-check/thresholds/armv7l/upstart-property-watcher.threshold (+73/-0)
tests/health-check/thresholds/armv7l/upstart-socket-bridge.threshold (+73/-0)
tests/health-check/thresholds/armv7l/upstart-udev-bridge.threshold (+73/-0)
tests/health-check/thresholds/armv7l/url-dispatcher.threshold (+73/-0)
tests/health-check/thresholds/armv7l/whoopsie.threshold (+73/-0)
tests/health-check/thresholds/armv7l/window-stack-bridge.threshold (+73/-0)
tests/health-check/thresholds/armv7l/wpa_supplicant.threshold (+73/-0)
tests/health-check/thresholds/i686/NetworkManager.threshold (+73/-0)
tests/health-check/thresholds/i686/accounts-daemon.threshold (+73/-0)
tests/health-check/thresholds/i686/acpid.threshold (+73/-0)
tests/health-check/thresholds/i686/avahi-daemon:.threshold (+74/-0)
tests/health-check/thresholds/i686/colord.threshold (+73/-0)
tests/health-check/thresholds/i686/cupsd.threshold (+73/-0)
tests/health-check/thresholds/i686/dbus-daemon.threshold.blacklist (+73/-0)
tests/health-check/thresholds/i686/dconf-service.threshold (+73/-0)
tests/health-check/thresholds/i686/default.threshold (+73/-0)
tests/health-check/thresholds/i686/deja-dup-monitor.threshold (+73/-0)
tests/health-check/thresholds/i686/dhclient.threshold (+73/-0)
tests/health-check/thresholds/i686/dnsmasq.threshold (+73/-0)
tests/health-check/thresholds/i686/evolution-calendar-factory.threshold (+73/-0)
tests/health-check/thresholds/i686/evolution-source-registry.threshold (+73/-0)
tests/health-check/thresholds/i686/gnome-fallback-mount-helper.threshold (+73/-0)
tests/health-check/thresholds/i686/gnome-keyring-daemon.threshold.blacklist (+73/-0)
tests/health-check/thresholds/i686/gnome-settings-daemon.threshold (+73/-0)
tests/health-check/thresholds/i686/gvfs-afc-volume-monitor.threshold (+73/-0)
tests/health-check/thresholds/i686/gvfs-gphoto2-volume-monitor.threshold (+73/-0)
tests/health-check/thresholds/i686/gvfs-mtp-volume-monitor.threshold (+73/-0)
tests/health-check/thresholds/i686/gvfs-udisks2-volume-monitor.threshold (+73/-0)
tests/health-check/thresholds/i686/gvfsd-burn.threshold (+73/-0)
tests/health-check/thresholds/i686/gvfsd-fuse.threshold (+73/-0)
tests/health-check/thresholds/i686/gvfsd.threshold (+73/-0)
tests/health-check/thresholds/i686/hud-service.threshold (+73/-0)
tests/health-check/thresholds/i686/ibus-dconf.threshold (+73/-0)
tests/health-check/thresholds/i686/ibus-engine-simple.threshold (+73/-0)
tests/health-check/thresholds/i686/indicator-application-service.threshold (+73/-0)
tests/health-check/thresholds/i686/indicator-bluetooth-service.threshold (+73/-0)
tests/health-check/thresholds/i686/indicator-datetime-service.threshold (+74/-0)
tests/health-check/thresholds/i686/indicator-keyboard-service.threshold (+73/-0)
tests/health-check/thresholds/i686/indicator-messages-service.threshold (+73/-0)
tests/health-check/thresholds/i686/indicator-power-service.threshold (+73/-0)
tests/health-check/thresholds/i686/indicator-printers-service.threshold (+73/-0)
tests/health-check/thresholds/i686/indicator-session-service.threshold (+73/-0)
tests/health-check/thresholds/i686/indicator-sound-service.threshold (+73/-0)
tests/health-check/thresholds/i686/indicator-sync-service.threshold (+73/-0)
tests/health-check/thresholds/i686/kerneloops.threshold (+73/-0)
tests/health-check/thresholds/i686/modem-manager.threshold (+73/-0)
tests/health-check/thresholds/i686/nm-applet.threshold (+73/-0)
tests/health-check/thresholds/i686/polkitd.threshold (+73/-0)
tests/health-check/thresholds/i686/rtkit-daemon.threshold (+73/-0)
tests/health-check/thresholds/i686/systemd-logind.threshold (+73/-0)
tests/health-check/thresholds/i686/systemd-udevd.threshold (+73/-0)
tests/health-check/thresholds/i686/telepathy-indicator.threshold (+73/-0)
tests/health-check/thresholds/i686/udisksd.threshold (+73/-0)
tests/health-check/thresholds/i686/unity-panel-service.threshold (+73/-0)
tests/health-check/thresholds/i686/update-notifier.threshold (+73/-0)
tests/health-check/thresholds/i686/upowerd.threshold (+73/-0)
tests/health-check/thresholds/i686/upstart-dbus-bridge.threshold (+73/-0)
tests/health-check/thresholds/i686/upstart-event-bridge.threshold (+73/-0)
tests/health-check/thresholds/i686/upstart-file-bridge.threshold (+73/-0)
tests/health-check/thresholds/i686/upstart-socket-bridge.threshold (+73/-0)
tests/health-check/thresholds/i686/upstart-udev-bridge.threshold (+73/-0)
tests/health-check/thresholds/i686/whoopsie.threshold (+73/-0)
tests/health-check/thresholds/i686/zeitgeist-daemon.threshold (+73/-0)
tests/health-check/thresholds/i686/zeitgeist-datahub.threshold (+73/-0)
tests/health-check/thresholds/i686/zeitgeist-fts.threshold (+73/-0)
tests/health-check/thresholds/x86_64/NetworkManager.threshold (+73/-0)
tests/health-check/thresholds/x86_64/accounts-daemon.threshold (+73/-0)
tests/health-check/thresholds/x86_64/acpid.threshold (+73/-0)
tests/health-check/thresholds/x86_64/avahi-daemon:.threshold (+74/-0)
tests/health-check/thresholds/x86_64/colord.threshold (+73/-0)
tests/health-check/thresholds/x86_64/cupsd.threshold (+73/-0)
tests/health-check/thresholds/x86_64/dbus-daemon.threshold.blacklist (+73/-0)
tests/health-check/thresholds/x86_64/dconf-service.threshold (+73/-0)
tests/health-check/thresholds/x86_64/default.threshold (+73/-0)
tests/health-check/thresholds/x86_64/deja-dup-monitor.threshold (+73/-0)
tests/health-check/thresholds/x86_64/dhclient.threshold (+73/-0)
tests/health-check/thresholds/x86_64/dnsmasq.threshold (+73/-0)
tests/health-check/thresholds/x86_64/evolution-calendar-factory.threshold (+73/-0)
tests/health-check/thresholds/x86_64/evolution-source-registry.threshold (+73/-0)
tests/health-check/thresholds/x86_64/gnome-fallback-mount-helper.threshold (+73/-0)
tests/health-check/thresholds/x86_64/gnome-keyring-daemon.threshold.blacklist (+73/-0)
tests/health-check/thresholds/x86_64/gnome-settings-daemon.threshold (+73/-0)
tests/health-check/thresholds/x86_64/gvfs-afc-volume-monitor.threshold (+73/-0)
tests/health-check/thresholds/x86_64/gvfs-gphoto2-volume-monitor.threshold (+73/-0)
tests/health-check/thresholds/x86_64/gvfs-mtp-volume-monitor.threshold (+73/-0)
tests/health-check/thresholds/x86_64/gvfs-udisks2-volume-monitor.threshold (+73/-0)
tests/health-check/thresholds/x86_64/gvfsd-burn.threshold (+73/-0)
tests/health-check/thresholds/x86_64/gvfsd-fuse.threshold (+73/-0)
tests/health-check/thresholds/x86_64/gvfsd.threshold (+73/-0)
tests/health-check/thresholds/x86_64/hud-service.threshold (+73/-0)
tests/health-check/thresholds/x86_64/ibus-dconf.threshold (+73/-0)
tests/health-check/thresholds/x86_64/ibus-engine-simple.threshold (+73/-0)
tests/health-check/thresholds/x86_64/indicator-application-service.threshold (+73/-0)
tests/health-check/thresholds/x86_64/indicator-bluetooth-service.threshold (+73/-0)
tests/health-check/thresholds/x86_64/indicator-datetime-service.threshold (+74/-0)
tests/health-check/thresholds/x86_64/indicator-keyboard-service.threshold (+73/-0)
tests/health-check/thresholds/x86_64/indicator-messages-service.threshold (+73/-0)
tests/health-check/thresholds/x86_64/indicator-power-service.threshold (+73/-0)
tests/health-check/thresholds/x86_64/indicator-printers-service.threshold (+73/-0)
tests/health-check/thresholds/x86_64/indicator-session-service.threshold (+73/-0)
tests/health-check/thresholds/x86_64/indicator-sound-service.threshold (+73/-0)
tests/health-check/thresholds/x86_64/indicator-sync-service.threshold (+73/-0)
tests/health-check/thresholds/x86_64/kerneloops.threshold (+73/-0)
tests/health-check/thresholds/x86_64/modem-manager.threshold (+73/-0)
tests/health-check/thresholds/x86_64/nm-applet.threshold (+73/-0)
tests/health-check/thresholds/x86_64/polkitd.threshold (+73/-0)
tests/health-check/thresholds/x86_64/rtkit-daemon.threshold (+73/-0)
tests/health-check/thresholds/x86_64/systemd-logind.threshold (+73/-0)
tests/health-check/thresholds/x86_64/systemd-udevd.threshold (+73/-0)
tests/health-check/thresholds/x86_64/telepathy-indicator.threshold (+73/-0)
tests/health-check/thresholds/x86_64/udisksd.threshold (+73/-0)
tests/health-check/thresholds/x86_64/unity-panel-service.threshold (+73/-0)
tests/health-check/thresholds/x86_64/update-notifier.threshold (+73/-0)
tests/health-check/thresholds/x86_64/upowerd.threshold (+73/-0)
tests/health-check/thresholds/x86_64/upstart-dbus-bridge.threshold (+73/-0)
tests/health-check/thresholds/x86_64/upstart-event-bridge.threshold (+73/-0)
tests/health-check/thresholds/x86_64/upstart-file-bridge.threshold (+73/-0)
tests/health-check/thresholds/x86_64/upstart-socket-bridge.threshold (+73/-0)
tests/health-check/thresholds/x86_64/upstart-udev-bridge.threshold (+73/-0)
tests/health-check/thresholds/x86_64/whoopsie.threshold (+73/-0)
tests/health-check/thresholds/x86_64/zeitgeist-daemon.threshold (+73/-0)
tests/health-check/thresholds/x86_64/zeitgeist-datahub.threshold (+73/-0)
tests/health-check/thresholds/x86_64/zeitgeist-fts.threshold (+73/-0)
tests/health-check/ts_control (+2/-0)
tests/health-check/tslist.auto (+3/-0)
tests/memevent/master.run (+5/-0)
tests/memevent/setup.sh (+8/-0)
tests/memevent/ts_control (+1/-0)
tests/memevent/tslist.auto (+7/-0)
tests/memevent/ubuntu_test_cases/__init__.py (+6/-0)
tests/memevent/ubuntu_test_cases/memory_usage_measurement/__init__.py (+1/-0)
tests/memevent/ubuntu_test_cases/memory_usage_measurement/apps/__init__.py (+21/-0)
tests/memevent/ubuntu_test_cases/memory_usage_measurement/apps/browser.py (+103/-0)
tests/memevent/ubuntu_test_cases/memory_usage_measurement/apps/camera.py (+58/-0)
tests/memevent/ubuntu_test_cases/memory_usage_measurement/apps/gallery.py (+41/-0)
tests/memevent/ubuntu_test_cases/memory_usage_measurement/apps/media_player.py (+53/-0)
tests/memevent/ubuntu_test_cases/memory_usage_measurement/matchers.py (+37/-0)
tests/memevent/ubuntu_test_cases/memory_usage_measurement/probes.py (+169/-0)
tests/memevent/ubuntu_test_cases/memory_usage_measurement/smem-tabs (+687/-0)
tests/memevent/ubuntu_test_cases/memory_usage_measurement/tests/__init__.py (+1/-0)
tests/memevent/ubuntu_test_cases/memory_usage_measurement/tests/test_memory_usage.py (+79/-0)
tests/sdk/master.run (+5/-0)
tests/sdk/run_test.sh (+20/-0)
tests/sdk/setup.sh (+12/-0)
tests/sdk/ts_control (+1/-0)
tests/sdk/tslist.auto (+3/-0)
tests/security/master.run (+5/-0)
tests/security/run_test.sh (+26/-0)
tests/security/setup.sh (+12/-0)
tests/security/ts_control (+1/-0)
tests/security/tslist.auto (+6/-0)
tests/smem/master.run (+15/-0)
tests/smem/smem/smem-tabs (+687/-0)
tests/smem/smem/smem.sh (+12/-0)
tests/smem/smem/tc_control (+11/-0)
tests/smem/tslist.run (+1/-0)
tests/suspend-blocker/master.run (+5/-0)
tests/suspend-blocker/suspend-blocker/README (+7/-0)
tests/suspend-blocker/suspend-blocker/setup.sh (+10/-0)
tests/suspend-blocker/suspend-blocker/susblock.conf (+5/-0)
tests/suspend-blocker/suspend-blocker/suspend-blocker.sh (+25/-0)
tests/suspend-blocker/suspend-blocker/tc_control (+16/-0)
tests/suspend-blocker/tslist.run (+1/-0)
tests/systemsettle/systemsettle-after/tc_control (+9/-0)
tests/systemsettle/systemsettle-before/tc_control (+9/-0)
tests/systemsettle/systemsettle.sh (+127/-0)
tests/systemsettle/tslist.run (+2/-0)
utils/host/adb-shell (+23/-0)
utils/host/autopilot-list (+23/-0)
utils/host/autopilot-run (+12/-0)
utils/host/prepare-autopilot-test.sh (+7/-0)
utils/host/reboot-and-unlock.sh (+15/-0)
utils/target/autopilot-list (+13/-0)
utils/target/autopilot-run (+6/-0)
utils/target/check-clickhook-rules (+10/-0)
utils/target/prepare-autopilot-test.sh (+16/-0)
Conflict adding file scripts.  Moved existing file to scripts.moved.
To merge this branch: bzr merge lp://staging/~nskaggs/ubuntu-test-cases/increase-ap-timeout
Reviewer Review Type Date Requested Status
Ubuntu Test Case Developers Pending
Review via email: mp+235705@code.staging.launchpad.net

This proposal has been superseded by a proposal from 2014-09-23.

Commit message

This increases the default timeout for autopilot test runs to 'long', which is currently defined as 30 seconds.

Description of the change

This increases the default timeout for autopilot test runs to 'long', which is currently defined as 30 seconds. This is intended to prevent bug 369990, but it will affect all timeouts for autopilot, including wait_select's, which are currently timing out after the default timeout of 10 seconds. This could cause tests to take an additional 20 seconds to fail (when they do).

Test case authors that require or expect the default timeout would also be affected. In general waiting longer shouldn't impose any additional risk beyond taking longer to fail (any assertions made on a specific time should already be defined inside a test). This is somewhat a heavy handed approach towards fixing the bug, and if possible perhaps it could be applied on a individual testsuite basis.

The underlying causes for why applications don't launch within 10 seconds is concerning and worthy of study, but that is outside of the scope of running these tests. I would rather see more reflective results of the application than using it as a proxy to potential app startup performance concerns. As to tests not reflecting reality and hiding long startup issues, I would only point out the apps incur some additional overhead to load with autopilot (including memory), and thus take longer to launch than launching an application without introspection. Thus this is not reflective of a real world user experience from the start.

To post a comment you must log in.
301. By Nicholas Skaggs

fix unintended whitespace changes

Unmerged revisions

301. By Nicholas Skaggs

fix unintended whitespace changes

300. By Nicholas Skaggs

change default AP timeout to long (30 seconds), fix bug 1369990

299. By Paul Larson

remove calendar app

298. By Paul Larson

run health check with sudo

297. By Paul Larson

Add python3-dateutil for calendar tests

296. By Paul Larson

Apply the same workaround to phablet-network that we previously used for phablet-click-test-setup

295. By Paul Larson

match a bit more of the upstart session env variable

294. By Paul Larson

correct for some oddness that seems to only happen when running it from a script

293. By Paul Larson

Don't run phablet-click-test-setup until we're sure it should work

292. By Paul Larson

retry phablet-click-test-setup if the session is not yet available

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
The diff is not available at this time. You can reload the page or download it.

Subscribers

People subscribed via source and target branches