Mir

Merge lp://staging/~afrantzis/mir/fix-1576760-increase-timeout into lp://staging/mir

Proposed by Alexandros Frantzis
Status: Merged
Approved by: Alexandros Frantzis
Approved revision: no longer in the source branch.
Merged at revision: 3572
Proposed branch: lp://staging/~afrantzis/mir/fix-1576760-increase-timeout
Merge into: lp://staging/mir
Diff against target: 263 lines (+29/-29)
1 file modified
tests/acceptance-tests/test_nested_mir.cpp (+29/-29)
To merge this branch: bzr merge lp://staging/~afrantzis/mir/fix-1576760-increase-timeout
Reviewer Review Type Date Requested Status
Cemil Azizoglu (community) Approve
Mir CI Bot continuous-integration Approve
Review via email: mp+298753@code.staging.launchpad.net

Commit message

tests: Increase timeouts to accommodate valgrind slowness

Description of the change

tests: Increase timeouts to accommodate valgrind slowness

Use the existing 'timeout' variable which was set in the code, but not used until now.

To post a comment you must log in.
Revision history for this message
Mir CI Bot (mir-ci-bot) wrote :

PASSED: Continuous integration, rev:3567
https://mir-jenkins.ubuntu.com/job/mir-ci/1216/
Executed test runs:
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-mir/1397
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-0-fetch/1448
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-1-sourcepkg/release=vivid+overlay/1439
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-1-sourcepkg/release=xenial/1439
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=amd64,compiler=clang,platform=mesa,release=vivid+overlay/1411
        deb: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=amd64,compiler=clang,platform=mesa,release=vivid+overlay/1411/artifact/output/*zip*/output.zip
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=amd64,compiler=gcc,platform=mesa,release=xenial/1411
        deb: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=amd64,compiler=gcc,platform=mesa,release=xenial/1411/artifact/output/*zip*/output.zip
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=cross-armhf,compiler=gcc,platform=android,release=vivid+overlay/1411
        deb: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=cross-armhf,compiler=gcc,platform=android,release=vivid+overlay/1411/artifact/output/*zip*/output.zip
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=i386,compiler=gcc,platform=android,release=vivid+overlay/1411
        deb: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=i386,compiler=gcc,platform=android,release=vivid+overlay/1411/artifact/output/*zip*/output.zip
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=i386,compiler=gcc,platform=mesa,release=xenial/1411
        deb: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=i386,compiler=gcc,platform=mesa,release=xenial/1411/artifact/output/*zip*/output.zip

Click here to trigger a rebuild:
https://mir-jenkins.ubuntu.com/job/mir-ci/1216/rebuild

review: Approve (continuous-integration)
Revision history for this message
Cemil Azizoglu (cemil-azizoglu) wrote :

I suspect these timeouts are overly conservative. If these tests start failing they may take a really long time to run since the timeout value is set to 10 at the moment. We should probably change the timeout to something like 2 or 3, and the (two) instances that require more than that can be hard-coded to have 5 or 10.

review: Needs Fixing
Revision history for this message
Alexandros Frantzis (afrantzis) wrote :

> I suspect these timeouts are overly conservative

valgrind under high load (which we can't control since our CI infrastructure is shared) can be *extremely* slow.

> If these tests start failing they may take a really long time to run since the timeout value
> is set to 10 at the moment

Can't say I am really concerned with this, since it's an extremely rare event and will add in the absolute worst case 5 minutes of total time.

> We should probably change the timeout to something like 2 or 3, and the (two) instances
> that require more than that can be hard-coded to have 5 or 10.

The reason the two instances have 5 and 10 respectively (which are arbitrary values), is because they failed at some point in the past due to valgrind slowness and high CI load, so we had to increase their timeouts.

Now NestedServer.when_monitor_plugged_in_client_is_notified_of_new_display_configuration happened to fail, so we need to increase its timeout too. In the future another test will fail for the same reason.

I tried setting the timeout to 3 seconds, but I got some failures locally under valgrind + sufficient load in various NestedServer.* tests. I couldn't get failures locally with 5 seconds, so I think it's a good start, although only time will tell if it will be enough for CI.

Revision history for this message
Mir CI Bot (mir-ci-bot) wrote :

PASSED: Continuous integration, rev:3568
https://mir-jenkins.ubuntu.com/job/mir-ci/1220/
Executed test runs:
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-mir/1403
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-0-fetch/1454
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-1-sourcepkg/release=vivid+overlay/1445
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-1-sourcepkg/release=xenial/1445
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=amd64,compiler=clang,platform=mesa,release=vivid+overlay/1417
        deb: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=amd64,compiler=clang,platform=mesa,release=vivid+overlay/1417/artifact/output/*zip*/output.zip
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=amd64,compiler=gcc,platform=mesa,release=xenial/1417
        deb: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=amd64,compiler=gcc,platform=mesa,release=xenial/1417/artifact/output/*zip*/output.zip
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=cross-armhf,compiler=gcc,platform=android,release=vivid+overlay/1417
        deb: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=cross-armhf,compiler=gcc,platform=android,release=vivid+overlay/1417/artifact/output/*zip*/output.zip
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=i386,compiler=gcc,platform=android,release=vivid+overlay/1417
        deb: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=i386,compiler=gcc,platform=android,release=vivid+overlay/1417/artifact/output/*zip*/output.zip
    SUCCESS: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=i386,compiler=gcc,platform=mesa,release=xenial/1417
        deb: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=i386,compiler=gcc,platform=mesa,release=xenial/1417/artifact/output/*zip*/output.zip

Click here to trigger a rebuild:
https://mir-jenkins.ubuntu.com/job/mir-ci/1220/rebuild

review: Approve (continuous-integration)
Revision history for this message
Cemil Azizoglu (cemil-azizoglu) wrote :

ok

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
The diff is not available at this time. You can reload the page or download it.

Subscribers

People subscribed via source and target branches