Ubuntu Autopilot Tests

Merge lp://staging/~brendan-donegan/ubuntu-autopilot-tests/bug1297295 into lp://staging/ubuntu-autopilot-tests/ubuntu-experience-tests

bug1297295
Merge into ubuntu-experience-tests

Proposed by Brendan Donegan on 2014-05-29

Status:

Needs review

Proposed branch:

lp://staging/~brendan-donegan/ubuntu-autopilot-tests/bug1297295

Merge into:

lp://staging/ubuntu-autopilot-tests/ubuntu-experience-tests

Diff against target:

106 lines (+89/-0)

2 files modified

ubuntu_experience_tests/helpers.py (+14/-0)
ubuntu_experience_tests/tests/messaging_app/test_messaging_app.py (+75/-0)

To merge this branch:

bzr merge lp://staging/~brendan-donegan/ubuntu-autopilot-tests/bug1297295

Undecided

In Progress

Link a bug report

Reviewer	Review Type	Date Requested	Status
PS Jenkins bot	continuous-integration		Needs Fixing on 2014-07-24
Leo Arias (community)		2014-05-29	Needs Fixing on 2014-07-07
Review via email: mp+221342@code.staging.launchpad.net

This proposal supersedes a proposal from 2014-05-29.

Commit message

Add UX test for adding a number to a contact from an SMS message

Revision history for this message

Leo Arias (elopio) wrote on 2014-06-13:

Looks really nice. Just you have a lot of things to move to the messaging and address book apps. From your TODOs, I think you have that on your roadmap.

In the end, this tests should be nicely readable, just calling methods with names that represent user actions, like open_thread, or add_to_contacts

Revision history for this message

Brendan Donegan (brendan-donegan) wrote on 2014-06-16:

Thanks for the good feedback - yes my plan is to avoid any explicit calls to wait_select and so on in the test, so I will need to do some work in messaging-app and address-book-app. I'll ask for another review later when I've done that.

Revision history for this message

Brendan Donegan (brendan-donegan) wrote on 2014-06-16:

Followed up on most of the comments. The updates to messaging-app and address-book-app will take a bit longer.

Revision history for this message

Leo Arias (elopio) wrote on 2014-06-30:

contactsdb/main.cpp doesn't have tests. And this is not the right project for contacts helpers. Have you tried to merge them to the messaging app? Also, I would prefer the test helpers to be python, unless there's a good reason not to. Why did you chose cpp?

Revision history for this message

Leo Arias (elopio) on 2014-06-30:

review: Needs Fixing

Revision history for this message

Leo Arias (elopio) wrote on 2014-06-30:

s/messaging app/address book

Revision history for this message

Leo Arias (elopio) on 2014-06-30:

lp://staging/~brendan-donegan/ubuntu-autopilot-tests/bug1297295 updated on 2014-06-30

35. By Brendan Donegan on 2014-06-30: Removed contactsdb directory since it wasn't used

Revision history for this message

Brendan Donegan (brendan-donegan) wrote on 2014-06-30:

The contactsdb stuff is what I was writing before I realised there was a dbus service available for that. I removed it in my local branch but forgot to push it up. Deleted it now.

lp://staging/~brendan-donegan/ubuntu-autopilot-tests/bug1297295 updated on 2014-06-30

36. By Brendan Donegan on 2014-06-30: Tidy up comments

Revision history for this message

PS Jenkins bot (ps-jenkins) wrote on 2014-07-01:

FAILED: Continuous integration, rev:36
No commit message was specified in the merge proposal. Click on the following link and set the commit message (if you want a jenkins rebuild you need to trigger it yourself):
https://code.launchpad.net/~brendan-donegan/ubuntu-autopilot-tests/bug1297295/+merge/221342/+edit-commit-message

http://jenkins.qa.ubuntu.com/job/ubuntu-autopilot-tests-ubuntu-experience-tests-ci/7/
Executed test runs:
    FAILURE: http://jenkins.qa.ubuntu.com/job/generic-deb-autopilot-utopic-touch/1430/console
    FAILURE: http://jenkins.qa.ubuntu.com/job/ubuntu-autopilot-tests-ubuntu-experience-tests-utopic-amd64-ci/7/console
    FAILURE: http://jenkins.qa.ubuntu.com/job/ubuntu-autopilot-tests-ubuntu-experience-tests-utopic-armhf-ci/7/console
    FAILURE: http://jenkins.qa.ubuntu.com/job/ubuntu-autopilot-tests-ubuntu-experience-tests-utopic-i386-ci/7/console
    FAILURE: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-utopic-armhf/2385/console

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/ubuntu-autopilot-tests-ubuntu-experience-tests-ci/7/rebuild

review: Needs Fixing (continuous-integration)

Revision history for this message

PS Jenkins bot (ps-jenkins) wrote on 2014-07-05:

FAILED: Continuous integration, rev:36
http://jenkins.qa.ubuntu.com/job/ubuntu-autopilot-tests-ubuntu-experience-tests-ci/14/
Executed test runs:
    FAILURE: http://jenkins.qa.ubuntu.com/job/generic-deb-autopilot-utopic-touch/1584/console
    FAILURE: http://jenkins.qa.ubuntu.com/job/ubuntu-autopilot-tests-ubuntu-experience-tests-utopic-amd64-ci/14/console
    FAILURE: http://jenkins.qa.ubuntu.com/job/ubuntu-autopilot-tests-ubuntu-experience-tests-utopic-armhf-ci/14/console
    FAILURE: http://jenkins.qa.ubuntu.com/job/ubuntu-autopilot-tests-ubuntu-experience-tests-utopic-i386-ci/14/console
    FAILURE: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-utopic-armhf/2581/console

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/ubuntu-autopilot-tests-ubuntu-experience-tests-ci/14/rebuild

review: Needs Fixing (continuous-integration)

lp://staging/~brendan-donegan/ubuntu-autopilot-tests/bug1297295 updated on 2014-07-07

37. By Brendan Donegan on 2014-07-03: Added test_mtp_transfer.py
38. By Brendan Donegan on 2014-07-07: Cleaned flake8 errors

Revision history for this message

PS Jenkins bot (ps-jenkins) wrote on 2014-07-07:

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/ubuntu-autopilot-tests-ubuntu-experience-tests-ci/15/rebuild

review: Needs Fixing (continuous-integration)

Revision history for this message

PS Jenkins bot (ps-jenkins) wrote on 2014-07-07:

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/ubuntu-autopilot-tests-ubuntu-experience-tests-ci/16/rebuild

review: Needs Fixing (continuous-integration)

Revision history for this message

Leo Arias (elopio) wrote on 2014-07-07:

You need to add messaging-app-autopilot to the dependencies. But that's py2, so we'll have to wait for the py3 branch to land.

review: Needs Fixing

Revision history for this message

Leo Arias (elopio) wrote on 2014-07-09:

I'm trying to understand why you are using the MainView as base class. I haven't found anything useful yet, but a couple of things:

86 +class MessagingAddressBookTestCase(messaging_tests.BaseMessagingTestCase):

Some of these tests will need to interact with unity, so we need to restart unity with testability. Currently, that's done telling jenkins to start without a shell. So on your test you will need to restart unity even if you are not interacting with unity. It might be a good idea to kill unity only if you need it, but I think it's good to start with a clean unity. What do you think?

Anyway, in order to run with the current jenkins job, you'll need to inherit from UbuntuExperienceTestCase.

In addition to that, it's probably a bad idea to inherit from that BaseMessagingTestCase. It's on the wrong file, because we should split that test_messaging.py in smaller suites that test the different parts of the app. I think that the right thing to do is to use Fixtures to set up the app environment you need instead of inherit from an app test case that's likely to change. Take a look at what omer is doing here:
https://code.launchpad.net/~om26er/messaging-app/use_fixture/+merge/225350

Revision history for this message

Leo Arias (elopio) wrote on 2014-07-09:

ahhh, I get it. Wow, it took me a long time.

108 + address_book = helpers.get_proxy_object_by_process_name(
109 + 'address-book-app'
110 + )

There what you are getting is an autopilot proxy object, that's like one level higher than the root of the QML tree. So with that proxy object you have access to the full tree.
So, what you should do is something like this

import address_book_app

[...]

address_book_proxy_object = helpers.get_proxy_object_by_process_name(...)
address_book = address_book_app.AddressBookApp(address_book_proxy_object)
address_book.main_view.contact_view_page.save()

So, please remove the _get_emulator_base method, and just hardcode the toolkit base class there.

lp://staging/~brendan-donegan/ubuntu-autopilot-tests/bug1297295 updated on 2014-07-09

39. By Brendan Donegan on 2014-07-09: Merged from trunk
40. By Brendan Donegan on 2014-07-09: Use toolkit base class to get proxy object, and fix the expected result of the test.

Revision history for this message

PS Jenkins bot (ps-jenkins) wrote on 2014-07-09:

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/ubuntu-autopilot-tests-ubuntu-experience-tests-ci/19/rebuild

review: Needs Fixing (continuous-integration)

lp://staging/~brendan-donegan/ubuntu-autopilot-tests/bug1297295 updated on 2014-07-21

41. By Brendan Donegan on 2014-07-09: Use TestabilityEnvironment from this project
42. By Brendan Donegan on 2014-07-16: Use messaging fixture and UbuntuExperienceTestCase
43. By Brendan Donegan on 2014-07-16: Remove accidently added mtp test code

Revision history for this message

PS Jenkins bot (ps-jenkins) wrote on 2014-07-21:

FAILED: Continuous integration, rev:43
http://jenkins.qa.ubuntu.com/job/ubuntu-autopilot-tests-ubuntu-experience-tests-ci/26/
Executed test runs:
    FAILURE: http://jenkins.qa.ubuntu.com/job/generic-deb-autopilot-utopic-touch/2231/console
    FAILURE: http://jenkins.qa.ubuntu.com/job/ubuntu-autopilot-tests-ubuntu-experience-tests-utopic-amd64-ci/26/console
    FAILURE: http://jenkins.qa.ubuntu.com/job/ubuntu-autopilot-tests-ubuntu-experience-tests-utopic-armhf-ci/26/console
    FAILURE: http://jenkins.qa.ubuntu.com/job/ubuntu-autopilot-tests-ubuntu-experience-tests-utopic-i386-ci/26/console
    FAILURE: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-utopic-armhf/3413/console

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/ubuntu-autopilot-tests-ubuntu-experience-tests-ci/26/rebuild

review: Needs Fixing (continuous-integration)

lp://staging/~brendan-donegan/ubuntu-autopilot-tests/bug1297295 updated on 2014-07-24

44. By Brendan Donegan on 2014-07-24: Update to use new helpers which will be in address-book-app and fix imports

Revision history for this message

PS Jenkins bot (ps-jenkins) wrote on 2014-07-24:

FAILED: Continuous integration, rev:44
http://jenkins.qa.ubuntu.com/job/ubuntu-autopilot-tests-ubuntu-experience-tests-ci/29/
Executed test runs:
    FAILURE: http://jenkins.qa.ubuntu.com/job/generic-deb-autopilot-utopic-touch/2426/console
    FAILURE: http://jenkins.qa.ubuntu.com/job/ubuntu-autopilot-tests-ubuntu-experience-tests-utopic-amd64-ci/29/console
    FAILURE: http://jenkins.qa.ubuntu.com/job/ubuntu-autopilot-tests-ubuntu-experience-tests-utopic-armhf-ci/29/console
    FAILURE: http://jenkins.qa.ubuntu.com/job/ubuntu-autopilot-tests-ubuntu-experience-tests-utopic-i386-ci/29/console
    FAILURE: http://jenkins.qa.ubuntu.com/job/generic-mediumtests-builder-utopic-armhf/3658/console

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/ubuntu-autopilot-tests-ubuntu-experience-tests-ci/29/rebuild

review: Needs Fixing (continuous-integration)

Unmerged revisions

44. By Brendan Donegan on 2014-07-24: Update to use new helpers which will be in address-book-app and fix imports
43. By Brendan Donegan on 2014-07-16: Remove accidently added mtp test code
42. By Brendan Donegan on 2014-07-16: Use messaging fixture and UbuntuExperienceTestCase
41. By Brendan Donegan on 2014-07-09: Use TestabilityEnvironment from this project
40. By Brendan Donegan on 2014-07-09: Use toolkit base class to get proxy object, and fix the expected result of the test.
39. By Brendan Donegan on 2014-07-09: Merged from trunk
38. By Brendan Donegan on 2014-07-07: Cleaned flake8 errors
37. By Brendan Donegan on 2014-07-03: Added test_mtp_transfer.py
36. By Brendan Donegan on 2014-06-30: Tidy up comments
35. By Brendan Donegan on 2014-06-30: Removed contactsdb directory since it wasn't used