Ubuntu Autopilot Tests

Merge lp://staging/~adam-disc0tech/ubuntu-autopilot-tests/rhythmbox into lp://staging/ubuntu-autopilot-tests

rhythmbox
Merge into trunk

Proposed by Adam Smith on 2014-02-14

Status:

Merged

Approved by:

Dan Chapman  on 2014-03-11

Approved revision:

111

Merged at revision:

Proposed branch:

lp://staging/~adam-disc0tech/ubuntu-autopilot-tests/rhythmbox

Merge into:

lp://staging/ubuntu-autopilot-tests

Diff against target:

1138 lines (+1093/-0)

6 files modified

ubuntu_autopilot_tests/rhythmbox/emulators/gtk.py (+437/-0)
ubuntu_autopilot_tests/rhythmbox/resources/playlists.xml (+34/-0)
ubuntu_autopilot_tests/rhythmbox/resources/rhythmdb.xml (+177/-0)
ubuntu_autopilot_tests/rhythmbox/tests/base.py (+247/-0)
ubuntu_autopilot_tests/rhythmbox/tests/test_import_and_play.py (+107/-0)
ubuntu_autopilot_tests/rhythmbox/tests/test_rbox.py (+91/-0)

To merge this branch:

bzr merge lp://staging/~adam-disc0tech/ubuntu-autopilot-tests/rhythmbox

Related bugs:

Bug #1155457: Test Needed: Rhythmbox

Undecided

In Progress

Link a bug report

Reviewer	Date Requested	Status
Dan Chapman  (community)	2014-02-14	Approve on 2014-03-11
Nicholas Skaggs (community)		Approve on 2014-03-10
Review via email: mp+206494@code.staging.launchpad.net

Description of the change

First release of tests for Rhythmbox, including launch test, launching help, playing internet radio, playing WAV and OGG files, play / pause / skip track buttons

Revision history for this message

Nicholas Skaggs (nskaggs) wrote on 2014-02-14:

I'll be away next week and won't have proper time to give this a good review. However, it looks like you are missing the resources with this commit :-)

review: Needs Fixing

Revision history for this message

Nicholas Skaggs (nskaggs) wrote on 2014-02-14:

Once that's fixed I'll give it a run again. A couple quick comments; we might need to change how you are deriving the home directory, jenkins may not like it. Did the emulator for gtk work out ok?

lp://staging/~adam-disc0tech/ubuntu-autopilot-tests/rhythmbox updated on 2014-02-15

68. By Adam Smith on 2014-02-15: Added resources

Revision history for this message

Adam Smith (adam-disc0tech) wrote on 2014-02-15:

> I'll be away next week and won't have proper time to give this a good review.
> However, it looks like you are missing the resources with this commit :-)

Resources added!

Revision history for this message

Adam Smith (adam-disc0tech) wrote on 2014-02-15:

> Once that's fixed I'll give it a run again. A couple quick comments; we might
> need to change how you are deriving the home directory, jenkins may not like
> it. Did the emulator for gtk work out ok?

Sure, should I use an absolute path running under jenkins?

The emulator - required significant hacking... Mostly because I couldn't rely on pressing HOME to get to the start of a drop down - as that would select my home directory and rhythmbox would immediately start importing everything from there.

Revision history for this message

Dan Chapman  (dpniel) wrote on 2014-02-15:

I did think the combobox emulator would need a better solution. I ripped those emulators out of ubiquity tests which it works for it's purpose there. This seems to be the tricky part with gtk emulators as it ultimately depends on the developers implementation choices. I will give your modifications a go on the ubiquity tests and a few other apps and see if it can be used as a more general purpose solution rather than just wacking 'Home'.

Revision history for this message

Adam Smith (adam-disc0tech) wrote on 2014-02-15:

OK - but note that it is very hacky emulator code... needs revisiting.

A couple of key changes I made were to ignore items with an accessible role of 49 (invisible separators), and to check which item is focussed (unreliable).

I also disabled some of the code that checks the item by name rather than by index. With hindsight, the reasons I disabled it may be invalid now I have implemented the above.

Revision history for this message

Dan Chapman  (dpniel) wrote on 2014-02-16:

This test is looking good, there are a couple of minor errors that are a quick fix.

see http://people.ubuntu.com/~dpniel/docs/rhythmbox.xml

The first is just a case of cleaning up tabs n spaces. As a side note tests ideally should be python3 compliant so I would recommend using that going forward as it will be used when running tests on Jenkins. it's just a case of using the python3-autopilot package and autopilot3 run/list etc

The Second is just a case of fixing imports in general all testsuites are usually run from the ubuntu-autopilot-tests/ubuntu_autopilot_tests/* directory. So 'autopilot3 run rhythmbox' would be the general start of the testsuite

Also to expand on what Nicholas said about deriving the home directory i personally use os.getenv("HOME") to get the current users home.

If you could make these little fixes first then I will give them a good run :-)

Great job Adam

review: Needs Fixing

lp://staging/~adam-disc0tech/ubuntu-autopilot-tests/rhythmbox updated on 2014-02-23

69. By Adam Smith on 2014-02-17

Fixes following review

70. By Adam Smith on 2014-02-17

corrected import statement

71. By Adam Smith on 2014-02-17

Fixed tabulation issues

72. By Adam Smith on 2014-02-17

Added init file

73. By Adam Smith on 2014-02-17

fixed import error

74. By Adam Smith on 2014-02-17

Fixed properly this time :)

75. By Adam Smith on 2014-02-17

Fixed path errors to handle remote directory structure for jenkins

76. By Adam Smith on 2014-02-17

...

77. By Adam Smith on 2014-02-17

Now checks for existence of Rhythmbox config dir and creates if necessary

78. By Adam Smith on 2014-02-17

referenced test_dir correctly

79. By Adam Smith on 2014-02-17

Changed filter() to enumeration as failing with py3

80. By Adam Smith on 2014-02-18

Split out base class in new structure

81. By Adam Smith on 2014-02-18

Added main_window as a property

82. By Adam Smith on 2014-02-18

Adjusted properties

83. By Adam Smith on 2014-02-18

Modified to work with python 3, refactored. Combo box is now not working in local test environment - pushing for Test Runner test

84. By Adam Smith on 2014-02-18

Changed assertion type in emulator

85. By Adam Smith on 2014-02-22

Removed assertion in emulator

86. By Adam Smith on 2014-02-22

Fixed bug where combo box goes out of range if last item is a separator

87. By Adam Smith on 2014-02-22

Additional assertions added

88. By Adam Smith on 2014-02-22

Corrected max_index naming error

89. By Adam Smith on 2014-02-22

Additional debug statement to investigate out of range issue

90. By Adam Smith on 2014-02-22

fix

91. By Adam Smith on 2014-02-22

fix

92. By Adam Smith on 2014-02-23

Refactored GTK emulator. Changed fallback mode to first assume gail box will be returned in display order, if that fails use combo accessible name to fall back.

Changed import tests to use Pictures directory rather than Downloads, to support local testing.

93. By Adam Smith on 2014-02-23

Stable on local

94. By Adam Smith on 2014-02-23

Moved library parser to use xpath directly

95. By Adam Smith on 2014-02-23

Works locally for single import test

96. By Adam Smith on 2014-02-23

hacks

97. By Adam Smith on 2014-02-23

Added additional debug logging around expected track names

98. By Adam Smith on 2014-02-23

Reversed the order of track play to match that on live image

99. By Adam Smith on 2014-02-23

Performance improvement, only check the treeview tracklist once

100. By Adam Smith on 2014-02-23

Tested all positive import scenarios and work locally. Enabled all tests

Revision history for this message

Adam Smith (adam-disc0tech) wrote on 2014-02-25:

Fixed issues - now runs fine in test-runner

lp://staging/~adam-disc0tech/ubuntu-autopilot-tests/rhythmbox updated on 2014-02-25

101. By Adam Smith on 2014-02-25: Fixed loggin convention
Refactored variable scope
Removed hardcoded resource paths

Revision history for this message

Nicholas Skaggs (nskaggs) wrote on 2014-03-04:

First of all, this looks like a really solid base. Lots of cool stuff being explored here; umockdev, emulator tweaks. Kudos to you Adam!

My one comment would be the function names. I would rather we dropped the given, when, and then from the function names and used comments and logger debug messages instead to achieve a similar thing. Going forward, we're attempting to adopt a new model, based upon the page-object model -- all objects are classes. This tends to make the tests more readable in a similar way without having to name utility functions in such a way.

Overall, good work. I support merging this with the naming tweaks I mentioned.

Ohh, one other thing -- this might also be a good candidate to mock the /home env for as well to avoid colliding with user's data.

review: Needs Fixing

Revision history for this message

Dan Chapman  (dpniel) wrote on 2014-03-04:

Nice job Adam :-)

I agree with what Nicholas mentions above but, i think those kind of refactoring changes could be incorporated into you current working branch lp:~adam-disc0tech/ubuntu-autopilot-tests/rhythmbox2.

The only issue I would want resolved before this merges is in combobox.select_item it needs some kind of condition to fail the test if we didn't manage to select it in 3-4 attempts. A couple of times the combobox lost focus and it ended up in a loop trying to select without focus and needed to keyboard interrupt. Also ensure we still have focus between attempts. Relying on keyboard input is not ideal so it does become quite tricky with comboboxes.

Also the select_item looks like it has become a little over complex so i think we could refactor that as well at some point.

All in all though I think it's great, and once the combox selection has changed I happy for this to merge

review: Needs Fixing

Revision history for this message

Adam Smith (adam-disc0tech) wrote on 2014-03-05:

OK thanks for the feedback everyone.

So in this branch I will fix the combox/select item issues. For the second branch mentioned above I'll (1) move to a page object model (2) add additional comments / logging (3) look at mocking the home directory

I also need to remove umockdev from this branch, as I'm actually not using it yet...

lp://staging/~adam-disc0tech/ubuntu-autopilot-tests/rhythmbox updated on 2014-03-05

102. By Adam Smith on 2014-03-05: Removed unnecessary resources
103. By Adam Smith on 2014-03-05: Updated the emulator to do extra checks around whether focus is held, and ensure it does not enter an endless loop
104. By Adam Smith on 2014-03-05: Included minor changes from second branch
105. By Adam Smith on 2014-03-05: Spelling in comment corrected
106. By Adam Smith on 2014-03-05: Change to print tree on failure to select combo box
107. By Adam Smith on 2014-03-05: Added another exception handling situation that generates a print_tree
108. By Adam Smith on 2014-03-05: Change print tree to look at root object
109. By Adam Smith on 2014-03-05: Added some attempts to regain focus where necessary
110. By Adam Smith on 2014-03-05: fixed tab issue
111. By Adam Smith on 2014-03-05: Added exceptions and stack trace logging