Merge lp://staging/~andrea.corbellini/a4/test-suite into lp://staging/a4

I couldn't make the tests work:

$ bzr branch lp:~andrea.corbellini/a4/test-suite
Branched 25 revision(s).
$ cd test-suite/
$ make check
python test.py

** (-c:2332): WARNING **: Trying to register gtype 'WnckWindowState' as enum when in fact it is of type 'GFlags'

** (-c:2332): WARNING **: Trying to register gtype 'WnckWindowActions' as enum when in fact it is of type 'GFlags'

** (-c:2332): WARNING **: Trying to register gtype 'WnckWindowMoveResizeMask' as enum when in fact it is of type 'GFlags'
GTK Accessibility Module initialized
GTK Accessibility Module initialized
GTK Accessibility Module initialized
Failed test case: opening_files.TestOpeningFiles.test_open_existent_file
  Traceback (most recent call last):
    File "tests/opening_files.py", line 20, in test_open_existent_file
      dialog.settextvalue('txtLocation', TEST_IMAGE_FILE)
    File "tests/testhelpers.py", line 32, in wrapper
      retval = ldtp_method(self.window_id, *args, **kwargs)
    File "/usr/lib/pymodules/python2.6/ldtp/client.py", line 50, in __call__
      return self.__send(self.__name, args)
    File "/usr/lib/python2.6/xmlrpclib.py", line 1489, in __request
      verbose=self.__verbose
    File "/usr/lib/pymodules/python2.6/ldtp/client.py", line 99, in request
      raise LdtpExecutionError(e.faultString)
  LdtpExecutionError: Unable to find object name "txtLocation" in application map
Failed test case: opening_files.TestOpeningFiles.test_open_incorrect_file
  Traceback (most recent call last):
    File "tests/opening_files.py", line 57, in test_open_incorrect_file
      dialog.settextvalue('txtLocation', os.path.abspath(__file__))
    File "tests/testhelpers.py", line 32, in wrapper
      retval = ldtp_method(self.window_id, *args, **kwargs)
    File "/usr/lib/pymodules/python2.6/ldtp/client.py", line 50, in __call__
      return self.__send(self.__name, args)
    File "/usr/lib/python2.6/xmlrpclib.py", line 1489, in __request
      verbose=self.__verbose
    File "/usr/lib/pymodules/python2.6/ldtp/client.py", line 99, in request
      raise LdtpExecutionError(e.faultString)
  LdtpExecutionError: Unable to find object name "txtLocation" in application map
Failed test case: opening_files.TestOpeningFiles.test_open_inexistent_file
  Traceback (most recent call last):
    File "tests/opening_files.py", line 38, in test_open_inexistent_file
      dialog.settextvalue('txtLocation', 'this-file-does-not-exist')
    File "tests/testhelpers.py", line 32, in wrapper
      retval = ldtp_method(self.window_id, *args, **kwargs)
    File "/usr/lib/pymodules/python2.6/ldtp/client.py", line 50, in __call__
      return self.__send(self.__name, args)
    File "/usr/lib/python2.6/xmlrpclib.py", line 1489, in __request
      verbose=self.__verbose
    File "/usr/lib/pymodules/python2.6/ldtp/client.py", line 99, in request
      raise LdtpExecutionError(e.faultString)
  LdtpExecutionError: Unable to find object name "txtLocation" in application map
Ran 3 tests (3 failed)

Hi,
the roi.svg test image is something I use to develop A4; all the objects are positioned to allow me to see the trajectories during transitions and so on. Probably it's only useful to me, but I'd prefer to have it in the mainline repository so that I can open it any time I run a test version.

Also, you made several changes to the properties of the objects in A4_nested_transforms.svg; maybe they were useful that way (or maybe not, we should ask Gaspa).

Anyway, the handling of test files is a complex matter, I would split the deletion of test_images/ into a separate branch so that we can finalize this one before taking a definitive position on the test images directory.

One more (possibly unrelated) thing: LDTP is a GUI automation framework, which is cool for testing that the GUI remains functional.
I think we should also have traditional unit tests for things like SVG parsing, maths, rendering.
Perhaps we should make separate make targets for unit and GUI tests.

> I couldn't make the tests work...
I suppose that's because you have set Italian as your language. Using English should work, could you confirm, please?

> the roi.svg test image is something I use to develop A4...
Then I will re-add it to the branch. I deleted it because I found it very similar to the other image, sorry.

> Also, you made several changes to the properties of the objects in A4_nested_transforms.svg...
I've just run the 'Delete unused defs' function of Inkscape, so it should be fine.

> Anyway, the handling of test files is a complex matter...
I will put them in tests/images. However I would like to give them more descriptive names, but this can be done in the future.

> One more (possibly unrelated) thing: LDTP is a GUI automation framework...
I generally prefer to write short merge proposals focused on a single feature to make them easier to review. That's why you see just the behaviour-driven tests, but writing tests for the code itself is on my roadmap too (in fact, many things are difficult to test with LDTP).

> I suppose that's because you have set Italian as your language. Using English
> should work, could you confirm, please?

No, my system is set up in English.
I am using Lucid on amd64.

Also, if I have other applications open when I run the tests, they will be closed as if I had clicked on the X button. Don't know if this is useful.

> A4_nested_transforms.svg...
> I've just run the 'Delete unused defs' function of Inkscape, so it should be
> fine.

Yes. After a closer look, I see that the only properties that have changed are in the inkscape namespace, so I don't think we are using them.

> > Anyway, the handling of test files is a complex matter...
> I will put them in tests/images. However I would like to give them more
> descriptive names, but this can be done in the future.

This is okay for now.
In the future we may have to handle three different kinds of test presentations:
- images that will actually be installed on the user's system (in /usr/share/a4/...); ideally these should be beautiful presentations about A4's amazing features, but for now we will have to use the images we have; these are also useful for samples and documentations, see here for example: http://wiki.ubuntu-it.org/GruppoTest/Casi/A4
- images used for development, with a special selection of items, transformations and so on;
- images used by the automated tests.

Same failure on a different machine: Lucid x86, English :-(

I've fixed the test. The problem was that by default the 'Location' textbox of the 'Open file' dialog is hidden on GNOME.

GREAT!
I'm going to to test it ASAP.

Looks like we still need some work aroun it. I tried to run test.py and I got this:

andrea@starfleet:~/A4/a4$ python ./test.py

** (-c:18310): WARNING **: Trying to register gtype 'WnckWindowState' as enum when in fact it is of type 'GFlags'

** (-c:18310): WARNING **: Trying to register gtype 'WnckWindowActions' as enum when in fact it is of type 'GFlags'

** (-c:18310): WARNING **: Trying to register gtype 'WnckWindowMoveResizeMask' as enum when in fact it is of type 'GFlags'
GTK Accessibility Module initialized
GTK Accessibility Module initialized
Traceback (most recent call last):
  File "/home/andrea/A4/a4/a4lib/app.py", line 198, in on_drawing_area_expose
    self.player.render(context)
  File "/home/andrea/A4/a4/a4lib/player.py", line 139, in render
    if not self.presentation:
AttributeError: 'CairoPlayer' object has no attribute 'presentation'
Traceback (most recent call last):
  File "/home/andrea/A4/a4/a4lib/app.py", line 198, in on_drawing_area_expose
    self.player.render(context)
  File "/home/andrea/A4/a4/a4lib/player.py", line 139, in render
    if not self.presentation:
AttributeError: 'CairoPlayer' object has no attribute 'presentation'
GTK Accessibility Module initialized
Traceback (most recent call last):
  File "/home/andrea/A4/a4/a4lib/app.py", line 198, in on_drawing_area_expose
    self.player.render(context)
  File "/home/andrea/A4/a4/a4lib/player.py", line 139, in render
    if not self.presentation:
AttributeError: 'CairoPlayer' object has no attribute 'presentation'
Traceback (most recent call last):
  File "/home/andrea/A4/a4/a4lib/app.py", line 198, in on_drawing_area_expose
    self.player.render(context)
  File "/home/andrea/A4/a4/a4lib/player.py", line 139, in render
    if not self.presentation:
AttributeError: 'CairoPlayer' object has no attribute 'presentation'
Traceback (most recent call last):
  File "/home/andrea/A4/a4/a4lib/app.py", line 198, in on_drawing_area_expose
    self.player.render(context)
  File "/home/andrea/A4/a4/a4lib/player.py", line 139, in render
    if not self.presentation:
AttributeError: 'CairoPlayer' object has no attribute 'presentation'
Ran 3 tests without failures
andrea@starfleet:~/A4/a4$ Traceback (most recent call last):
  File "/home/andrea/A4/a4/a4lib/app.py", line 198, in on_drawing_area_expose
    self.player.render(context)
  File "/home/andrea/A4/a4/a4lib/player.py", line 139, in render
    if not self.presentation:
AttributeError: 'CairoPlayer' object has no attribute 'presentation'

Immediately before showing the Traceback, it tried to close the terminal (terminator prevented it).

The tracebacks you are seeing are bugs in the trunk branch (and that's why I like unit testing). The test suite should be fine.

Sadly, it's still not working. This time it can't find the control called "tbtnTypeafilename". Also, even in the previous version the Location text field was visible, so the problem seems to be that it can't find the controls by name.

This is what happens: an A4 main window opens, then an Open File dialog titled "a4" opens and gets focus, then nothing happens for a few seconds; then it happens again twice (a main window and a dialog for each test). None of these windows gets closed automatically.
It doesn't close random windows anymore.

Is there any way to see which window the "dialog" variable actually captures? or to get the names of the window controls?

Output:

$ make check
python test.py

** (-c:23319): WARNING **: Trying to register gtype 'WnckWindowState' as enum when in fact it is of type 'GFlags'

** (-c:23319): WARNING **: Trying to register gtype 'WnckWindowActions' as enum when in fact it is of type 'GFlags'

** (-c:23319): WARNING **: Trying to register gtype 'WnckWindowMoveResizeMask' as enum when in fact it is of type 'GFlags'
GTK Accessibility Module initialized
GTK Accessibility Module initialized
GTK Accessibility Module initialized
Failed test case: opening_files.TestOpeningFiles.test_open_existent_file
  Traceback (most recent call last):
    File "tests/opening_files.py", line 24, in test_open_existent_file
      dialog.click('tbtnTypeafilename')
    File "tests/testhelpers.py", line 32, in wrapper
      retval = ldtp_method(self.window_id, *args, **kwargs)
    File "/usr/lib/pymodules/python2.6/ldtp/client.py", line 50, in __call__
      return self.__send(self.__name, args)
    File "/usr/lib/python2.6/xmlrpclib.py", line 1489, in __request
      verbose=self.__verbose
    File "/usr/lib/pymodules/python2.6/ldtp/client.py", line 99, in request
      raise LdtpExecutionError(e.faultString)
  LdtpExecutionError: Unable to find object name "tbtnTypeafilename" in application map
Failed test case: opening_files.TestOpeningFiles.test_open_incorrect_file
  Traceback (most recent call last):
    File "tests/opening_files.py", line 69, in test_open_incorrect_file
      dialog.click('tbtnTypeafilename')
    File "tests/testhelpers.py", line 32, in wrapper
      retval = ldtp_method(self.window_id, *args, **kwargs)
    File "/usr/lib/pymodules/python2.6/ldtp/client.py", line 50, in __call__
      return self.__send(self.__name, args)
    File "/usr/lib/python2.6/xmlrpclib.py", line 1489, in __request
      verbose=self.__verbose
    File "/usr/lib/pymodules/python2.6/ldtp/client.py", line 99, in request
      raise LdtpExecutionError(e.faultString)
  LdtpExecutionError: Unable to find object name "tbtnTypeafilename" in application map
Failed test case: opening_files.TestOpeningFiles.test_open_inexistent_file
  Traceback (most recent call last):
    File "tests/opening_files.py", line 46, in test_open_inexistent_file
      dialog.click('tbtnTypeafilename')
    File "tests/testhelpers.py", line 32, in wrapper
      retval = ldtp_method(self.window_id, *args, **kwargs)
    File "/usr/lib/pymodules/python2.6/ldtp/client.py", line 50, in __call__
      return self.__send(self.__name, args)
    File "/usr/lib/python2.6/xmlrpclib.py"...

With the latest revision (32) now everything should work. Instead of opening files using the dialog (that is the cause of the problems), I've given the file name as an argument to A4.

The only exceptions that you should see when running `make check` should be two `AttributeError`s, but these are bugs in the software (that I'd happy to fix with my next branch), not something in the test suite itself.

As usual, feel free to ask if you have questions, doubts or problems.

> Ran 3 tests without failures

It finally works for me too :-)