Merge lp://staging/~michihenning/unity-scopes-api/qt-coverage into lp://staging/unity-scopes-api/devel

PASSED: Continuous integration, rev:327
http://jenkins.qa.ubuntu.com/job/unity-scopes-api-ci/581/
Executed test runs:
    SUCCESS: http://jenkins.qa.ubuntu.com/job/unity-scopes-api-vivid-amd64-ci/115
    SUCCESS: http://jenkins.qa.ubuntu.com/job/unity-scopes-api-vivid-armhf-ci/106
        deb: http://jenkins.qa.ubuntu.com/job/unity-scopes-api-vivid-armhf-ci/106/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/unity-scopes-api-vivid-i386-ci/107

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/unity-scopes-api-ci/581/rebuild

FAILED: Continuous integration, rev:328
http://jenkins.qa.ubuntu.com/job/unity-scopes-api-ci/582/
Executed test runs:
    FAILURE: http://jenkins.qa.ubuntu.com/job/unity-scopes-api-vivid-amd64-ci/116/console
    FAILURE: http://jenkins.qa.ubuntu.com/job/unity-scopes-api-vivid-armhf-ci/107/console
    FAILURE: http://jenkins.qa.ubuntu.com/job/unity-scopes-api-vivid-i386-ci/108/console

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/unity-scopes-api-ci/582/rebuild

FAILED: Continuous integration, rev:328
http://jenkins.qa.ubuntu.com/job/unity-scopes-api-ci/583/
Executed test runs:
    SUCCESS: http://jenkins.qa.ubuntu.com/job/unity-scopes-api-vivid-amd64-ci/117
    SUCCESS: http://jenkins.qa.ubuntu.com/job/unity-scopes-api-vivid-armhf-ci/108
        deb: http://jenkins.qa.ubuntu.com/job/unity-scopes-api-vivid-armhf-ci/108/artifact/work/output/*zip*/output.zip
    FAILURE: http://jenkins.qa.ubuntu.com/job/unity-scopes-api-vivid-i386-ci/109/console

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/unity-scopes-api-ci/583/rebuild

PASSED: Continuous integration, rev:328
http://jenkins.qa.ubuntu.com/job/unity-scopes-api-ci/584/
Executed test runs:
    SUCCESS: http://jenkins.qa.ubuntu.com/job/unity-scopes-api-vivid-amd64-ci/118
    SUCCESS: http://jenkins.qa.ubuntu.com/job/unity-scopes-api-vivid-armhf-ci/109
        deb: http://jenkins.qa.ubuntu.com/job/unity-scopes-api-vivid-armhf-ci/109/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/unity-scopes-api-vivid-i386-ci/110

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/unity-scopes-api-ci/584/rebuild

Oh wow, quite a lot of changes and cleanups, thanks for that Michi!

A couple of minor remarks:

1235 +// TODO: These are in the public API, so they need documentation.

Good point; should they actually be part of the public API? I think we're hiding scopes::Variant everywhere and only expose QVariant and do the conversions internally. Would it make sense to hide them instead?

766 + * Copyright (C) 2013 Canonical Ltd
2353 + * Copyright (C) 2014 Canonical Ltd

Bump the year please (there are more of them, just pasting two examples) :)

35 +unity-scopes-api (0.6.15+15.04.20150213-0ubuntu2) UNRELEASED; urgency=medium

The beloved changelog! Should say 0.6.15-0ubuntu1 ;)

I think it would probably make sense to put _ENABLE_QT_EXPERIMENTAL_ in a separate include file, and just #include that everywhere, but feel free to leave it as is; since it's a temporary measure there is no need to put more effort into changing all that again. On second thought, you probably had a good reason to copy-paste it everywhere (better error message from the compiler about offending include file?) ;)

I'm a bit baffled by the amount of changes to symbols file; is it just about re-sorting them?

Regarding: 1235 +// TODO: These are in the public API, so they need documentation.

I agree with Pawel. Maybe we should hide the full header from the public API and move it to the internal directory.

Those functions should not be used by any Qt user as they should only deal with QVariant.

Regarding the dates: most of the files were created in 2015, so not sure if we should use 2015 instead.

Regarding the _ENABLE_QT_EXPERIMENTAL_ I added it explicitly to all the headers to make any potencial user completely aware that he/she must use the flag. I thought that "hiding" the flag into another header would possibly be less clear to the final user, but it's fine with me to use that approach instead.

> 1235 +// TODO: These are in the public API, so they need documentation.
>
> Good point; should they actually be part of the public API? I think we're
> hiding scopes::Variant everywhere and only expose QVariant and do the
> conversions internally. Would it make sense to hide them instead?
>
> 766 + * Copyright (C) 2013 Canonical Ltd
> 2353 + * Copyright (C) 2014 Canonical Ltd
>
> Bump the year please (there are more of them, just pasting two examples) :)

I've gone through and checked them all. I found one that had 2013 instead of 2014 (HttpAsyncReader.h). The others all have 2014. I think that's right, seeing that Xavi was working on this last year.

> 35 +unity-scopes-api (0.6.15+15.04.20150213-0ubuntu2) UNRELEASED;
> urgency=medium
>
> The beloved changelog! Should say 0.6.15-0ubuntu1 ;)

Ah, my good old friend, the changelog... Neither "dch" nor "dch -i" does the right thing. Is there a magic incantation that does?

> > I think it would probably make sense to put _ENABLE_QT_EXPERIMENTAL_ in a
> separate include file, and just #include that everywhere, but feel free to
> leave it as is; since it's a temporary measure there is no need to put more
> effort into changing all that again. On second thought, you probably had a
> good reason to copy-paste it everywhere (better error message from the
> compiler about offending include file?) ;)

Yes, that's one reason. Also, I don't like "artificial" public headers. I've removed the EXPERIMENTAL thingy from all the internal header files though. It's not needed there; we only need it in the public headers.

> I'm a bit baffled by the amount of changes to symbols file; is it just about
> re-sorting them?

Yes. It turns out that the qt lib wasn't processed by the symbol_diff script, so we didn't get automatic diffs if bzr bd complained. I added processing for the qt lib to the script, so that's fixed now. The script sorts the symbol file and puts it through uniq (it has to because, otherwise, we end up with more and more duplicate lines in the file), which is why there is that large diff.

I've moved the QUtils functions into the internal namespace. (That also caused a change in the symbols file.)

PASSED: Continuous integration, rev:330
http://jenkins.qa.ubuntu.com/job/unity-scopes-api-ci/588/
Executed test runs:
    SUCCESS: http://jenkins.qa.ubuntu.com/job/unity-scopes-api-vivid-amd64-ci/122
    SUCCESS: http://jenkins.qa.ubuntu.com/job/unity-scopes-api-vivid-armhf-ci/113
        deb: http://jenkins.qa.ubuntu.com/job/unity-scopes-api-vivid-armhf-ci/113/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/unity-scopes-api-vivid-i386-ci/114

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/unity-scopes-api-ci/588/rebuild

FAILED: Continuous integration, rev:331
http://jenkins.qa.ubuntu.com/job/unity-scopes-api-ci/593/
Executed test runs:
    FAILURE: http://jenkins.qa.ubuntu.com/job/unity-scopes-api-vivid-amd64-ci/127/console
    SUCCESS: http://jenkins.qa.ubuntu.com/job/unity-scopes-api-vivid-armhf-ci/118
        deb: http://jenkins.qa.ubuntu.com/job/unity-scopes-api-vivid-armhf-ci/118/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/unity-scopes-api-vivid-i386-ci/119

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/unity-scopes-api-ci/593/rebuild

Thank you so much, Mr Jenkins.

FAILED: Continuous integration, rev:331
http://jenkins.qa.ubuntu.com/job/unity-scopes-api-ci/596/
Executed test runs:
    FAILURE: http://jenkins.qa.ubuntu.com/job/unity-scopes-api-vivid-amd64-ci/130/console
    FAILURE: http://jenkins.qa.ubuntu.com/job/unity-scopes-api-vivid-armhf-ci/121/console
    SUCCESS: http://jenkins.qa.ubuntu.com/job/unity-scopes-api-vivid-i386-ci/122

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/unity-scopes-api-ci/596/rebuild

PASSED: Continuous integration, rev:331
http://jenkins.qa.ubuntu.com/job/unity-scopes-api-ci/599/
Executed test runs:
    SUCCESS: http://jenkins.qa.ubuntu.com/job/unity-scopes-api-vivid-amd64-ci/133
    SUCCESS: http://jenkins.qa.ubuntu.com/job/unity-scopes-api-vivid-armhf-ci/124
        deb: http://jenkins.qa.ubuntu.com/job/unity-scopes-api-vivid-armhf-ci/124/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/unity-scopes-api-vivid-i386-ci/125

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/unity-scopes-api-ci/599/rebuild

> > The beloved changelog! Should say 0.6.15-0ubuntu1 ;)
>
> Ah, my good old friend, the changelog... Neither "dch" nor "dch -i" does the
> right thing. Is there a magic incantation that does?

No idea; I think it can't deal with the version numbers generated by our autolanding machinery.

>
> I've moved the QUtils functions into the internal namespace. (That also caused
> a change in the symbols file.)
Thanks!

Michi, I've no luck running the updated diff_script, it's failing to do the job in different ways:

4006 + with open('/tmp/symbols.diff', 'a') as outfile:

this change seems to be problematic, I'm getting an error saying /tmp/symbols.diff doesn't exist on start. It proceeds if I create an empty file in tmp. But then it fails in later steps.

The line #94 of the script which prints the exception needs to use .decode(...) on the line, e.g.

"raise Exception('Cannot parse demangled line: ' + line.decode('utf8'))"

otherwise it fails with error about str + bytes concatenation.

After fixing that line, it fails with this:

vivid@vivid:/src/unity-scopes-api/build-area/unity-scopes-api-0.6.15$ ./obj-x86_64-linux-gnu/tools/symbol_diff
Traceback (most recent call last):
  File "./obj-x86_64-linux-gnu/tools/symbol_diff", line 113, in <module>
    run()
  File "./obj-x86_64-linux-gnu/tools/symbol_diff", line 110, in run
    compare_syms(old_file, new_file)
  File "./obj-x86_64-linux-gnu/tools/symbol_diff", line 94, in compare_syms
    raise Exception('Cannot parse demangled line: ' + line.decode('utf8'))
Exception: Cannot parse demangled line: libunity-scopes.so.3 libunity-scopes3 #MINVER#

To clarify - this fails to regenerate symbols for me in my canned-query-data branch, based on this branch.

> Michi, I've no luck running the updated diff_script, it's failing to do the
> job in different ways:
>
> 4006 + with open('/tmp/symbols.diff', 'a') as outfile:
>
> this change seems to be problematic, I'm getting an error saying
> /tmp/symbols.diff doesn't exist on start. It proceeds if I create an empty
> file in tmp. But then it fails in later steps.
>
> The line #94 of the script which prints the exception needs to use
> .decode(...) on the line, e.g.
>
> "raise Exception('Cannot parse demangled line: ' + line.decode('utf8'))"
>
> otherwise it fails with error about str + bytes concatenation.
>
> After fixing that line, it fails with this:
>
> vivid@vivid:/src/unity-scopes-api/build-area/unity-scopes-api-0.6.15$ ./obj-
> x86_64-linux-gnu/tools/symbol_diff
> Traceback (most recent call last):
> File "./obj-x86_64-linux-gnu/tools/symbol_diff", line 113, in <module>
> run()
> File "./obj-x86_64-linux-gnu/tools/symbol_diff", line 110, in run
> compare_syms(old_file, new_file)
> File "./obj-x86_64-linux-gnu/tools/symbol_diff", line 94, in compare_syms
> raise Exception('Cannot parse demangled line: ' + line.decode('utf8'))
> Exception: Cannot parse demangled line: libunity-scopes.so.3 libunity-scopes3
> #MINVER#

Aargh, my apologies! Will fix straight away.

BTW, as an emergency fix, just back out the last commit to the script, and it should work for the libunity-scopes3 symbols file.

OK, should be good now. I've also re-targeted qt-coverage and qt-coverage-2 for devel.

One not-so-nice thing is that, when you build with bzr bd and get a symbol error, it stops after it finds the first problem. This means that, if a symbol change affects *both* libunity-scopes and libunity-scopes-qt, it stops after generating the symbols for libunity-scopes. So, you have to run the script for the first error, commit the symbols file, and the re-start bzr bd, and do the whole thing again to get the diff for the qt symbols.