Merge ~nacc/git-ubuntu:lp1718019-custom-sourcepackage-classes into git-ubuntu:master

An alternative choice would be to override the parent classes _source_urls methods to just not use mirrors/masters. I think the approach in this MP is safer in case other internal methods use the variables.

FAILED: Continuous integration, rev:b7d822253cdc20b831a7f9840f2cffe7f2077c2a
No commit message was specified in the merge proposal. Click on the following link and set the commit message (if you want a jenkins rebuild you need to trigger it yourself):
https://code.launchpad.net/~nacc/usd-importer/+git/usd-importer/+merge/330629/+edit-commit-message

https://jenkins.ubuntu.com/server/job/git-ubuntu-ci/73/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Style Check
    SUCCESS: Unit Tests
    SUCCESS: Integration Tests
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/git-ubuntu-ci/73/rebuild

This code is doing what was requested in the bug, but I wonder if that's the right change.

The code in python3-ubuntutools seems to do the right thing when we pull a package:

            for url in self._source_urls(name):
                try:
                    if self._download_file(url, name):
                        break

_source_urls() will iterate over mirrors first, and then masters, and finally LP. That seems robust. If one is down, move to the next one, until a good one is found.

What may not be working well enough is this "switch to the next one". Maybe detecting a broken mirror doesn't work. Maybe it takes too much time to move to the next one. Maybe it lacks a way to mark a mirror as broken, so that it is not tried again the next time.

Replacing this mechanism with having it all centered on LP may not be the best fix. For example, there is that one comment in the source that not all debian packages get imported into LP. Perhaps we should instead enhance how these source_urls are tried.

Do we have more information on exactly what was failing? Maybe we should try to reproduce this problem by blacklisting (firewall or proxy) the debian resources, or just some of them, and retry an import and see what happens when that blacklisted url is tried. Does the code move to the next one quickly enough?

I'll mark this MP as needs-info, and add this comment to the bug too.

To be clear, we only use LP publishing information for srcpkgs, which by definitions means they are present on LP (I'm not sure what the mirroring comment refers to).

The point/intent, though, is to minimize our depdendency on external infrastructure. We should be able to import anything LP says exists with LP only.

From discussion: we'd like this to really be about "depend only on Launchpad".