QA Regression Testing

Merge ~iconstantin/qa-regression-testing:master into qa-regression-testing:master

Proposed by Ian Constantin on 2021-11-29

Status:	Needs review
Proposed branch:	~iconstantin/qa-regression-testing:master
Merge into:	qa-regression-testing:master
Diff against target:	126 lines (+92/-2) 4 files modified scripts/test-python3.10.py (+45/-0) scripts/test-python3.6.py (+1/-1) scripts/test-python3.7.py (+1/-1) scripts/test-python3.9.py (+45/-0)
Related bugs:	Link a bug report

Reviewer	Review Type	Date Requested	Status
Alex Murray		2021-11-29	Approve on 2021-12-16
Review via email: mp+412549@code.staging.launchpad.net

Commit message

Added qa scripts for python 3.9-10 and made small changes to the descriptions test-python3.[6-7].py

Description of the change

This merge request ~~

Creates:

> test-python3.9.py
> test-python3.10.py

Both of these being identical to the already existing scripts that set their respective version of Python and then call test-python.py

Changes:

> test-python3.6.py
> test-python3.7.py

Updating the headers/descriptions of these scripts to reference their respective versions of Python 3 (no code change)

Revision history for this message

Alex Murray (alexmurray) wrote on 2021-12-03:

LGTM although I don't love the copy-pasta to all these test-python3.X.py files - could these instead be symlinks to one primary test-python-x.y.py which sets PYTHON based on the value of argv[0]?

Revision history for this message

Ian Constantin (iconstantin) wrote on 2021-12-06:

Hey Alex,

I like the concept and have been playing around with it. From my understanding, the test-python-x.y.py scripts act as set-up/staging scripts that do two things:

- Specify QRT-Depends, QRT-Packages, and the version of Python being tested:

    # QRT-Depends: test-python.py
    # QRT-Packages: python3.10 netcat-openbsd w3m
    .
    .
    .
    PYTHON='python3.10'

- Runs 'test-python.py' (providing PYTHON as an arg) which is the script containing all the qa unit tests:

subprocess.Popen([ 'python2', 'test-python.py', PYTHON]

When I want to get these over on a VM to test, I use:

$ vm-qrt -p sec-impish-amd64 test-python3.10.py

At one point, vm-qrt runs:

$ grep 'QRT-Packages' ./`basename $test_script`
$ sudo apt-get install -y --force-yes <QRT-Packages>

This looks at the hard coded line in the source of test-python-x.y.py which specifies the Python version (and other package dependencies). This is where the implementation of an argv/single source script fails (for the current process) since the version would not be part of the source code when vm-qrt runs.

Revision history for this message

Alex Murray (alexmurray) wrote on 2021-12-16:

Ah right, ok so we need to keep them separate to allow the different QRT-Depends etc - fair enough.

review: Approve

Revision history for this message

Alex Murray (alexmurray) wrote on 2023-02-07:

Ian are you still working on this?

There was an error fetching revisions from git servers. Please try again in a few minutes. If the problem persists, contact Launchpad support.