Merge lp://staging/~stylesen/lava-dispatcher/testdef-from-repo into lp://staging/lava-dispatcher

Proposed by Senthil Kumaran S
Status: Merged
Approved by: Michael Hudson-Doyle
Approved revision: 450
Merged at revision: 460
Proposed branch: lp://staging/~stylesen/lava-dispatcher/testdef-from-repo
Merge into: lp://staging/lava-dispatcher
Diff against target: 191 lines (+115/-19)
1 file modified
lava_dispatcher/actions/lava_test_shell.py (+115/-19)
To merge this branch: bzr merge lp://staging/~stylesen/lava-dispatcher/testdef-from-repo
Reviewer Review Type Date Requested Status
Michael Hudson-Doyle (community) Approve
Review via email: mp+134835@code.staging.launchpad.net

Description of the change

This branch implements the following features:

1) Adds ability to pull a bzr/git repo that includes the test def.
2) Adds ability to pull a specific revision of the repo.

To post a comment you must log in.
Revision history for this message
Michael Hudson-Doyle (mwhudson) wrote :
Download full text (9.7 KiB)

Senthil Kumaran S <email address hidden> writes:

> Senthil Kumaran S has proposed merging lp:~stylesen/lava-dispatcher/testdef-from-repo into lp:lava-dispatcher.
>
> Requested reviews:
> Linaro Validation Team (linaro-validation)
>
> For more details, see:
> https://code.launchpad.net/~stylesen/lava-dispatcher/testdef-from-repo/+merge/134835
>
> This branch implements the following features:
>
> 1) Adds ability to pull a bzr/git repo that includes the test def.
> 2) Adds ability to pull a specific revision of the repo.
>
> --
> https://code.launchpad.net/~stylesen/lava-dispatcher/testdef-from-repo/+merge/134835
> You are subscribed to branch lp:lava-dispatcher.

Hi Senthil,

This branch looks really pretty great (I have a test branch based on
this and some other branches that I'm going to be trying to implement
audio capture with it today). Thanks so much for taking Andy's and my
suggestions on board. I have a few suggestions/quibbles, but I think
this will be ready for landing very soon.

> === modified file 'lava_dispatcher/actions/lava_test_shell.py'
> --- lava_dispatcher/actions/lava_test_shell.py 2012-11-08 14:08:30 +0000
> +++ lava_dispatcher/actions/lava_test_shell.py 2012-11-19 06:43:11 +0000
> @@ -22,6 +22,8 @@
>
> import json
> import yaml
> +import glob
> +import time
> import logging
> import os
> import pexpect
> @@ -100,17 +102,20 @@
> parameters_schema = {
> 'type': 'object',
> 'properties': {
> - 'testdef_urls': {'type': 'array', 'items': {'type': 'string'}},
> + 'testdef_urls': {'type': 'array', 'items': {'type': 'string'},
> + 'optional': True},
> + 'testdef_repos': {'type': 'array', 'items': {'type': 'object'},
> + 'optional': True},

I think it's possible to be more specific here isn't it? You could
specify the properties the objects in the testdef_repos array is allowed
to have. But maybe let's not worry about this for now.

> 'timeout': {'type': 'integer', 'optional': True},
> },
> 'additionalProperties': False,
> }
>
> - def run(self, testdef_urls, timeout=-1):
> + def run(self, testdef_urls=None, testdef_repos=None, timeout=-1):
> target = self.client.target_device
> self._assert_target(target)
>
> - self._configure_target(target, testdef_urls)
> + self._configure_target(target, testdef_urls, testdef_repos)
>
> with target.runner() as runner:
> patterns = [
> @@ -128,11 +133,86 @@
>
> self._bundle_results(target)
>
> - def _get_test_definition(self, testdef_url, tmpdir):
> - testdef_file = download_image(testdef_url, self.context, tmpdir)
> - with open(testdef_file, 'r') as f:
> - logging.info('loading test definition')
> - return yaml.load(f)
> + def _get_test_definition_from_url(self, d, ldir, testdef_urls, tmpdir):
> + tdirs = []
> + for url in testdef_urls:
> + testdef_file = download_image(url, self.context, tmpdir)
> + with open(testdef_file, 'r') as f:
> + logging.info(...

Read more...

Revision history for this message
Michael Hudson-Doyle (mwhudson) wrote :

Michael Hudson-Doyle <email address hidden> writes:

>> + return tdirs
>> +
>> + def _get_testdef_git_repo(self, testdef_repo, tmpdir, revision):
>> + cwd = os.getcwd()
>> + gitdir = os.path.join(tmpdir, 'gittestrepo')
>> + try:
>> + status = subprocess.call(['git', 'clone', testdef_repo,
>> + gitdir])

Oh, I forgot a point. You don't seem to examine status here -- perhaps
calling check_call instead is what you want?

Cheers,
mwh

Revision history for this message
Senthil Kumaran S (stylesen) wrote :

Hi Mike,

On Tuesday 20 November 2012 03:06 AM, Michael Hudson-Doyle wrote:
> This branch looks really pretty great (I have a test branch based on
> this and some other branches that I'm going to be trying to implement
> audio capture with it today). Thanks so much for taking Andy's and my
> suggestions on board. I have a few suggestions/quibbles, but I think
> this will be ready for landing very soon.

Thanks for your detailed review. I shall make the necessary changes as
per your review comments and then push the code today.

Thank You.
--
Senthil Kumaran S
http://www.stylesen.org/
http://www.sasenthilkumaran.com/

450. By Senthil Kumaran S

Code change as per review comments from mwhudson. Following are notable ones:

1) Define the parameter schema for job which specifies the test definitions.
2) Refactor _get_test_definition_from_*, which now got merged into
   _get_test_definition
3) Use subprocess.check_call instead of subprocess.call
4) Fix some indentation.

Revision history for this message
Senthil Kumaran S (stylesen) wrote :
Download full text (4.4 KiB)

Hi Michael,

At revision 450 in branch lp:~stylesen/lava-dispatcher/testdef-from-repo
I fixed all the things which you have pointed out.

On Tuesday 20 November 2012 03:06 AM, Michael Hudson-Doyle wrote:
>> === modified file 'lava_dispatcher/actions/lava_test_shell.py'
>> --- lava_dispatcher/actions/lava_test_shell.py 2012-11-08 14:08:30 +0000
>> +++ lava_dispatcher/actions/lava_test_shell.py 2012-11-19 06:43:11 +0000
>> @@ -22,6 +22,8 @@
>>
>> import json
>> import yaml
>> +import glob
>> +import time
>> import logging
>> import os
>> import pexpect
>> @@ -100,17 +102,20 @@
>> parameters_schema = {
>> 'type': 'object',
>> 'properties': {
>> - 'testdef_urls': {'type': 'array', 'items': {'type': 'string'}},
>> + 'testdef_urls': {'type': 'array', 'items': {'type': 'string'},
>> + 'optional': True},
>> + 'testdef_repos': {'type': 'array', 'items': {'type': 'object'},
>> + 'optional': True},
>
> I think it's possible to be more specific here isn't it? You could
> specify the properties the objects in the testdef_repos array is allowed
> to have. But maybe let's not worry about this for now.

Modified the parameter_schema to be more specific.

>> 'timeout': {'type': 'integer', 'optional': True},
>> },
>> 'additionalProperties': False,
>> }
>>
>> - def run(self, testdef_urls, timeout=-1):
>> + def run(self, testdef_urls=None, testdef_repos=None, timeout=-1):
>> target = self.client.target_device
>> self._assert_target(target)
>>
>> - self._configure_target(target, testdef_urls)
>> + self._configure_target(target, testdef_urls, testdef_repos)
>>
>> with target.runner() as runner:
>> patterns = [
>> @@ -128,11 +133,86 @@
>>
>> self._bundle_results(target)
>>
>> - def _get_test_definition(self, testdef_url, tmpdir):
>> - testdef_file = download_image(testdef_url, self.context, tmpdir)
>> - with open(testdef_file, 'r') as f:
>> - logging.info('loading test definition')
>> - return yaml.load(f)
>> + def _get_test_definition_from_url(self, d, ldir, testdef_urls, tmpdir):
>> + tdirs = []
>> + for url in testdef_urls:
>> + testdef_file = download_image(url, self.context, tmpdir)
>> + with open(testdef_file, 'r') as f:
>> + logging.info('loading test definition')
>> + testdef = yaml.load(f)
>> +
>> + # android mount the partition under /system, while ubuntu
>> + # mounts under /, so we have hdir for where it is on the
>> + # host and tdir for how the target will see the path
>> + timestamp = str(time.time())
>> + hdir = '%s/tests/%s_%s' % \
>> + (d, timestamp, testdef['metadata']['name'])
>> + tdir = '%s/tests/%s_%s' % \
>> + (ldir, timestamp, testdef['metadata']['name'])
>> + self._copy_test(hdir, tdir, testdef)
>> + tdirs.append(tdir)
>
> I think n...

Read more...

Revision history for this message
Senthil Kumaran S (stylesen) wrote :

Hi Michael,

On Tuesday 20 November 2012 04:16 AM, Michael Hudson-Doyle wrote:
> Michael Hudson-Doyle <email address hidden> writes:
>
>>> + return tdirs
>>> +
>>> + def _get_testdef_git_repo(self, testdef_repo, tmpdir, revision):
>>> + cwd = os.getcwd()
>>> + gitdir = os.path.join(tmpdir, 'gittestrepo')
>>> + try:
>>> + status = subprocess.call(['git', 'clone', testdef_repo,
>>> + gitdir])
>
> Oh, I forgot a point. You don't seem to examine status here -- perhaps
> calling check_call instead is what you want?

Changed the above to check_call @ revision 450.

Thank You.
--
Senthil Kumaran S
http://www.stylesen.org/
http://www.sasenthilkumaran.com/

Revision history for this message
Michael Hudson-Doyle (mwhudson) wrote :

Nice work, please merge.

review: Approve
Revision history for this message
Senthil Kumaran S (stylesen) wrote :

Hi Michael,

On Wednesday 21 November 2012 02:35 AM, Michael Hudson-Doyle wrote:
> The proposal to merge lp:~stylesen/lava-dispatcher/testdef-from-repo into lp:lava-dispatcher has been updated.
>
> Status: Needs review => Approved

I spent time on merging this branch to trunk today. But due to new
changes in trunk I could not make the merge. My json job files and
testdef.yaml which used to work started giving the following error in
dashboard:

"Unrecognized or missing document format"

I am unsure what should be changed in order to get over this :( Need
your help here to merge my branch to trunk.

Thank You.
--
Senthil Kumaran S
http://www.stylesen.org/
http://www.sasenthilkumaran.com/

Revision history for this message
Senthil Kumaran S (stylesen) wrote :

On Wednesday 21 November 2012 05:19 PM, Senthil Kumaran S wrote:
> On Wednesday 21 November 2012 02:35 AM, Michael Hudson-Doyle wrote:
>> The proposal to merge lp:~stylesen/lava-dispatcher/testdef-from-repo into lp:lava-dispatcher has been updated.
>>
>> Status: Needs review => Approved
>
> I spent time on merging this branch to trunk today. But due to new
> changes in trunk I could not make the merge. My json job files and
> testdef.yaml which used to work started giving the following error in
> dashboard:
>
> "Unrecognized or missing document format"
>
> I am unsure what should be changed in order to get over this :( Need
> your help here to merge my branch to trunk.

Nevermind, I managed to get past this by upgrading my instance to the
latest versions available and my branch seems to work!!! Will merge it
shortly.

Thank You.
--
Senthil Kumaran S
http://www.stylesen.org/
http://www.sasenthilkumaran.com/

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
The diff is not available at this time. You can reload the page or download it.

Subscribers

People subscribed via source and target branches