Merge lp://staging/~qzhang/lava-test/add-libjpeg-turbo into lp://staging/lava-test/0.0

I think you can reference page. https://wiki.linaro.org/ChaoYang/Sandbox/tjbench
and we can get nightshot_iso_100.ppm from here http://www.imagecompression.info/test_images/rgb8bit.zip.

The zip package is 331M, and the file nightshot_iso_100.ppm is 22M, both of them are a little bigger,
Can we put the nightshot_iso_100.ppm file somewhere on our linaro web site.

lava-android-test can also use that file then.

Hi Spring, for the libjpeg-turbo-progs, you need the ~linaro-maintainers/overlay ppa. I'm not sure if there's an x86 version there

38 +INSTALLSTEPS = ['apt-get install libjpeg-turbo-progs',\
The \ at the end of the line is unnecessary, also you should include this in the DEPS, not as an install step. It will fail right now because you don't force it to answer yes.

Need to add wget to the deps also

44 +RUNSTEPS = ['echo "test_case_id nightshot_iso_100_jpg_95_quiet_rgb_scale_1/2"',\
45 + 'tjbench nightshot_iso_100.jpg 95 -quiet -rgb scale 1/2',\
46 + 'echo "test_case_id nightshot_iso_100_ppm_95_quiet_rgb_scale_1/2"',\
47 + 'tjbench nightshot_iso_100_full.ppm 95 -rgb -quiet scale 1/2',\
48 + 'echo "test_case_id nightshot_iso_100_ppm_95_quiet_rgb"',\
49 + 'tjbench nightshot_iso_100_full.ppm 95 -rgb -quiet'\
50 + ]
51 +PATTERN = "^(?P<test_case_id>\w+):\W+(?P<measurement>\d+\.\d+)"
This won't work, you are not producing a single line that can be matched

Also, some of these seem to have multiple results you should look at:
LAVA: Running shell command: 'tjbench nightshot_iso_100_full.ppm 95 -rgb -quiet scale 1/2'
LAVA: (stdout)
LAVA: (stdout) All performance values in Mpixels/sec
LAVA: (stdout)
LAVA: (stdout) Bitmap Bitmap JPEG JPEG Image Image Comp Comp Decomp
LAVA: (stdout) Format Order Subsamp Qual Width Height Perf Ratio Perf
LAVA: (stdout)
LAVA: (stdout) RGB TD GRAY 95 700 525 19.14 15.49 34.13
LAVA: (stdout)
LAVA: (stdout) RGB TD 4:2:0 95 700 525 17.75 13.39 20.93
LAVA: (stdout)
LAVA: (stdout) RGB TD 4:2:2 95 700 525 14.45 12.11 18.58
LAVA: (stdout)
LAVA: (stdout) RGB TD 4:4:4 95 700 525 12.08 10.39 18.93
Which decomp perf are you trying to save there? Are the other numbers not important?

What I suggested before, is to make a test suite around this tool. tjbench is just a tool, the test suite is going to define the test cases around it, and ensure that we output things in a way that is predictable so that we can parse it. It doesn't *have* to be a single line, but that makes things easier. If it's not, then we just need to extend the parser. That's easy though, and lots of other test wrappers do it so there are plenty of examples if that's needed. First, we need the actual test suite script though, not a list of a bunch of run steps that call the tool separately.

yes, lost lots of details, will try on the board

> The zip package is 331M, and the file nightshot_iso_100.ppm is 22M, both of
> them are a little bigger,
> Can we put the nightshot_iso_100.ppm file somewhere on our linaro web site.
I put it here: https://wiki.linaro.org/TestDataLinkPage?action=AttachFile&do=get&target=nightshot_iso_100.ppm
We can use the above link.

What's the current status of this?

Sorry it's pending, I think I need more time to investigate on it.

The test_results in bundle now is:

     "test_results": [
        {
          "log_lineno": 7,
          "result": "pass",
          "measurement": 20.02,
          "units": "Mpixels/s",
          "log_filename": "testoutput.log",
          "test_case_id": "GRAY"
        },
        {
          "log_lineno": 9,
          "result": "pass",
          "measurement": 19.22,
          "units": "Mpixels/s",
          "log_filename": "testoutput.log",
          "test_case_id": "420"
        },
        {
          "log_lineno": 11,
          "result": "pass",
          "measurement": 16.42,
          "units": "Mpixels/s",
          "log_filename": "testoutput.log",
          "test_case_id": "422"
        },
        {
          "log_lineno": 13,
          "result": "pass",
          "measurement": 11.60,
          "units": "Mpixels/s",
          "log_filename": "testoutput.log",
          "test_case_id": "444"
        }

The test_results in bundle now is:

      "test_results": [
        {
          "units": "Mpixels/s",
          "log_filename": "testoutput.log",
          "test_case_id": "RGB_TD_GRAY_95-comp_perf",
          "result": "pass",
          "measurement": 20.11
        },
        {
          "units": "Mpixels/s",
          "log_filename": "testoutput.log",
          "test_case_id": "RGB_TD_GRAY_95-comp_ratio",
          "result": "pass",
          "measurement": 18.87
        },
        {
          "units": "Mpixels/s",
          "log_filename": "testoutput.log",
          "test_case_id": "RGB_TD_GRAY_95-dcomp_perf",
          "result": "pass",
          "measurement": 40.02
        },
        {
          "units": "Mpixels/s",
          "log_filename": "testoutput.log",
          "test_case_id": "RGB_TD_420_95-comp_perf",
          "result": "pass",
          "measurement": 19.31
        },
        {
          "units": "Mpixels/s",
          "log_filename": "testoutput.log",
          "test_case_id": "RGB_TD_420_95-comp_ratio",
          "result": "pass",
          "measurement": 15.53
        },
        {
          "units": "Mpixels/s",
          "log_filename": "testoutput.log",
          "test_case_id": "RGB_TD_420_95-dcomp_perf",
          "result": "pass",
          "measurement": 23.53
        },
        {
          "units": "Mpixels/s",
          "log_filename": "testoutput.log",
          "test_case_id": "RGB_TD_422_95-comp_perf",
          "result": "pass",
          "measurement": 16.44
        },
        {
          "units": "Mpixels/s",
          "log_filename": "testoutput.log",
          "test_case_id": "RGB_TD_422_95-comp_ratio",
          "result": "pass",
          "measurement": 13.48
        },
        {
          "units": "Mpixels/s",
          "log_filename": "testoutput.log",
          "test_case_id": "RGB_TD_422_95-dcomp_perf",
          "result": "pass",
          "measurement": 20.59
        },
        {
          "units": "Mpixels/s",
          "log_filename": "testoutput.log",
          "test_case_id": "RGB_TD_444_95-comp_perf",
          "result": "pass",
          "measurement": 11.45
        },
        {
          "units": "Mpixels/s",
          "log_filename": "testoutput.log",
          "test_case_id": "RGB_TD_444_95-comp_ratio",
          "result": "pass",
          "measurement": 10.84
        },
        {
          "units": "Mpixels/s",
          "log_filename": "testoutput.log",
          "test_case_id": "RGB_TD_444_95-dcomp_perf",
          "result": "pass",
          "measurement": 21.98
        },
        {
          "units": "Mpixels/s",
          "log_filename": "testoutput.log",
          "test_case_id": "RGB_TD_GRAY_95-comp_perf",
          "result": "pass",
          "measurement": 20.14
        },
        {
          "units": "Mpixels/s",
          "log_filename": "testoutput.log",
          "test_case_id": "RGB_TD_GRAY_95-comp_ratio",
          "result": "pass",
          "measurement": 18.87
        },
        {
          "units": "Mpixels/s",
          "log_filename": "testoutput.log",
          "test_case_id": "RGB_TD_GRAY_95-dcomp_perf",
          "result": "pass",
          "measurement": 39.62
...

Sorry this branch is delayed heavily, I make a workable one now.

Some comments from me:
1. The URL of test jpeg file is not defined, is there a server to host them? Please advise.
2. There are two commands here, please see if it's enough:
45 + 'tjbench /mnt/mytmpfs/nightshot_iso_100.ppm 95 -rgb -quiet scale 1/2',
46 + 'tjbench /mnt/mytmpfs/nightshot_iso_100.ppm 95 -rgb -quiet',

From one of my previous comments:
What I suggested before, is to make a test suite around this tool. tjbench is just a tool, the test suite is going to define the test cases around it, and ensure that we output things in a way that is predictable so that we can parse it. It doesn't *have* to be a single line, but that makes things easier. If it's not, then we just need to extend the parser. That's easy though, and lots of other test wrappers do it so there are plenty of examples if that's needed. First, we need the actual test suite script though, not a list of a bunch of run steps that call the tool separately.

> Some comments from me:
> 1. The URL of test jpeg file is not defined, is there a server to host them?
> Please advise.
Well, one option is to stick it on some server like people.linaro.org - I would prefer that to the wiki. Another is to do a script, or set of scripts like I mentioned above, put it in bzr or git, and check out the source branch (including data files such as this) as part of the installation of the test.

> 2. There are two commands here, please see if it's enough:
> 45 + 'tjbench /mnt/mytmpfs/nightshot_iso_100.ppm 95 -rgb -quiet scale
> 1/2',
> 46 + 'tjbench /mnt/mytmpfs/nightshot_iso_100.ppm 95 -rgb -quiet',
That's a question for Rony I think, please ping him for that.

use a wiki location as input stream storage, it can work, I proposed to merge it now and leave the remaining issues(like is test cmd enough) to the future.

I'm really not comfortable with this test in it's current form. The recommendation I gave before was simply to:
1. create a git tree, bzr branch, tarball, whatever you want containing the test data and a script to run the commands you want for the test
2. stick it somewhere we can get to it
3. have the test wrapper here point at that, and run it

I don't see that as being a difficult thing, and it's something I think the mmwg should want to do anyway as it turns this into a more maintainable test that you can grow and expand on. It also solves all the problems we have here with things like where to host the data.

1. tjbench test script is uploaded to lp:~qzhang/libjpeg-turbo/tjbench
2. tested on local lava-test, local result bundle is ok, but when pushed to dashboard, the unit of comp_ratio is always not % but Mpixel/s, don't know why, may be my local cache issue?

1. Aligned with Yongqin's android tjbench, add scale_half test case id prefix.
2. Change the stream file link to people.linaro.org

For no more comment, I'll merge it