Merge lp://staging/~andreas-pokorny/mir/map-touchscreen-to-output into lp://staging/mir
- map-touchscreen-to-output
- Merge into development-branch
Status: | Merged |
---|---|
Approved by: | Daniel van Vugt |
Approved revision: | no longer in the source branch. |
Merged at revision: | 4067 |
Proposed branch: | lp://staging/~andreas-pokorny/mir/map-touchscreen-to-output |
Merge into: | lp://staging/mir |
Prerequisite: | lp://staging/~andreas-pokorny/mir/add-touchscreen-settings |
Diff against target: |
1859 lines (+946/-126) 29 files modified
include/client/mir_toolkit/client_types.h (+1/-0) include/client/mir_toolkit/mir_input_device.h (+68/-0) include/core/mir_toolkit/mir_input_device_types.h (+2/-2) include/platform/mir/input/input_sink.h (+34/-1) include/platform/mir/input/touchscreen_settings.h (+5/-0) include/test/mir_test_framework/fake_input_device.h (+9/-0) src/client/mir_input_device_api.cpp (+35/-0) src/client/symbols.map (+6/-0) src/include/server/mir/input/seat.h (+5/-1) src/platforms/evdev/libinput_device.cpp (+49/-4) src/platforms/evdev/libinput_device.h (+3/-0) src/server/graphics/nested/input_platform.cpp (+18/-2) src/server/input/basic_seat.cpp (+132/-12) src/server/input/basic_seat.h (+15/-7) src/server/input/default_configuration.cpp (+1/-1) src/server/input/default_input_device_hub.cpp (+9/-1) src/server/input/default_input_device_hub.h (+2/-1) src/server/input/seat_input_device_tracker.cpp (+8/-4) src/server/input/seat_input_device_tracker.h (+11/-2) src/server/scene/surface_stack.cpp (+1/-5) tests/acceptance-tests/test_client_input.cpp (+207/-33) tests/include/mir/test/doubles/mock_input_seat.h (+3/-1) tests/include/mir/test/doubles/mock_input_sink.h (+1/-0) tests/integration-tests/input/test_single_seat_setup.cpp (+47/-14) tests/mir_test_framework/fake_input_device_impl.cpp (+76/-14) tests/mir_test_framework/fake_input_device_impl.h (+14/-0) tests/unit-tests/input/evdev/test_libinput_device.cpp (+181/-15) tests/unit-tests/input/test_seat_input_device_tracker.cpp (+1/-4) tests/unit-tests/scene/test_surface_stack.cpp (+2/-2) |
To merge this branch: | bzr merge lp://staging/~andreas-pokorny/mir/map-touchscreen-to-output |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Daniel van Vugt | Abstain | ||
Mir CI Bot | continuous-integration | Approve | |
Alan Griffiths | Approve | ||
Review via email: mp+316709@code.staging.launchpad.net |
Commit message
Map Touchscreen to Output: Allow clients to configure a relation between touchscreen and display outputs
When configured the touch contact coordinates will be mapped into the screen space occupied by the output. Previously the first output was used as default and orientation information was ignored. This default behavior is still in place - when not configured the builtin output on android devices will be used. Additionally the connection between input devices and outputs is used to disable input events from deactivated outputs.
Description of the change
This change allows clients to configure a mapping of touchscreen onto actual outputs. Such that the coordinates originating from the configured device will map into the screen space occupied by the output.
This change also bypasses the existing code around InputRegion which removed the output information to early in the stack..
The information about output to input device relation is forwarded to the host server, and handled in stub and evdev platform. Thus the feature needed an ABI/API break in input platform - which is the second inside the yet unreleased 0.27 series - so there is no ABI bump in this MP.
Mir CI Bot (mir-ci-bot) wrote : | # |
Mir CI Bot (mir-ci-bot) wrote : | # |
FAILED: Continuous integration, rev:4010
https:/
Executed test runs:
FAILURE: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
Click here to trigger a rebuild:
https:/
Mir CI Bot (mir-ci-bot) wrote : | # |
FAILED: Continuous integration, rev:4011
https:/
Executed test runs:
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
Click here to trigger a rebuild:
https:/
Mir CI Bot (mir-ci-bot) wrote : | # |
FAILED: Continuous integration, rev:4012
https:/
Executed test runs:
FAILURE: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
Click here to trigger a rebuild:
https:/
Mir CI Bot (mir-ci-bot) wrote : | # |
FAILED: Continuous integration, rev:4013
https:/
Executed test runs:
FAILURE: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
Click here to trigger a rebuild:
https:/
Mir CI Bot (mir-ci-bot) wrote : | # |
FAILED: Continuous integration, rev:4014
https:/
Executed test runs:
FAILURE: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
FAILURE: https:/
Click here to trigger a rebuild:
https:/
Mir CI Bot (mir-ci-bot) wrote : | # |
PASSED: Continuous integration, rev:4015
https:/
Executed test runs:
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
Click here to trigger a rebuild:
https:/
Mir CI Bot (mir-ci-bot) wrote : | # |
FAILED: Continuous integration, rev:4016
https:/
Executed test runs:
FAILURE: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
FAILURE: https:/
SUCCESS: https:/
deb: https:/
FAILURE: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
Click here to trigger a rebuild:
https:/
Mir CI Bot (mir-ci-bot) wrote : | # |
PASSED: Continuous integration, rev:4017
https:/
Executed test runs:
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
Click here to trigger a rebuild:
https:/
Mir CI Bot (mir-ci-bot) wrote : | # |
PASSED: Continuous integration, rev:4018
https:/
Executed test runs:
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
Click here to trigger a rebuild:
https:/
Alan Griffiths (alan-griffiths) wrote : | # |
+void mir_touchscreen
We shouldn't need to type "enum MirTouchscreenM
enum MirTouchscreenM
{
...
};
=>
typedef enum MirTouchscreenM
{
...
} MirTouchscreenM
Alan Griffiths (alan-griffiths) wrote : | # |
1. Is it worthwhile having both mutable and immutable MirTouchscreenC
2. Are we confident this API won't ever change? (If it might change then it ought to be accessed via the extensions mechanism.)
Andreas Pokorny (andreas-pokorny) wrote : | # |
> 1. Is it worthwhile having both mutable and immutable MirTouchscreenC
We have a non const / const API for all configurations. Touchscreen is not different or special in that reguard.
> 2. Are we confident this API won't ever change? (If it might change then it
> ought to be accessed via the extensions mechanism.)
Andreas Pokorny (andreas-pokorny) wrote : | # |
> 2. Are we confident this API won't ever change? (If it might change then it
> ought to be accessed via the extensions mechanism.)
I cannot respond for the group but this proposal contains the minimum necessary information to get touchscreen coordinates into the system.. I do not expect we would need less in the future - maybe more.. but even that is rather unlikely.
Mir CI Bot (mir-ci-bot) wrote : | # |
FAILED: Continuous integration, rev:4019
https:/
Executed test runs:
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
Click here to trigger a rebuild:
https:/
Mir CI Bot (mir-ci-bot) wrote : | # |
PASSED: Continuous integration, rev:4020
https:/
Executed test runs:
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
Click here to trigger a rebuild:
https:/
Alan Griffiths (alan-griffiths) wrote : | # |
Nits:
+enum MirTouchscreenM
...
+void mir_touchscreen
Don't need "enum" now.
Daniel van Vugt (vanvugt) wrote : | # |
I'm slightly concerned about this (but only if it hasn't been tested):
527 + if (output.
528 + {
529 + data.output_size = output.
530 + if (data.active)
531 + output_
532 + }
Can you confirm that the touch coordinates are correct (like in target or fingerpaint) when the current mode is lower resolution than the screen's native mode? In future the opposite might also be true (screen is mirroring a higher res output and is scaled down)...
Daniel van Vugt (vanvugt) wrote : | # |
Don't we want preferred_
preferred_
Daniel van Vugt (vanvugt) wrote : | # |
The current_mode_index is already taken into consideration in the implementation of extents(). So that's your (target) logical rectangle. Your (source) physical touchscreen rectangle should be from preferred_
Andreas Pokorny (andreas-pokorny) wrote : | # |
> Don't we want preferred_
>
> preferred_
> that will more likely match with touchscreen coordinates.
But the intent is not to get the original screen size but the position size and orientation of the output inside the scene. Actually the native resolution does not matter all..
If the touchscreen coordinate values are not in the range defined by the evdev axis for x and y. The system vendor should either fix the kernel driver or the users has to set a calibration matrix via udev. libinput uses the evdev axis ranges and the calibration matrix to return proper device coordinates.
Daniel van Vugt (vanvugt) wrote : | # |
If preferred_
Soon "output.
Andreas Pokorny (andreas-pokorny) wrote : | # |
> If preferred_
> the near future we'll have a logical output size that's different to the
> current mode dimensions (bug 1639226). So the current mode dimensions must be
> ignored. Instead please use "extents().size" to get the correct size of the
> output in the scene.
So you are requiring a change based on an future change that may or may not affect the validity of the modes vector?
But you do understand that I do need an untransformed coordinate range for the x and y axis of the touchscreen? I can use the preferred_
extents().size would be incorrect now and later since it already applies rotation.
I do think about changing the Output to expose a transformation matrix instead of the output orientation and output position. But then still libinput needs an output size.
Mir CI Bot (mir-ci-bot) wrote : | # |
FAILED: Continuous integration, rev:4022
https:/
Executed test runs:
FAILURE: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
Click here to trigger a rebuild:
https:/
Mir CI Bot (mir-ci-bot) wrote : | # |
FAILED: Continuous integration, rev:4023
https:/
Executed test runs:
FAILURE: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
Click here to trigger a rebuild:
https:/
Mir CI Bot (mir-ci-bot) wrote : | # |
FAILED: Continuous integration, rev:4024
https:/
Executed test runs:
FAILURE: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
Click here to trigger a rebuild:
https:/
Mir CI Bot (mir-ci-bot) wrote : | # |
FAILED: Continuous integration, rev:4025
https:/
Executed test runs:
FAILURE: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
Click here to trigger a rebuild:
https:/
Daniel van Vugt (vanvugt) wrote : | # |
> So you are requiring a change based on an future change that may or may not
> affect the validity of the modes vector?
Yes, that "future change" I am told is my #1 priority to get working for Mir 1.0 (bug 1639226).
> But you do understand that I do need an untransformed coordinate range for the
> x and y axis of the touchscreen? I can use the preferred_
> for that,
Yes, that's why I suggested preferred_
> I do think about changing the Output to expose a transformation matrix instead
> of the output orientation and output position. But then still libinput needs
> an output size.
Yes, I agree and have been thinking that Output may soon need to expose a transformation matrix. DisplayBuffer already does (since r4001), but I think it may need to be guided by a transformation matrix of the output in future.
Mir CI Bot (mir-ci-bot) wrote : | # |
FAILED: Continuous integration, rev:4026
https:/
Executed test runs:
FAILURE: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
FAILURE: https:/
Click here to trigger a rebuild:
https:/
Mir CI Bot (mir-ci-bot) wrote : | # |
FAILED: Continuous integration, rev:4027
https:/
Executed test runs:
FAILURE: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
FAILURE: https:/
Click here to trigger a rebuild:
https:/
Mir CI Bot (mir-ci-bot) wrote : | # |
FAILED: Continuous integration, rev:4028
https:/
Executed test runs:
FAILURE: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
FAILURE: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
Click here to trigger a rebuild:
https:/
Mir CI Bot (mir-ci-bot) wrote : | # |
PASSED: Continuous integration, rev:4029
https:/
Executed test runs:
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
Click here to trigger a rebuild:
https:/
Daniel van Vugt (vanvugt) wrote : | # |
Oh, sorry... Are you using current_mode_index because the raw touch coordinates from the kernel/libinput change based on the current display mode? I was assuming raw touch coordinates from the kernel would always just match the native resolution (preferred_
Andreas Pokorny (andreas-pokorny) wrote : | # |
> Oh, sorry... Are you using current_mode_index because the raw touch
> coordinates from the kernel/libinput change based on the current display mode?
> I was assuming raw touch coordinates from the kernel would always just match
> the native resolution (preferred_
No neither is true. The kernel gives touch coordinates in the ranges the device offers. The two ranges sometimes contain information to calculate physical coordinates (as in mm or inch), but on android device that factor is either zero or one. We cannot assume that the ranges given will match the display resolution. (side note: one can affect the values by setting udev properties, but those are meant to make the coordinate axis align with the display axis, those calibration settings cannot account for runtime configuration of the display) Because of that libinput requires a range parameter when reading pixel coordinates from touch screens. Since the touchscreen itself is not rotated when the display is rotated via kms or software rotation I use the number of pixels in unrotated x and y direction (extent() gives me the axis ranges after rotation). By already using the (currently) final scalings of the x and y coordinate axis the code that does rotation and translation to the scene setup of the output is easier to follow.. i.e does not involve further scaling steps..
So with the last change I tried to detangle the display configuration properties from the touchscreen to output mapping. No more orientation enum and position parameter. So with a transformation matrix in place you will be able to modify that as needed.. I.e. make all scalings inside the matrix and make the output sizes such that the coordinates get mapped into unit-square.. or supply an already scaled output size..
Daniel van Vugt (vanvugt) wrote : | # |
OK. If there are any real problems with this approach, we won't notice any time soon. It's unlikely many people will be using touchscreens in the non-native mode or as clones other outputs.
I mean we may hit bugs in this code but we'll really need to release it first to see what those bugs are...
FAILED: Continuous integration, rev:4009 /mir-jenkins. ubuntu. com/job/ mir-ci/ 2954/ /mir-jenkins. ubuntu. com/job/ build-mir/ 3915/console /mir-jenkins. ubuntu. com/job/ build-0- fetch/4001/ console /mir-jenkins. ubuntu. com/job/ build-1- sourcepkg/ release= vivid+overlay/ 3991/console /mir-jenkins. ubuntu. com/job/ build-1- sourcepkg/ release= xenial+ overlay/ 3991/console /mir-jenkins. ubuntu. com/job/ build-1- sourcepkg/ release= zesty/3991/ console /mir-jenkins. ubuntu. com/job/ build-2- binpkg- mir/arch= amd64,compiler= clang,platform= mesa,release= zesty/3942/ console /mir-jenkins. ubuntu. com/job/ build-2- binpkg- mir/arch= amd64,compiler= gcc,platform= mesa,release= xenial+ overlay/ 3942/console /mir-jenkins. ubuntu. com/job/ build-2- binpkg- mir/arch= amd64,compiler= gcc,platform= mesa,release= zesty/3942/ console /mir-jenkins. ubuntu. com/job/ build-2- binpkg- mir/arch= cross-armhf, compiler= gcc,platform= android, release= vivid+overlay/ 3942/console /mir-jenkins. ubuntu. com/job/ build-2- binpkg- mir/arch= i386,compiler= gcc,platform= android, release= vivid+overlay/ 3942/console /mir-jenkins. ubuntu. com/job/ build-2- binpkg- mir/arch= i386,compiler= gcc,platform= mesa,release= xenial+ overlay/ 3942/console
https:/
Executed test runs:
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
Click here to trigger a rebuild: /mir-jenkins. ubuntu. com/job/ mir-ci/ 2954/rebuild
https:/