Xpra: Ticket #1611: uinput virtual pointer device

split from #173 with a more limited scope: just implement the pointer events using uinput instead of xtest.

This can help us with:



Mon, 31 Jul 2017 13:34:45 GMT - Antoine Martin: status changed

Based on Bug 92772 - RFE: extra option to define the vdist/hdist for scrolling event, we might be able to generate "smooth-scrolling" / small scrolling deltas by creating our uinput device with a very small MOUSE_WHEEL_CLICK_ANGLE (or better, use a high MOUSE_WHEEL_CLICK_COUNT) then synthesizing events using the correct multiplier.

This should be doable using libinput: Static device configuration via udev. Similar to Change scroll speed with libinput.

This looks also interesting for touch devices: Create a virtual multitouch device using uinput driver.

Problem is that no matter what I specify for the uinput device or udev rules, the properties are not set. (and not even the built-in rules are? system bug?)


Mon, 31 Jul 2017 16:22:22 GMT - Antoine Martin:

Code added in r16572.

Still TODO:


Mon, 31 Jul 2017 16:38:36 GMT - Antoine Martin:

udev meanderings, trying to use hwdb to force the click angle / count:

cat > /lib/udev/hwdb.d/71-mouse-xpra.hwdb <<EOF
# allow xpra to use smooth scrolling
mouse:usb:*:name:Xpra Virtual Pointer:
 MOUSE_WHEEL_CLICK_ANGLE=1
 MOUSE_WHEEL_CLICK_COUNT=360
mouse:*:name:Xpra Virtual Pointer:
 MOUSE_WHEEL_CLICK_ANGLE=1
 MOUSE_WHEEL_CLICK_COUNT=360
mouse:usb:v0088p0088:*
 MOUSE_WHEEL_CLICK_ANGLE=1
 MOUSE_WHEEL_CLICK_COUNT=360
EOF
udevadm hwdb --update

Then starting a server:

sudo xpra start-desktop --uid=1000 --gid=1000 :100 --no-daemon \
    --start=xterm --bind-tcp=0.0.0.0:10000 -d mouse --input-devices=uinput
(..)
pointer device emulation using UInput device /dev/input/event20
(..)

Shows no new udev attributes:

$ sudo udevadm info -a --name /dev/input/event20
  looking at device '/devices/virtual/input/input44/event20':
    KERNEL=="event20"
    SUBSYSTEM=="input"
    DRIVER==""
  looking at parent device '/devices/virtual/input/input44':
    KERNELS=="input44"
    SUBSYSTEMS=="input"
    DRIVERS==""
    ATTRS{name}=="Xpra Virtual Pointer"
    ATTRS{phys}==""
    ATTRS{properties}=="0"
    ATTRS{uniq}==""

Trying with a plain udev rule:

cat > /lib/udev/rules.d/71-xpra-mouse.rules << EOF
ACTION=="add|change", SUBSYSTEM=="input", ATTRS{name}=="Xpra Virtual Pointer", ENV{MOUSE_WHEEL_CLICK_ANGLE}="1"
EOF
udevadm control --reload-rules && udevadm trigger

Same result.

$ libinput list-devices
(...)
Device:           Xpra Virtual Pointer
Kernel:           /dev/input/event20
Group:            10
Seat:             seat0, default
Capabilities:     pointer
Tap-to-click:     n/a
Tap-and-drag:     n/a
Tap drag lock:    n/a
Left-handed:      disabled
Nat.scrolling:    disabled
Middle emulation: disabled
Calibration:      n/a
Scroll methods:   button
Click methods:    none
Disable-w-typing: n/a
Accel profiles:   flat *adaptive
Rotation:         n/a

Did the flat acceleration option not work? (could cause problems when we use relative motion to go where we want)

Or maybe it does do something (the plain udev rule), but not on the correct device? (/devices/virtual/input/input46 but not on /devices/virtual/input/input46/event20)

$ udevadm monitor -ukp
KERNEL[21719.101574] add      /devices/virtual/input/input46 (input)
ACTION=add
DEVPATH=/devices/virtual/input/input46
EV=7
KEY=7f0000 0 0 0 0
MODALIAS=input:b0003v0000p0000e0000-e0,1,2,k110,111,112,113,114,115,116,r0,1,8,amlsfw
NAME="Xpra Virtual Pointer"
PRODUCT=3/0/0/0
PROP=0
REL=103
SEQNUM=11953
SUBSYSTEM=input
KERNEL[21719.101695] add      /devices/virtual/input/input46/mouse2 (input)
ACTION=add
DEVNAME=/dev/input/mouse2
DEVPATH=/devices/virtual/input/input46/mouse2
MAJOR=13
MINOR=34
SEQNUM=11954
SUBSYSTEM=input
KERNEL[21719.101768] add      /devices/virtual/input/input46/event20 (input)
ACTION=add
DEVNAME=/dev/input/event20
DEVPATH=/devices/virtual/input/input46/event20
MAJOR=13
MINOR=84
SEQNUM=11955
SUBSYSTEM=input
UDEV  [21719.102443] add      /devices/virtual/input/input46 (input)
.INPUT_CLASS=mouse
ACTION=add
DEVPATH=/devices/virtual/input/input46
EV=7
ID_INPUT=1
ID_INPUT_MOUSE=1
ID_SERIAL=noserial
KEY=7f0000 0 0 0 0
MODALIAS=input:b0003v0000p0000e0000-e0,1,2,k110,111,112,113,114,115,116,r0,1,8,amlsfw
MOUSE_WHEEL_CLICK_ANGLE=1
NAME="Xpra Virtual Pointer"
PRODUCT=3/0/0/0
PROP=0
REL=103
SEQNUM=11953
SUBSYSTEM=input
TAGS=:seat:
USEC_INITIALIZED=21719102303
UDEV  [21719.103088] add      /devices/virtual/input/input46/mouse2 (input)
.INPUT_CLASS=mouse
ACTION=add
DEVNAME=/dev/input/mouse2
DEVPATH=/devices/virtual/input/input46/mouse2
ID_INPUT=1
ID_INPUT_MOUSE=1
ID_SERIAL=noserial
MAJOR=13
MINOR=34
SEQNUM=11954
SUBSYSTEM=input
USEC_INITIALIZED=21719102990
UDEV  [21719.129999] add      /devices/virtual/input/input46/event20 (input)
.INPUT_CLASS=mouse
ACTION=add
DEVNAME=/dev/input/event20
DEVPATH=/devices/virtual/input/input46/event20
ID_INPUT=1
ID_INPUT_MOUSE=1
ID_SERIAL=noserial
MAJOR=13
MINOR=84
SEQNUM=11955
SUBSYSTEM=input
USEC_INITIALIZED=21719129875

Mon, 31 Jul 2017 17:16:05 GMT - Antoine Martin:

Actually, the udev variant does do something - the scrolling is slower than usual with these values:

ACTION=="add|change", ATTRS{name}=="Xpra Virtual Pointer", \
ENV{MOUSE_WHEEL_CLICK_ANGLE}="1", ENV{MOUSE_WHEEL_CLICK_COUNT}="360"

Success!


Thu, 03 Aug 2017 17:23:38 GMT - Antoine Martin:

New issue: Xorg version 1.17.x as found in centos7 doesn't support the xorg.conf options we need (caught by the rpmbuild unit tests):

Parse error on line 17 of section InputClass in file /usr/src/rpmbuild/BUILD/xpra-2.2-python2/unittests/../etc/xpra/xorg.conf
        "NoMatchProduct" is not a valid keyword in this section.

Does it support MatchProduct? Can we get away with not specifying NoMatchProduct and have Ignore=True for all devices, then override for the devices we do want to use?

If not, we'll need a different xorg.conf for centos7, and disable uinput. (we could also rely on the selftests in case someone enables uinput on centos7) Also looks like centos 7.4 will support this: https://bugzilla.redhat.com/show_bug.cgi?id=1401641 (updated to xorg 1.19.x)


Mon, 07 Aug 2017 10:20:12 GMT - Antoine Martin:

Updates:

With all this in place, we have the ability to send fine grained scrolling events. This is what libinput sees (event3 is the real pointer, event19 our uinput virtual one - which is a high precision one and uses higher values):

-event3   POINTER_MOTION   +14.46s	  1.08/  0.00
 event3   POINTER_MOTION   +14.46s	  0.00/  1.08
-event19  POINTER_MOTION   +14.46s	  0.00/  0.82
-event3   POINTER_MOTION   +14.46s	  0.00/  1.07
-event19  POINTER_MOTION   +14.46s	  0.83/  0.83
 event19  POINTER_MOTION   +14.47s	  0.00/  0.85
-event3   POINTER_MOTION   +14.47s	  1.04/  0.00
-event19  POINTER_MOTION   +14.47s	  0.86/  0.86
 event19  POINTER_MOTION   +14.47s	  0.00/  0.87
 event19  POINTER_MOTION   +14.48s	  0.87/  0.00
-event3   POINTER_AXIS     +14.55s	vert -15.00* horiz 0.00 (wheel)
-event19  POINTER_AXIS     +14.55s	vert -450.00* horiz 0.00 (wheel)
-event3   POINTER_AXIS     +14.57s	vert -15.00* horiz 0.00 (wheel)
-event19  POINTER_AXIS     +14.58s	vert -450.00* horiz 0.00 (wheel)
-event3   POINTER_AXIS     +14.60s	vert -15.00* horiz 0.00 (wheel)
-event19  POINTER_AXIS     +14.61s	vert -450.00* horiz 0.00 (wheel)
-event3   POINTER_AXIS     +14.71s	vert 15.00* horiz 0.00 (wheel)
-event19  POINTER_AXIS     +14.71s	vert 450.00* horiz 0.00 (wheel)
-event3   POINTER_AXIS     +14.73s	vert 15.00* horiz 0.00 (wheel)
-event19  POINTER_AXIS     +14.73s	vert 450.00* horiz 0.00 (wheel)
-event3   POINTER_AXIS     +14.75s	vert 15.00* horiz 0.00 (wheel)
-event19  POINTER_AXIS     +14.75s	vert 450.00* horiz 0.00 (wheel)
-event3   POINTER_MOTION   +14.75s	  0.00/  0.96
-event19  POINTER_MOTION   +14.75s	  0.00/  0.73
-event3   POINTER_MOTION   +14.78s	  0.00/  0.71
-event19  POINTER_MOTION   +14.78s	  0.00/  0.60

Mon, 07 Aug 2017 16:23:07 GMT - Antoine Martin: owner, status changed

Client side support added in r16654 which also cleans up the code nicely (+ minor fixups in r16655). This can be tested without a precise wheel or tablet by using environment variables to adjust the deltas we send to the server, then using an application like Firefox which should be listening for the precise scroll events rather than the old chunky ones. (tools like xev do not, but libinput or xinput can show the precise wheel deltas)

Start the server as root with (your actual uid/gid may vary):

modprobe uinput
xpra start --uid=1000 --gid=1000 --start=firefox --input-devices=uinput -d mouse

And verify that the server does use the uinput virtual device, it should print something like this:

pointer device emulation using UInput device /dev/input/eventNNN

Values greater than 15 will scroll less, lower values will scroll more.

Since those are multipliers, increase the value(s) to scroll more. (those multiplier values may not be correct.. as they're just guesses since the apple documentation is as bad as their API - TBC with hardware testing in #1157)

Some useful debugging tools:


@maxmylyn: with the default scroll values things should work exactly as before, input devices with non-standard scroll values should now be handled better.

Will follow up in #1615 for touch devices (most useful for supporting tablets and phones).


Tue, 08 Aug 2017 02:13:13 GMT - J. Max Mena:

Trying to start my trunk Fedora 25 server as root(using the command in comment:7 and changing it to the correct uid and gid values) it errors out with the following:

Failed to create bus connection: Operation not permitted

I'm sure it's something simple, but I'm stuck at this. Earlier today it was erroring out with a much longer traceback, but I can't seem to get that far anymore.


Tue, 08 Aug 2017 02:14:32 GMT - Antoine Martin:

Failed to create bus connection: Operation not permitted

Try with sudo xpra ... or add --systemd-run=no


Tue, 08 Aug 2017 13:46:57 GMT - Antoine Martin:

r16661 recommends python-uinput in the RPM spec file, but there is a problem with that: it seems that the Fedora RPM package is fundamentally broken at the moment: python-uinput-0.11.2 is available.


Tue, 08 Aug 2017 18:01:05 GMT - J. Max Mena:

Okay that makes sense why I was seeing tracebacks. I'm not at home today, so I'll try that tomorrow morning.


Thu, 10 Aug 2017 20:31:53 GMT - J. Max Mena:

Alright I got my server to finally work nicely with starting as root. Is there a workaround for the broken uinput module?

I'll try downloading the tarball from the upstream source and building it, maybe that'll do it.


Fri, 11 Aug 2017 08:15:22 GMT - Antoine Martin:

I'll try downloading the tarball from the upstream source and building it, maybe that'll do it.

You can do that, but this is easier:

easy_install -U -Z python-uinput

To make it even easier, I've added python2-uinput rpm to the stable Fedora and centos repositories: r16678. So dnf upgrade --refresh should be all you need now.


Fri, 11 Aug 2017 17:18:10 GMT - J. Max Mena: owner changed

Okay, I finally got the time to actually test this very thoroughly. And, it works fantastically. One thing I've noted is that you have to change the XPRA_XINPUT_WHEEL_DIV=15 flag by large numbers before you see a notable affect on scroll speed. Setting it to 60 doesn't seem to notably affect scrolling (except it takes an extra wheel "click" to get it to start scrolling), but setting it to something extreme like 500 makes scrolling painfully slow (intended).

It also doesn't appear to have broken anything in OSX, surprisingly (r16657 client). So that's a win. However, changing the XPRA_OSX_WHEEL_MULTIPLIER=100 doesn't appear to have any effect at all. But, leaving it at the default works fine.

The Windows client also doesn't seem to have broken anything. However I'm noting that changing the XPRA_WIN32_WHEEL_DELTA doesn't appear to have any effect at all. BUT, leaving it at the default works just as fine as before.

---

Tempted to close this and open a new ticket to follow up the OSX and Win32 variables not having any effect. Unless that's intended? (probably not but I feel like asking just in case)


Fri, 11 Aug 2017 17:24:17 GMT - J. Max Mena:

Okay I should have double checked the buttons were working earlier, but I was focused on scrolling.

Right now right and middle click are reversed in all the clients.


Fri, 11 Aug 2017 17:32:03 GMT - Antoine Martin: owner changed

Changing the env settings on macos and win32 should have a noticeable effect:

XPRA_OSX_WHEEL_MULTIPLIER=100 is the default value, so that's not expected to change anything, try 1000 or 10 instead.

As for the middle and right clicks being reversed: try r16685 (tiny server side fix).


Fri, 11 Aug 2017 18:36:12 GMT - J. Max Mena:

Okay the reversed mouse buttons was fixed by that update. That being said, I'll follow up on the OSX/Win32 things after lunch.


Fri, 25 Aug 2017 20:04:24 GMT - J. Max Mena: owner changed

So "after lunch" turned into two weeks later. Long story short I was reassigned to other things around lunch time, and then the next two weeks I was given a list of other things to do, that was a higher priority than this.


Anyways, back to this ticket.

I updated my server to r16705, but left the client versions the same.

I double checked the Windows logs and the XPRA_WIN32_WHEEL_DELTA is indeed getting changed. Same thing for OSX, the normalize_precision prints are off by about a factor of 100 from each other when setting XPRA_OSX_WHEEL_MULTIPLIER to 10 and 1000.

As far as I can tell, that was the only outstanding thing left in this ticket; everything else seems to be working fine with uinput (been using it for the last few hours and everything plays nicely for now, and all buttons seem to work properly on my Logitech Performance MX mouse). Do you want to go ahead and close it, or is there still any more work to be done?

I do need to get ahold of a special Apple mouse (that weird Magic Mouse) to double check that, but there's no reason we can't follow up with that in a new ticket if something is broken. However, I did test with my Apple Laptop's (A 2014ish Macbook Pro something or other) trackpad and it behaved just fine.


Sat, 26 Aug 2017 04:46:13 GMT - Antoine Martin: status changed; resolution set

Will follow up in #1615 and #1631


Mon, 06 Nov 2017 08:29:21 GMT - Antoine Martin:

Minor fixes in r17315, r17110


Thu, 14 Dec 2017 06:28:13 GMT - Antoine Martin:

Minor fix in r17631: check uinput device permissions early, so we don't use the xorg-uinput config unless uinput is actually likely to be used.


Fri, 16 Feb 2018 05:51:49 GMT - Antoine Martin:

As of r18445 the custom udev script is now called "xpra_udev_product_version".


Fri, 16 Feb 2018 17:46:14 GMT - J. Max Mena: status changed; resolution deleted

This revision (r18445) caused a build to fail on both my Fedora machines with the following error:

error: file '/root/trunk/src/scripts/xpra_udev_product_version' does not exist

r18444 builds fine


Fri, 16 Feb 2018 17:48:39 GMT - Antoine Martin: owner, status changed

This revision (r18445) caused a build to fail ...

Try r18446.


Tue, 20 Feb 2018 18:32:12 GMT - J. Max Mena: status changed; resolution set

r18446 and subsequent builds no longer fail - closing again (must have forgot to hit submit on Friday)


Fri, 30 Mar 2018 13:23:24 GMT - Antoine Martin:

Important fixes in r18916, see ticket:1615#comment:2.

For html5 client support, see #1797. Follow up: #1615 touch device support.


Fri, 31 Aug 2018 05:50:48 GMT - Antoine Martin:

See also Reporting high-resolution scroll events - almost exactly the same findings, with one major difference: apparently SDL has problems so they ("Chromium OS Touch / Input Team") are proposing REL_WHEEL_HI_RES and REL_HWHEEL_HI_RES to report high resolution wheel events. We can do that.


Fri, 07 Dec 2018 19:58:37 GMT - Antoine Martin:

Update: newer kernels may include this updated patch: MS and Logitech high-resolution scroll wheel support


Tue, 11 Dec 2018 00:08:09 GMT - Antoine Martin:

New issues:


Wed, 12 Dec 2018 18:29:12 GMT - Antoine Martin:

Re REL_WHEEL_HI_RES, see also: High resolution wheel scrolling on Linux v4.21


Sat, 23 Jan 2021 05:29:04 GMT - migration script:

this ticket has been moved to: https://github.com/Xpra-org/xpra/issues/1611