xpra icon
Bug tracker and wiki

Opened 5 months ago

Closed 4 months ago

Last modified 5 weeks ago

#1611 closed task (fixed)

uinput virtual pointer device

Reported by: Antoine Martin Owned by: Antoine Martin
Priority: major Milestone: 2.2
Component: server Version: trunk
Keywords: Cc:

Description

split from #173 with a more limited scope: just implement the pointer events using uinput instead of xtest.

This can help us with:

Change History (20)

comment:1 Changed 5 months ago by Antoine Martin

Status: newassigned

Based on Bug 92772 - RFE: extra option to define the vdist/hdist for scrolling event, we might be able to generate "smooth-scrolling" / small scrolling deltas by creating our uinput device with a very small MOUSE_WHEEL_CLICK_ANGLE (or better, use a high MOUSE_WHEEL_CLICK_COUNT) then synthesizing events using the correct multiplier.

This should be doable using libinput: Static device configuration via udev.
Similar to Change scroll speed with libinput.

This looks also interesting for touch devices: Create a virtual multitouch device using uinput driver.

Problem is that no matter what I specify for the uinput device or udev rules, the properties are not set. (and not even the built-in rules are? system bug?)

Last edited 5 months ago by Antoine Martin (previous) (diff)

comment:2 Changed 5 months ago by Antoine Martin

Code added in r16572.

Still TODO:

  • modprobe uinput to ensure we can use it?
  • bug: server shutdown no longer cleans up the sockets?
  • when we upgrade an existing server... we don't have the device details any more - should we save those somewhere?
  • we should verify that the uinput device works before using it? (send motion, wait a little, verify that the core pointer has moved)
  • the udev race is annoying and any call to udevadm trigger can reset the device permissions to something that prevents the Xorg process from accessing our uinput device.. we cannot write a udev rule because we don't know the uid in advance, so how can we tell udev to just leave it alone? Do it the hard way and just mknod + chown a private one?
  • scroll speed as per comment:1
Last edited 5 months ago by Antoine Martin (previous) (diff)

comment:3 Changed 5 months ago by Antoine Martin

udev meanderings, trying to use hwdb to force the click angle / count:

cat > /lib/udev/hwdb.d/71-mouse-xpra.hwdb <<EOF
# allow xpra to use smooth scrolling
mouse:usb:*:name:Xpra Virtual Pointer:
 MOUSE_WHEEL_CLICK_ANGLE=1
 MOUSE_WHEEL_CLICK_COUNT=360

mouse:*:name:Xpra Virtual Pointer:
 MOUSE_WHEEL_CLICK_ANGLE=1
 MOUSE_WHEEL_CLICK_COUNT=360

mouse:usb:v0088p0088:*
 MOUSE_WHEEL_CLICK_ANGLE=1
 MOUSE_WHEEL_CLICK_COUNT=360
EOF
udevadm hwdb --update

Then starting a server:

sudo xpra start-desktop --uid=1000 --gid=1000 :100 --no-daemon \
    --start=xterm --bind-tcp=0.0.0.0:10000 -d mouse --input-devices=uinput
(..)
pointer device emulation using UInput device /dev/input/event20
(..)

Shows no new udev attributes:

$ sudo udevadm info -a --name /dev/input/event20
  looking at device '/devices/virtual/input/input44/event20':
    KERNEL=="event20"
    SUBSYSTEM=="input"
    DRIVER==""

  looking at parent device '/devices/virtual/input/input44':
    KERNELS=="input44"
    SUBSYSTEMS=="input"
    DRIVERS==""
    ATTRS{name}=="Xpra Virtual Pointer"
    ATTRS{phys}==""
    ATTRS{properties}=="0"
    ATTRS{uniq}==""

Trying with a plain udev rule:

cat > /lib/udev/rules.d/71-xpra-mouse.rules << EOF
ACTION=="add|change", SUBSYSTEM=="input", ATTRS{name}=="Xpra Virtual Pointer", ENV{MOUSE_WHEEL_CLICK_ANGLE}="1"
EOF
udevadm control --reload-rules && udevadm trigger

Same result.

$ libinput list-devices
(...)
Device:           Xpra Virtual Pointer
Kernel:           /dev/input/event20
Group:            10
Seat:             seat0, default
Capabilities:     pointer 
Tap-to-click:     n/a
Tap-and-drag:     n/a
Tap drag lock:    n/a
Left-handed:      disabled
Nat.scrolling:    disabled
Middle emulation: disabled
Calibration:      n/a
Scroll methods:   button
Click methods:    none
Disable-w-typing: n/a
Accel profiles:   flat *adaptive
Rotation:         n/a

Did the flat acceleration option not work? (could cause problems when we use relative motion to go where we want)

Or maybe it does do something (the plain udev rule), but not on the correct device? (/devices/virtual/input/input46 but not on /devices/virtual/input/input46/event20)

$ udevadm monitor -ukp

KERNEL[21719.101574] add      /devices/virtual/input/input46 (input)
ACTION=add
DEVPATH=/devices/virtual/input/input46
EV=7
KEY=7f0000 0 0 0 0
MODALIAS=input:b0003v0000p0000e0000-e0,1,2,k110,111,112,113,114,115,116,r0,1,8,amlsfw
NAME="Xpra Virtual Pointer"
PRODUCT=3/0/0/0
PROP=0
REL=103
SEQNUM=11953
SUBSYSTEM=input

KERNEL[21719.101695] add      /devices/virtual/input/input46/mouse2 (input)
ACTION=add
DEVNAME=/dev/input/mouse2
DEVPATH=/devices/virtual/input/input46/mouse2
MAJOR=13
MINOR=34
SEQNUM=11954
SUBSYSTEM=input

KERNEL[21719.101768] add      /devices/virtual/input/input46/event20 (input)
ACTION=add
DEVNAME=/dev/input/event20
DEVPATH=/devices/virtual/input/input46/event20
MAJOR=13
MINOR=84
SEQNUM=11955
SUBSYSTEM=input

UDEV  [21719.102443] add      /devices/virtual/input/input46 (input)
.INPUT_CLASS=mouse
ACTION=add
DEVPATH=/devices/virtual/input/input46
EV=7
ID_INPUT=1
ID_INPUT_MOUSE=1
ID_SERIAL=noserial
KEY=7f0000 0 0 0 0
MODALIAS=input:b0003v0000p0000e0000-e0,1,2,k110,111,112,113,114,115,116,r0,1,8,amlsfw
MOUSE_WHEEL_CLICK_ANGLE=1
NAME="Xpra Virtual Pointer"
PRODUCT=3/0/0/0
PROP=0
REL=103
SEQNUM=11953
SUBSYSTEM=input
TAGS=:seat:
USEC_INITIALIZED=21719102303

UDEV  [21719.103088] add      /devices/virtual/input/input46/mouse2 (input)
.INPUT_CLASS=mouse
ACTION=add
DEVNAME=/dev/input/mouse2
DEVPATH=/devices/virtual/input/input46/mouse2
ID_INPUT=1
ID_INPUT_MOUSE=1
ID_SERIAL=noserial
MAJOR=13
MINOR=34
SEQNUM=11954
SUBSYSTEM=input
USEC_INITIALIZED=21719102990

UDEV  [21719.129999] add      /devices/virtual/input/input46/event20 (input)
.INPUT_CLASS=mouse
ACTION=add
DEVNAME=/dev/input/event20
DEVPATH=/devices/virtual/input/input46/event20
ID_INPUT=1
ID_INPUT_MOUSE=1
ID_SERIAL=noserial
MAJOR=13
MINOR=84
SEQNUM=11955
SUBSYSTEM=input
USEC_INITIALIZED=21719129875
Last edited 4 months ago by Antoine Martin (previous) (diff)

comment:4 Changed 5 months ago by Antoine Martin

Actually, the udev variant does do something - the scrolling is slower than usual with these values:

ACTION=="add|change", ATTRS{name}=="Xpra Virtual Pointer", \
ENV{MOUSE_WHEEL_CLICK_ANGLE}="1", ENV{MOUSE_WHEEL_CLICK_COUNT}="360"

Success!

Last edited 4 months ago by Antoine Martin (previous) (diff)

comment:5 Changed 4 months ago by Antoine Martin

New issue: Xorg version 1.17.x as found in centos7 doesn't support the xorg.conf options we need (caught by the rpmbuild unit tests):

Parse error on line 17 of section InputClass in file /usr/src/rpmbuild/BUILD/xpra-2.2-python2/unittests/../etc/xpra/xorg.conf
        "NoMatchProduct" is not a valid keyword in this section.

Does it support MatchProduct? Can we get away with not specifying NoMatchProduct and have Ignore=True for all devices, then override for the devices we do want to use?

If not, we'll need a different xorg.conf for centos7, and disable uinput. (we could also rely on the selftests in case someone enables uinput on centos7)
Also looks like centos 7.4 will support this: https://bugzilla.redhat.com/show_bug.cgi?id=1401641 (updated to xorg 1.19.x)

Last edited 4 months ago by Antoine Martin (previous) (diff)

comment:6 Changed 4 months ago by Antoine Martin

Updates:

  • r16648 + r16649: install uinput xorg config, and switch to this file if we find --input-devices=uinput
  • r16650: verify uinput device is being received by the X11 server, fallback to XTest if it isn't
  • r16651: make sure we match the device path exactly (oops!)
  • r16652: ship udev rule, XPRA_MOUSE_WHEEL_CLICK_MULTIPLIER=30 env var, logging

With all this in place, we have the ability to send fine grained scrolling events. This is what libinput sees (event3 is the real pointer, event19 our uinput virtual one - which is a high precision one and uses higher values):

-event3   POINTER_MOTION   +14.46s	  1.08/  0.00
 event3   POINTER_MOTION   +14.46s	  0.00/  1.08
-event19  POINTER_MOTION   +14.46s	  0.00/  0.82
-event3   POINTER_MOTION   +14.46s	  0.00/  1.07
-event19  POINTER_MOTION   +14.46s	  0.83/  0.83
 event19  POINTER_MOTION   +14.47s	  0.00/  0.85
-event3   POINTER_MOTION   +14.47s	  1.04/  0.00
-event19  POINTER_MOTION   +14.47s	  0.86/  0.86
 event19  POINTER_MOTION   +14.47s	  0.00/  0.87
 event19  POINTER_MOTION   +14.48s	  0.87/  0.00
-event3   POINTER_AXIS     +14.55s	vert -15.00* horiz 0.00 (wheel)
-event19  POINTER_AXIS     +14.55s	vert -450.00* horiz 0.00 (wheel)
-event3   POINTER_AXIS     +14.57s	vert -15.00* horiz 0.00 (wheel)
-event19  POINTER_AXIS     +14.58s	vert -450.00* horiz 0.00 (wheel)
-event3   POINTER_AXIS     +14.60s	vert -15.00* horiz 0.00 (wheel)
-event19  POINTER_AXIS     +14.61s	vert -450.00* horiz 0.00 (wheel)
-event3   POINTER_AXIS     +14.71s	vert 15.00* horiz 0.00 (wheel)
-event19  POINTER_AXIS     +14.71s	vert 450.00* horiz 0.00 (wheel)
-event3   POINTER_AXIS     +14.73s	vert 15.00* horiz 0.00 (wheel)
-event19  POINTER_AXIS     +14.73s	vert 450.00* horiz 0.00 (wheel)
-event3   POINTER_AXIS     +14.75s	vert 15.00* horiz 0.00 (wheel)
-event19  POINTER_AXIS     +14.75s	vert 450.00* horiz 0.00 (wheel)
-event3   POINTER_MOTION   +14.75s	  0.00/  0.96
-event19  POINTER_MOTION   +14.75s	  0.00/  0.73
-event3   POINTER_MOTION   +14.78s	  0.00/  0.71
-event19  POINTER_MOTION   +14.78s	  0.00/  0.60
Last edited 4 months ago by Antoine Martin (previous) (diff)

comment:7 Changed 4 months ago by Antoine Martin

Owner: changed from Antoine Martin to J. Max Mena
Status: assignednew

Client side support added in r16654 which also cleans up the code nicely (+ minor fixups in r16655).
This can be tested without a precise wheel or tablet by using environment variables to adjust the deltas we send to the server, then using an application like Firefox which should be listening for the precise scroll events rather than the old chunky ones. (tools like xev do not, but libinput or xinput can show the precise wheel deltas)

Start the server as root with (your actual uid/gid may vary):

modprobe uinput
xpra start --uid=1000 --gid=1000 --start=firefox --input-devices=uinput -d mouse

And verify that the server does use the uinput virtual device, it should print something like this:

pointer device emulation using UInput device /dev/input/eventNNN
  • testing with a linux client (maybe we should make input-devices=xi the default now):
    XPRA_XINPUT_WHEEL_DIV=15 xpra attach --input-devices=xi -d mouse
    

Values greater than 15 will scroll less, lower values will scroll more.

  • testing with a MS Windows client (see WM_MOUSEWHEEL for the value of 120):
    set XPRA_WIN32_WHEEL_DELTA=120
    xpra_cmd attach ...
    
  • with macos (see ticket:1157#comment:24):
    XPRA_OSX_WHEEL_MULTIPLIER=100 XPRA_OSX_WHEEL_PRECISE_MULTIPLIER=1 xpra attach ...
    

Since those are multipliers, increase the value(s) to scroll more.
(those multiplier values may not be correct.. as they're just guesses since the apple documentation is as bad as their API - TBC with hardware testing in #1157)

Some useful debugging tools:

  • udevadm monitor -ukp (as root): shows devices being created when starting the server, with their settings
  • libinput list-devices and libinput debug-events
  • xinput test-xi2 DEVICENO xinput2 events for a particular device

@maxmylyn: with the default scroll values things should work exactly as before, input devices with non-standard scroll values should now be handled better.

Will follow up in #1615 for touch devices (most useful for supporting tablets and phones).

Last edited 4 months ago by Antoine Martin (previous) (diff)

comment:8 Changed 4 months ago by J. Max Mena

Trying to start my trunk Fedora 25 server as root(using the command in comment:7 and changing it to the correct uid and gid values) it errors out with the following:

Failed to create bus connection: Operation not permitted

I'm sure it's something simple, but I'm stuck at this. Earlier today it was erroring out with a much longer traceback, but I can't seem to get that far anymore.

Last edited 4 months ago by J. Max Mena (previous) (diff)

comment:9 Changed 4 months ago by Antoine Martin

Failed to create bus connection: Operation not permitted

Try with sudo xpra ... or add --systemd-run=no

comment:10 Changed 4 months ago by Antoine Martin

r16661 recommends python-uinput in the RPM spec file, but there is a problem with that: it seems that the Fedora RPM package is fundamentally broken at the moment: python-uinput-0.11.2 is available.

comment:11 Changed 4 months ago by J. Max Mena

Okay that makes sense why I was seeing tracebacks. I'm not at home today, so I'll try that tomorrow morning.

comment:12 Changed 4 months ago by J. Max Mena

Alright I got my server to finally work nicely with starting as root. Is there a workaround for the broken uinput module?

I'll try downloading the tarball from the upstream source and building it, maybe that'll do it.

comment:13 Changed 4 months ago by Antoine Martin

I'll try downloading the tarball from the upstream source and building it, maybe that'll do it.

You can do that, but this is easier:

easy_install -U -Z python-uinput

To make it even easier, I've added python2-uinput rpm to the stable Fedora and centos repositories: r16678.
So dnf upgrade --refresh should be all you need now.

comment:14 Changed 4 months ago by J. Max Mena

Owner: changed from J. Max Mena to Antoine Martin

Okay, I finally got the time to actually test this very thoroughly. And, it works fantastically. One thing I've noted is that you have to change the XPRA_XINPUT_WHEEL_DIV=15 flag by large numbers before you see a notable affect on scroll speed. Setting it to 60 doesn't seem to notably affect scrolling (except it takes an extra wheel "click" to get it to start scrolling), but setting it to something extreme like 500 makes scrolling painfully slow (intended).

It also doesn't appear to have broken anything in OSX, surprisingly (r16657 client). So that's a win. However, changing the XPRA_OSX_WHEEL_MULTIPLIER=100 doesn't appear to have any effect at all. But, leaving it at the default works fine.

The Windows client also doesn't seem to have broken anything. However I'm noting that changing the XPRA_WIN32_WHEEL_DELTA doesn't appear to have any effect at all. BUT, leaving it at the default works just as fine as before.

---

Tempted to close this and open a new ticket to follow up the OSX and Win32 variables not having any effect. Unless that's intended? (probably not but I feel like asking just in case)

comment:15 Changed 4 months ago by J. Max Mena

Okay I should have double checked the buttons were working earlier, but I was focused on scrolling.

Right now right and middle click are reversed in all the clients.

comment:16 Changed 4 months ago by Antoine Martin

Owner: changed from Antoine Martin to J. Max Mena

Changing the env settings on macos and win32 should have a noticeable effect:

  • MS Windows: the value for XPRA_WIN32_WHEEL_DELTA should be printed as WHEEL_DELTA with "-d win32" and also with NativeGUI_info.exe -v. It will be shown on mouse wheel events with "-d mouse":
    win32 mousewheel: orientation=%s, distance=%i, wheel-delta=%s, ...
    
  • macos: the multiplier (either XPRA_OSX_WHEEL_MULTIPLIER or XPRA_OSX_WHEEL_PRECISE_MULTIPLIER for precise events) will be shown with "-d mouse" as:
    normalize_precision(%.3f)=%.3f (multiplier=%i)
    

XPRA_OSX_WHEEL_MULTIPLIER=100 is the default value, so that's not expected to change anything, try 1000 or 10 instead.

As for the middle and right clicks being reversed: try r16685 (tiny server side fix).

comment:17 Changed 4 months ago by J. Max Mena

Okay the reversed mouse buttons was fixed by that update. That being said, I'll follow up on the OSX/Win32 things after lunch.

comment:18 Changed 4 months ago by J. Max Mena

Owner: changed from J. Max Mena to Antoine Martin

So "after lunch" turned into two weeks later. Long story short I was reassigned to other things around lunch time, and then the next two weeks I was given a list of other things to do, that was a higher priority than this.


Anyways, back to this ticket.

I updated my server to r16705, but left the client versions the same.

I double checked the Windows logs and the XPRA_WIN32_WHEEL_DELTA is indeed getting changed. Same thing for OSX, the normalize_precision prints are off by about a factor of 100 from each other when setting XPRA_OSX_WHEEL_MULTIPLIER to 10 and 1000.

As far as I can tell, that was the only outstanding thing left in this ticket; everything else seems to be working fine with uinput (been using it for the last few hours and everything plays nicely for now, and all buttons seem to work properly on my Logitech Performance MX mouse). Do you want to go ahead and close it, or is there still any more work to be done?

I do need to get ahold of a special Apple mouse (that weird Magic Mouse) to double check that, but there's no reason we can't follow up with that in a new ticket if something is broken. However, I did test with my Apple Laptop's (A 2014ish Macbook Pro something or other) trackpad and it behaved just fine.

comment:19 Changed 4 months ago by Antoine Martin

Resolution: fixed
Status: newclosed

Will follow up in #1615 and #1631

Last edited 3 months ago by Antoine Martin (previous) (diff)

comment:20 Changed 5 weeks ago by Antoine Martin

Minor fixes in r17315, r17110

Note: See TracTickets for help on using tickets.