Xpra: Ticket #1339: Multimonitor Offset for Mouse Over

Title: Multimonitor Offset for Mouse Over

Situation: Using the 2nd monitor on windows client rotated to portrait and then shifted up higher then the primary in the multiple displays settings.

Result: Scroll wheel and other mouse over commands behave like the mouse pointer is off by the same distance the 2nd monitor is shifted above the primary monitor. So in all application irregardless of which monitor they are currently in will not scroll when mouse cursor is higher then that distance from the top of the application windows (doesn't matter if it's maximized or not). Below that offset it will scroll what is offset that distance above the mouse cursor. I hope this makes sense. Please check screenshot for monitor config.



Wed, 12 Oct 2016 22:42:57 GMT - jdawgzim: attachment set


Fri, 21 Oct 2016 12:50:49 GMT - Antoine Martin: attachment set

two side by side screens with windows XP


Fri, 21 Oct 2016 12:54:52 GMT - Antoine Martin: attachment set

the screen config as shown in the bug report screenshot


Fri, 21 Oct 2016 12:58:06 GMT - Antoine Martin: owner changed

Display data from the bug report tool for this Microsoft Windows 7 client:

{'screens': {
 0: {
    'name': '1\\WinSta0\\Default',
    'height_mm': 508, 'width_mm': 793,
    'settings': {}, 'resolution': -1, 'primary_monitor': 0,
    'height': 1920, 'width': 3000,
    'visual': {
        'rgb': {
            'visual_type': 'TRUE_COLOR', 'byte_order': 'LSB', 'depth': 24, 'bits_per_rgb': 42, 'colormap_size': 256
        },
        'system_visual': {
            'visual_type': 'TRUE_COLOR', 'byte_order': 'LSB', 'depth': 24, 'bits_per_rgb': 42, 'colormap_size': 256
        }
     },
     'root': (0, 0, 1920, 1200, 24),
     'monitors': 2,
     'monitor': {
         0: {
             'height_mm': 423, 'width_mm': 677, 'height': 1200, 'width': 1920, 'y': 120, 'x': 0, 'plug_name': '\\\\.\\DISPLAY1' \
         },
         1: {
             'height_mm': 677, 'width_mm': 381, 'height': 1920, 'width': 1080, 'y': 0, 'x': 1920, 'plug_name': '\\\\.\\DISPLAY2' \
            }
         }
     }
},
'device': {
    0: {
        '': 'Core Pointer', 'keys': [], 'num_axes': 2, 'axes': ((1, 0.0, 0.0), (2, 0.0, 0.0)), 'num_keys': 0, 'source': 'MOUSE', 'mode': 'SCREEN', 'has_cursor': True
    }
},
'maximal_cursor_size': (32, 32), 'pointer': (1623, 892),
'name': '1\\WinSta0\\Default',
'root': (0, 0, 1920, 1200, 24),
'root-size': (3000, 1920),
'devices': 1,
'default_cursor_size': 32L,
'pointer_is_grabbed': False,
'supports': {
    'clipboard_persistence': False, 'composite': False, 'cursor_color': True, 'shapes': True, 'selection_notification': True, 'cursor_alpha': True}
}

This seems to match the configuration in the screenshot you included: the screen config as shown in the bug report screenshot

So the first monitor has a "y" offset of 120. I believe that win32 and/or GTK may be using negative coordinates for the second monitor, and X11 will discard those. When I tested it though, it worked without problems: two  side by side screens with windows XP

So please try to run the client with the "-d mouse" flag and post the output. We want to see what coordinates are reported in the mouse events for when the window is near the top of the second monitor. (just in case, please also include the server log output so we can see the initial screen setup, and maybe also "xpra info")


Fri, 28 Oct 2016 16:39:25 GMT - jdawgzim: attachment set


Fri, 28 Oct 2016 16:43:29 GMT - jdawgzim:


Fri, 28 Oct 2016 16:46:06 GMT - Antoine Martin:

@jdawgzim: you've attached the bug report data, we need to see the "xpra_cmd.exe" command output instead. (with the misbehaving configuration obviously)


Fri, 28 Oct 2016 17:30:19 GMT - jdawgzim:

Replying to antoine: How do I do that?


Fri, 28 Oct 2016 17:40:16 GMT - Antoine Martin:

Run xpra from the command line, ie: open did command prompt then

cd ..
cd ..
cd Program Files
cd Xpra
Xpra_cmd.exe attach to:host:port -d mouse

(Paths may vary slightly)


Fri, 28 Oct 2016 18:00:07 GMT - jdawgzim:

I use cmd.exe and run "Xpra_cmd.exe attach to:host:port -d mouse" and I get a lot of text. It doesn't let me copy the text. Is there a .log output somewhere or some way to capture this? Thank you for your patience with me


Fri, 28 Oct 2016 18:05:33 GMT - jdawgzim: attachment set


Fri, 28 Oct 2016 18:29:59 GMT - alas:

If you left click at the top left of the cmd.exe window, you should find an 'edit' option, which will allow you to select/enable copying. Then you'll just need to left-click to highlight the text you want to copy, then left click on it again to sync it with the clipboard.


Fri, 28 Oct 2016 18:40:15 GMT - jdawgzim: attachment set


Sat, 29 Oct 2016 04:42:24 GMT - Antoine Martin:

A better and easier way of capturing log output, one that does not wrap the lines at 80 characters (which makes them so much harder to parse), is:

Xpra_cmd.exe attach to:host:port -d mouse > C:\xpra-log.txt 2>&1

Then you can just attach "C:\xpra-log.txt".


For my own reference, the code that converts the event's native coordinates is:

directly from the WM_MOUSEMOVE event message. As of r14323 we will now log both the root coordinates and plain (x,y). The good news is that the win32 code gives us the raw values. It's quite possible that the window ends up using negative coordinates on win32, which then gets mapped partially offscreen on the server and is therefore unable to receive any events in that region.


@jdawgzim: the mouse in your short log sample is not anywhere near the top of the screen, the y coordinate is near ~1000 which is not where the problem occurs from what I understand:

do_motion_notify_event(<gtk.gdk.Event at 0577E590: GDK_MOTION_NOTIFY x=1684.00, y=1115.00>) \
    wid=9 / focus=9, device=Core Pointer, pointer=(2773, 1431), modifiers=['mod2'], buttons=[]

please try the latest client beta build. It includes a workaround for #1284 which may help, and the extra debug logging should also help narrow things down. Please make sure to move the mouse in the problematic area, and keep the log sample as small as possible. Also, it would be easier to parse the output if you add "--desktop-scaling=no" to the command line. If the coordinates are negative, it may be useful to also have the "-d geometry" log output. Finnaly, a screenshot showing where the mouse was when the log was capture would be ideal.


Mon, 26 Dec 2016 09:41:24 GMT - Antoine Martin: owner changed

Not heard back, @afarr: can you help with this?


Wed, 04 Jan 2017 20:30:42 GMT - jdawgzim:

Sorry, I worked around this issue by putting my vertical monitor on the left and switching the vertical monitor to be the primary monitor (start menu).

If I switch back to making the right horizontal monitor the primary monitor the issue returns. If I right or left click the mouse it is correct. It's just a "mouse over scroll wheel action" that is the offset the same amount as the two monitors are offset relative to where the primary monitor (start menu) is. It's like Xpra needs the start menu or primary monitor to be on the far left bottom.

I don't run the Xpra server where I use Xpra so I can't update it. I could update my client, but that probably wouldn't fix it. If I get brave enough in the future I could try to recreate this problem on my home computer and get you more info.


Thu, 05 Jan 2017 17:58:37 GMT - J. Max Mena:

Retrying with a trunk r14619 Win8.1 client against a trunk r14712 Fedora 25 server and am only able to reproduce the scrolling offset. The mouse click actions behave nicely, but the scrolling does not. I'll attach a -d mouse log in a sec.


Thu, 05 Jan 2017 18:00:36 GMT - J. Max Mena: owner changed

There you go, hand it back to me if you want any other logs.


Thu, 05 Jan 2017 18:01:37 GMT - J. Max Mena: attachment set

Here's a screenshot of the monitor layout I'm using - both are 1080p


Fri, 06 Jan 2017 05:28:16 GMT - Antoine Martin: owner changed

@maxmylin: the log file you attached doesn't have any mouse event debugging in it.

It should look like this with "-d mouse":


Fri, 06 Jan 2017 16:54:22 GMT - J. Max Mena: owner changed

Oops It helps to remember to add -d mouse to the command line arguments. That would explain why the file was so small..

Anyways, steps are the same:

The clicks (left, middle, and right mouse) all registered in the correct place, but the scrolls were indeed offset.


Fri, 06 Jan 2017 16:54:40 GMT - J. Max Mena: attachment set

did some scrolling and clicking - the scrolls were offset but the clicks were not.


Fri, 06 Jan 2017 17:35:39 GMT - Antoine Martin: owner changed

Does the problem go away if you run the client with XPRA_WHEEL=0?

Pointer events are adjusted for the second monitor (964 vs 638, 1339 vs 730):

do_motion_notify_event(<gtk.gdk.Event at 0482FE18: GDK_MOTION_NOTIFY x=638.00, y=730.00>) wid=4 / focus=4 / window wid=4, device=Core Pointer, pointer=(964, 1339), modifiers=['mod2'], buttons=[]

But our wheel event throttling code (#1131) doesn't know about the screen adjustments made by GTK.. and so we send the raw values, and those will be offset:

mousewheel: orientation=vertical distance=-120.0, units=-1, new value=-120.0, keys=0x0, x=964, y=849, client=gtk2.client, wid=4

If you confirm that this is indeed the problem, I'll try to find a solution. Potential inelegant solutions I can think of:


Fri, 06 Jan 2017 18:04:39 GMT - J. Max Mena: owner changed

Yeah, setting XPRA_WHEEL=0 seems to make the problem go away.


Sat, 07 Jan 2017 06:44:55 GMT - Antoine Martin: owner changed

Should be fixed in r14717 + r14718 (trunk), already backported to v1.0.x in r14719 as I am reasonably confident that the fix is correct or at least that it shouldn't make things worse.


Mon, 09 Jan 2017 18:22:18 GMT - J. Max Mena: owner changed

Scrolling using the wheel seems to not work at all anymore, and instead seems to freeze the whole client. Some aggressive scrolling attempts caused the whole server to crash on me - I'll attach the serverside logs from that attempt (includes a traceback and an error I was seeing on the client side) and also another attempt with the client side -d mouse logs.


Mon, 09 Jan 2017 18:22:48 GMT - J. Max Mena: attachment set

some aggressive scrolling crashed the server


Mon, 09 Jan 2017 18:25:49 GMT - J. Max Mena: attachment set

same as before, but the client hung on the terminal after control-c'ing - error in log (caused a "Xpra has stopped working" dialogue to pop up)


Mon, 09 Jan 2017 20:04:48 GMT - Antoine Martin: owner changed

@maxmylin: which version were you testing with? My bet is on 2.0

I believe that the bug you are reporting is caused by #678 in version 2.0 (2.0 is going to be in a state of flux as development has just started) but this should not be affecting version 1.x or older - which I was very worried about.

r14735 should fix this particular issue in 2.0 for now - but this fix is likely to be in the "wrong" place: we should be sending signed 16-bit values from the wnd proc callbacks, pywin32 was giving us signed integers but ctypes isn't (or at least not yet). Other things are likely to misbehave in 2.0 until that is fixed. Anyway, new beta 2.0 build uploaded too.

Please confirm that the bug was with 2.0 only, and that 1.0 is fixed. (2.0 should also be fixed, but that's less of a concern at this point)


Mon, 09 Jan 2017 20:08:02 GMT - J. Max Mena:

@antoine - whoops sorry, should have mentioned trunk

Can you post new 1.0 builds?

Edit: nevermind - I forgot there's also /dists/


Tue, 10 Jan 2017 05:52:14 GMT - Antoine Martin:

Can you post new 1.0 builds?


1.0.1 was released with this fixed included, see: http://xpra.org/dists/windows/


Tue, 10 Jan 2017 17:47:59 GMT - J. Max Mena: owner changed

Sorry was busy yesterday and didn't get to this ticket.

Using the 1.0.1 r14723 Client against a Fedora 25 r14751 1.X server I've found the bug hasn't been fixed - the mouse wheel events are still being sent to the wrong area as described in the description.

edit: I'll attach another log of just scrolling and mouse movements.


Tue, 10 Jan 2017 17:49:30 GMT - J. Max Mena: attachment set

This time no clicks - just mouse movements and scrolling


Wed, 11 Jan 2017 03:39:11 GMT - Antoine Martin: owner changed

Oh, so there were actually two bugs then. The second may be fixed by r14754. The "-d mouse" output should now include the "gtk_offset" value which is applied to raw win32 wheel events to adjust for the gtk coordinate system. (it's always (0,0) on my test system.. but if I got my maths the right way up, it should cancel out the offset) New beta 2.0 win32 build uploaded. (with all the other known issues..)


Thu, 16 Feb 2017 04:30:20 GMT - Antoine Martin: priority changed

raising - we should include this in the 1.0 branch if the fix is confirmed


Thu, 16 Feb 2017 18:21:34 GMT - J. Max Mena:

Using the latest 1.X build (r15032) with a Win8.1 machine against a Fedora 25 trunk r15094 server:

Sample scroll output from scrolling:

2017-02-16 10:17:57,732 mousewheel: orientation=vertical distance=-120.0, units=1, new value=-120.0, keys=0x0, x=64676,
y=806, client=gtk2.client, wid=4
2017-02-16 10:17:57,733 mousewheel: sent 1 wheel events to the server for distance=-120, remainder=0
2017-02-16 10:17:57,740 mousewheel: orientation=vertical distance=-120.0, units=1, new value=-120.0, keys=0x0, x=64676,
y=806, client=gtk2.client, wid=4
2017-02-16 10:17:57,740 mousewheel: sent 1 wheel events to the server for distance=-120, remainder=0
2017-02-16 10:17:57,756 mousewheel: orientation=vertical distance=-120.0, units=1, new value=-120.0, keys=0x0, x=64676,
y=806, client=gtk2.client, wid=4
2017-02-16 10:17:57,756 mousewheel: sent 1 wheel events to the server for distance=-120, remainder=0
2017-02-16 10:17:57,763 mousewheel: orientation=vertical distance=-120.0, units=1, new value=-120.0, keys=0x0, x=64676,
y=806, client=gtk2.client, wid=4
2017-02-16 10:17:57,765 mousewheel: sent 1 wheel events to the server for distance=-120, remainder=0
2017-02-16 10:17:58,092 mousewheel: orientation=vertical distance=-120.0, units=1, new value=-120.0, keys=0x0, x=64676,
y=806, client=gtk2.client, wid=4
2017-02-16 10:17:58,092 mousewheel: sent 1 wheel events to the server for distance=-120, remainder=0
2017-02-16 10:17:58,099 mousewheel: orientation=vertical distance=-120.0, units=1, new value=-120.0, keys=0x0, x=64676,
y=806, client=gtk2.client, wid=4
2017-02-16 10:17:58,101 mousewheel: sent 1 wheel events to the server for distance=-120, remainder=0
2017-02-16 10:17:58,108 mousewheel: orientation=vertical distance=-120.0, units=1, new value=-120.0, keys=0x0, x=64676,
y=806, client=gtk2.client, wid=4
2017-02-16 10:17:58,108 mousewheel: sent 1 wheel events to the server for distance=-120, remainder=0
2017-02-16 10:17:58,115 mousewheel: orientation=vertical distance=-120.0, units=1, new value=-120.0, keys=0x0, x=64676,
y=806, client=gtk2.client, wid=4
2017-02-16 10:17:58,117 mousewheel: sent 1 wheel events to the server for distance=-120, remainder=0
2017-02-16 10:17:58,124 mousewheel: orientation=vertical distance=-240.0, units=2, new value=-240.0, keys=0x0, x=64676,

Thu, 16 Feb 2017 18:28:44 GMT - J. Max Mena:

The 2.X r15030 client works fine.


Thu, 16 Feb 2017 18:29:38 GMT - Antoine Martin: priority changed

The fix to test for this multi monitor bug is in the 2.0 branch.

If wheel events no longer work at all in the 1.x branch then this is serious issue - so raising to blocker. That's surprising seeing that your log samples indicate that the wheel events have been sent to the server.

And I have just tested 1.0.3 and I see the events:


Thu, 16 Feb 2017 19:30:24 GMT - J. Max Mena:

NOTE:

The 1.0.X client only stops scrolling if there more than monitor attached to the client machine - after disconnecting the second monitor, scrolling works again.


Thu, 16 Feb 2017 19:33:33 GMT - J. Max Mena:

Also:

Setting XPRA_WHEEL=0 gets scrolling to work again with both monitors plugged in.


Fri, 17 Feb 2017 12:34:58 GMT - Antoine Martin: priority changed

Right, so the 1.0.x client is still affected by this bug - which is totally expected since the fix wasn't in there yet. I was thinking that something broke in 1.x - stand down red alert.

I have uploaded a beta 1.0.4 with this change backported (r15100 + r15104, backport was made more difficult by the switch to ctypes #678): http://xpra.org/beta/windows.


Tue, 21 Feb 2017 18:10:38 GMT - J. Max Mena:

Latest 1.0.X build (r15129) fixed it - connecting against a 2.0 r15129 Fedora 25 server scrolling works fine with multimonitor in a number of configurations.

I'll hold on to this for a few hours to double check nothing else broke, and then close it if everything checks out.


Tue, 21 Feb 2017 22:10:38 GMT - J. Max Mena: status changed; resolution set

Yep, everything looks good - closing.


Mon, 05 Aug 2019 12:14:44 GMT - Antoine Martin: attachment set

context menu shifted up and with pointer offset


Mon, 05 Aug 2019 12:16:31 GMT - Antoine Martin: status changed; resolution deleted

Following this mailing list post, I am seeing this problem again with 3.0 and many applications:

Looks like the client OS decides to shift the popup to keep it all on-screen, but the server copy is not shifted so the events land in the wrong place? context menu shifted up and with pointer offset


Mon, 05 Aug 2019 14:18:21 GMT - Antoine Martin: owner, status changed

Downgrading the server all the way to v1.0.14 does not help. Bisecting the client:

So the bug was backported to the v2.3.x branch between r19350 (v2.3.1) and r19657 (v2.3.2). The list of changesets is here: rev 19657 - 61 changes


Mon, 05 Aug 2019 15:26:45 GMT - totaamwin32: owner, status changed

Potential problems in r19651? Less likely in:


Mon, 05 Aug 2019 16:22:54 GMT - totaamwin32: attachment set

this hack of a patch fixes the symptoms for this particular case..


Tue, 06 Aug 2019 15:20:26 GMT - totaamwin32: status changed; resolution set

Fixed in r23459.


Sat, 23 Jan 2021 05:21:33 GMT - migration script:

this ticket has been moved to: https://github.com/Xpra-org/xpra/issues/1339