xpra icon
Bug tracker and wiki

Opened 6 months ago

Closed 5 weeks ago

#1339 closed defect (fixed)

Multimonitor Offset for Mouse Over

Reported by: jdawgzim Owned by: maxmylyn
Priority: major Milestone: 1.0
Component: client Version: trunk
Keywords: mouse cursor scroll Cc:

Description

Title: Multimonitor Offset for Mouse Over

Situation: Using the 2nd monitor on windows client rotated to portrait and then shifted up higher then the primary in the multiple displays settings.

Result: Scroll wheel and other mouse over commands behave like the mouse pointer is off by the same distance the 2nd monitor is shifted above the primary monitor. So in all application irregardless of which monitor they are currently in will not scroll when mouse cursor is higher then that distance from the top of the application windows (doesn't matter if it's maximized or not). Below that offset it will scroll what is offset that distance above the mouse cursor. I hope this makes sense. Please check screenshot for monitor config.

Attachments (11)

xpra_scroll_bug.zip (464.4 KB) - added by jdawgzim 6 months ago.
xp-two-screens.png (714.7 KB) - added by Antoine Martin 5 months ago.
two side by side screens with windows XP
rotate-config.png (41.5 KB) - added by Antoine Martin 5 months ago.
the screen config as shown in the bug report screenshot
xpra_scroll_bug_2.zip (401.5 KB) - added by jdawgzim 5 months ago.
xpra_scroll_bug.jpg (1.5 MB) - added by jdawgzim 5 months ago.
xpra_scroll_bug.txt (20.7 KB) - added by jdawgzim 5 months ago.
1339 Monitor Layout.png (44.4 KB) - added by maxmylyn 3 months ago.
Here's a screenshot of the monitor layout I'm using - both are 1080p
1339dmouse.txt (312.7 KB) - added by maxmylyn 3 months ago.
did some scrolling and clicking - the scrolls were offset but the clicks were not.
1399crash.log (18.5 KB) - added by maxmylyn 3 months ago.
some aggressive scrolling crashed the server
1399newdmouselog.txt (250.0 KB) - added by maxmylyn 3 months ago.
same as before, but the client hung on the terminal after control-c'ing - error in log (caused a "Xpra has stopped working" dialogue to pop up)
1339dmouseround3.txt (384.4 KB) - added by maxmylyn 3 months ago.
This time no clicks - just mouse movements and scrolling

Change History (43)

Changed 6 months ago by jdawgzim

Attachment: xpra_scroll_bug.zip added

Changed 5 months ago by Antoine Martin

Attachment: xp-two-screens.png added

two side by side screens with windows XP

Changed 5 months ago by Antoine Martin

Attachment: rotate-config.png added

the screen config as shown in the bug report screenshot

comment:1 Changed 5 months ago by Antoine Martin

Owner: changed from Antoine Martin to jdawgzim

Display data from the bug report tool for this Microsoft Windows 7 client:

{'screens': {
 0: {
    'name': '1\\WinSta0\\Default',
    'height_mm': 508, 'width_mm': 793,
    'settings': {}, 'resolution': -1, 'primary_monitor': 0,
    'height': 1920, 'width': 3000,
    'visual': {
        'rgb': {
            'visual_type': 'TRUE_COLOR', 'byte_order': 'LSB', 'depth': 24, 'bits_per_rgb': 42, 'colormap_size': 256
        },
        'system_visual': {
            'visual_type': 'TRUE_COLOR', 'byte_order': 'LSB', 'depth': 24, 'bits_per_rgb': 42, 'colormap_size': 256
        }
     },
     'root': (0, 0, 1920, 1200, 24),
     'monitors': 2,
     'monitor': {
         0: {
             'height_mm': 423, 'width_mm': 677, 'height': 1200, 'width': 1920, 'y': 120, 'x': 0, 'plug_name': '\\\\.\\DISPLAY1' \
         },
         1: {
             'height_mm': 677, 'width_mm': 381, 'height': 1920, 'width': 1080, 'y': 0, 'x': 1920, 'plug_name': '\\\\.\\DISPLAY2' \
            }
         }
     }
},
'device': {
    0: {
        '': 'Core Pointer', 'keys': [], 'num_axes': 2, 'axes': ((1, 0.0, 0.0), (2, 0.0, 0.0)), 'num_keys': 0, 'source': 'MOUSE', 'mode': 'SCREEN', 'has_cursor': True
    }
},
'maximal_cursor_size': (32, 32), 'pointer': (1623, 892),
'name': '1\\WinSta0\\Default',
'root': (0, 0, 1920, 1200, 24),
'root-size': (3000, 1920),
'devices': 1,
'default_cursor_size': 32L,
'pointer_is_grabbed': False,
'supports': {
    'clipboard_persistence': False, 'composite': False, 'cursor_color': True, 'shapes': True, 'selection_notification': True, 'cursor_alpha': True}
}

This seems to match the configuration in the screenshot you included:
the screen config as shown in the bug report screenshot

So the first monitor has a "y" offset of 120.
I believe that win32 and/or GTK may be using negative coordinates for the second monitor, and X11 will discard those.
When I tested it though, it worked without problems:
two  side by side screens with windows XP

So please try to run the client with the "-d mouse" flag and post the output.
We want to see what coordinates are reported in the mouse events for when the window is near the top of the second monitor.
(just in case, please also include the server log output so we can see the initial screen setup, and maybe also "xpra info")

Changed 5 months ago by jdawgzim

Attachment: xpra_scroll_bug_2.zip added

comment:2 Changed 5 months ago by jdawgzim

  • Submitted xpra_scroll_bug_2.zip​ while "-d mouse" was enabled.
  • I can now fix my issue by moving my portrait monitor to the left and then making it the primary monitor.

comment:3 Changed 5 months ago by Antoine Martin

@jdawgzim: you've attached the bug report data, we need to see the "xpra_cmd.exe" command output instead. (with the misbehaving configuration obviously)

comment:4 in reply to:  3 Changed 5 months ago by jdawgzim

Replying to antoine:
How do I do that?

comment:5 Changed 5 months ago by Antoine Martin

Run xpra from the command line, ie: open did command prompt then

cd ..
cd ..
cd Program Files
cd Xpra
Xpra_cmd.exe attach to:host:port -d mouse

(Paths may vary slightly)

Last edited 3 months ago by Antoine Martin (previous) (diff)

comment:6 Changed 5 months ago by jdawgzim

I use cmd.exe and run "Xpra_cmd.exe attach to:host:port -d mouse" and I get a lot of text. It doesn't let me copy the text. Is there a .log output somewhere or some way to capture this? Thank you for your patience with me

Changed 5 months ago by jdawgzim

Attachment: xpra_scroll_bug.jpg added

comment:7 Changed 5 months ago by alas

If you left click at the top left of the cmd.exe window, you should find an 'edit' option, which will allow you to select/enable copying. Then you'll just need to left-click to highlight the text you want to copy, then left click on it again to sync it with the clipboard.

Changed 5 months ago by jdawgzim

Attachment: xpra_scroll_bug.txt added

comment:8 Changed 5 months ago by Antoine Martin

A better and easier way of capturing log output, one that does not wrap the lines at 80 characters (which makes them so much harder to parse), is:

Xpra_cmd.exe attach to:host:port -d mouse > C:\xpra-log.txt 2>&1

Then you can just attach "C:\xpra-log.txt".


For my own reference, the code that converts the event's native coordinates is:

directly from the WM_MOUSEMOVE event message. As of r14323 we will now log both the root coordinates and plain (x,y).
The good news is that the win32 code gives us the raw values. It's quite possible that the window ends up using negative coordinates on win32, which then gets mapped partially offscreen on the server and is therefore unable to receive any events in that region.


@jdawgzim: the mouse in your short log sample is not anywhere near the top of the screen, the y coordinate is near ~1000 which is not where the problem occurs from what I understand:

do_motion_notify_event(<gtk.gdk.Event at 0577E590: GDK_MOTION_NOTIFY x=1684.00, y=1115.00>) \
    wid=9 / focus=9, device=Core Pointer, pointer=(2773, 1431), modifiers=['mod2'], buttons=[]

please try the latest client beta build. It includes a workaround for #1284 which may help, and the extra debug logging should also help narrow things down.
Please make sure to move the mouse in the problematic area, and keep the log sample as small as possible.
Also, it would be easier to parse the output if you add "--desktop-scaling=no" to the command line.
If the coordinates are negative, it may be useful to also have the "-d geometry" log output.
Finnaly, a screenshot showing where the mouse was when the log was capture would be ideal.

Last edited 4 months ago by Antoine Martin (previous) (diff)

comment:9 Changed 3 months ago by Antoine Martin

Owner: changed from jdawgzim to alas

Not heard back, @afarr: can you help with this?

comment:10 Changed 3 months ago by jdawgzim

Sorry, I worked around this issue by putting my vertical monitor on the left and switching the vertical monitor to be the primary monitor (start menu).

If I switch back to making the right horizontal monitor the primary monitor the issue returns. If I right or left click the mouse it is correct. It's just a "mouse over scroll wheel action" that is the offset the same amount as the two monitors are offset relative to where the primary monitor (start menu) is. It's like Xpra needs the start menu or primary monitor to be on the far left bottom.

I don't run the Xpra server where I use Xpra so I can't update it. I could update my client, but that probably wouldn't fix it. If I get brave enough in the future I could try to recreate this problem on my home computer and get you more info.

Last edited 3 months ago by jdawgzim (previous) (diff)

comment:11 Changed 3 months ago by maxmylyn

Retrying with a trunk r14619 Win8.1 client against a trunk r14712 Fedora 25 server and am only able to reproduce the scrolling offset. The mouse click actions behave nicely, but the scrolling does not. I'll attach a -d mouse log in a sec.

comment:12 Changed 3 months ago by maxmylyn

Owner: changed from alas to Antoine Martin

There you go, hand it back to me if you want any other logs.

Changed 3 months ago by maxmylyn

Attachment: 1339 Monitor Layout.png added

Here's a screenshot of the monitor layout I'm using - both are 1080p

comment:13 Changed 3 months ago by Antoine Martin

Owner: changed from Antoine Martin to maxmylyn

@maxmylin: the log file you attached doesn't have any mouse event debugging in it.

It should look like this with "-d mouse":

  • for clicks:
    _button_action(4, <gtk.gdk.Event at 0x7f99a240bb20: GDK_SCROLL x=253.00, y=121.00, direction=GDK_SCROLL_UP>, False) \
        wid=1 / focus=1 / window wid=1, device=Core Pointer, pointer=(303, 242), modifiers=['mod2'], buttons=[]
    
  • for movements:
    do_motion_notify_event(<gtk.gdk.Event at 0x7f99a240bb20: GDK_MOTION_NOTIFY x=253.00, y=122.00>) \
        wid=1 / focus=1 / window wid=1, device=Core Pointer, pointer=(303, 243), modifiers=['mod2'], buttons=[]
    

comment:14 Changed 3 months ago by maxmylyn

Owner: changed from maxmylyn to Antoine Martin

Oops It helps to remember to add -d mouse to the command line arguments. That would explain why the file was so small..

Anyways, steps are the same:

  • Connected
  • Clicked a few times
  • Scrolled a few times
  • Clicked some more
  • Scrolled some more

The clicks (left, middle, and right mouse) all registered in the correct place, but the scrolls were indeed offset.

Changed 3 months ago by maxmylyn

Attachment: 1339dmouse.txt added

did some scrolling and clicking - the scrolls were offset but the clicks were not.

comment:15 Changed 3 months ago by Antoine Martin

Owner: changed from Antoine Martin to maxmylyn

Does the problem go away if you run the client with XPRA_WHEEL=0?

Pointer events are adjusted for the second monitor (964 vs 638, 1339 vs 730):

do_motion_notify_event(<gtk.gdk.Event at 0482FE18: GDK_MOTION_NOTIFY x=638.00, y=730.00>) wid=4 / focus=4 / window wid=4, device=Core Pointer, pointer=(964, 1339), modifiers=['mod2'], buttons=[]

But our wheel event throttling code (#1131) doesn't know about the screen adjustments made by GTK.. and so we send the raw values, and those will be offset:

mousewheel: orientation=vertical distance=-120.0, units=-1, new value=-120.0, keys=0x0, x=964, y=849, client=gtk2.client, wid=4

If you confirm that this is indeed the problem, I'll try to find a solution.
Potential inelegant solutions I can think of:

  • keep the last position of the mouse as reported by GTK and use that value instead
  • calculate the GTK offset on every mouse move event and apply the same value to our cooked wheel events

comment:16 Changed 3 months ago by maxmylyn

Owner: changed from maxmylyn to Antoine Martin

Yeah, setting XPRA_WHEEL=0 seems to make the problem go away.

comment:17 Changed 3 months ago by Antoine Martin

Owner: changed from Antoine Martin to maxmylyn

Should be fixed in r14717 + r14718 (trunk), already backported to v1.0.x in r14719 as I am reasonably confident that the fix is correct or at least that it shouldn't make things worse.

comment:18 Changed 3 months ago by maxmylyn

Owner: changed from maxmylyn to Antoine Martin

Scrolling using the wheel seems to not work at all anymore, and instead seems to freeze the whole client. Some aggressive scrolling attempts caused the whole server to crash on me - I'll attach the serverside logs from that attempt (includes a traceback and an error I was seeing on the client side) and also another attempt with the client side -d mouse logs.

Changed 3 months ago by maxmylyn

Attachment: 1399crash.log added

some aggressive scrolling crashed the server

Changed 3 months ago by maxmylyn

Attachment: 1399newdmouselog.txt added

same as before, but the client hung on the terminal after control-c'ing - error in log (caused a "Xpra has stopped working" dialogue to pop up)

comment:19 Changed 3 months ago by Antoine Martin

Owner: changed from Antoine Martin to maxmylyn

@maxmylin: which version were you testing with? My bet is on 2.0

I believe that the bug you are reporting is caused by #678 in version 2.0 (2.0 is going to be in a state of flux as development has just started) but this should not be affecting version 1.x or older - which I was very worried about.

r14735 should fix this particular issue in 2.0 for now - but this fix is likely to be in the "wrong" place: we should be sending signed 16-bit values from the wnd proc callbacks, pywin32 was giving us signed integers but ctypes isn't (or at least not yet). Other things are likely to misbehave in 2.0 until that is fixed. Anyway, new beta 2.0 build uploaded too.

Please confirm that the bug was with 2.0 only, and that 1.0 is fixed.
(2.0 should also be fixed, but that's less of a concern at this point)

comment:20 Changed 3 months ago by maxmylyn

@antoine - whoops sorry, should have mentioned trunk

Can you post new 1.0 builds?

Edit: nevermind - I forgot there's also /dists/

Last edited 3 months ago by maxmylyn (previous) (diff)

comment:21 Changed 3 months ago by Antoine Martin

Can you post new 1.0 builds?


1.0.1 was released with this fixed included, see: http://xpra.org/dists/windows/

comment:22 Changed 3 months ago by maxmylyn

Owner: changed from maxmylyn to Antoine Martin

Sorry was busy yesterday and didn't get to this ticket.

Using the 1.0.1 r14723 Client against a Fedora 25 r14751 1.X server I've found the bug hasn't been fixed - the mouse wheel events are still being sent to the wrong area as described in the description.

edit: I'll attach another log of just scrolling and mouse movements.

Last edited 3 months ago by maxmylyn (previous) (diff)

Changed 3 months ago by maxmylyn

Attachment: 1339dmouseround3.txt added

This time no clicks - just mouse movements and scrolling

comment:23 Changed 3 months ago by Antoine Martin

Owner: changed from Antoine Martin to maxmylyn

Oh, so there were actually two bugs then.
The second may be fixed by r14754.
The "-d mouse" output should now include the "gtk_offset" value which is applied to raw win32 wheel events to adjust for the gtk coordinate system.
(it's always (0,0) on my test system.. but if I got my maths the right way up, it should cancel out the offset)
New beta 2.0 win32 build uploaded. (with all the other known issues..)

comment:24 Changed 6 weeks ago by Antoine Martin

Priority: majorcritical

raising - we should include this in the 1.0 branch if the fix is confirmed

comment:25 Changed 6 weeks ago by maxmylyn

Using the latest 1.X build (r15032) with a Win8.1 machine against a Fedora 25 trunk r15094 server:

  • Scrolling no longer works whatsoever
    • Regular mouse events work fine (moving, clicking)

Sample scroll output from scrolling:

2017-02-16 10:17:57,732 mousewheel: orientation=vertical distance=-120.0, units=1, new value=-120.0, keys=0x0, x=64676,
y=806, client=gtk2.client, wid=4
2017-02-16 10:17:57,733 mousewheel: sent 1 wheel events to the server for distance=-120, remainder=0
2017-02-16 10:17:57,740 mousewheel: orientation=vertical distance=-120.0, units=1, new value=-120.0, keys=0x0, x=64676,
y=806, client=gtk2.client, wid=4
2017-02-16 10:17:57,740 mousewheel: sent 1 wheel events to the server for distance=-120, remainder=0
2017-02-16 10:17:57,756 mousewheel: orientation=vertical distance=-120.0, units=1, new value=-120.0, keys=0x0, x=64676,
y=806, client=gtk2.client, wid=4
2017-02-16 10:17:57,756 mousewheel: sent 1 wheel events to the server for distance=-120, remainder=0
2017-02-16 10:17:57,763 mousewheel: orientation=vertical distance=-120.0, units=1, new value=-120.0, keys=0x0, x=64676,
y=806, client=gtk2.client, wid=4
2017-02-16 10:17:57,765 mousewheel: sent 1 wheel events to the server for distance=-120, remainder=0
2017-02-16 10:17:58,092 mousewheel: orientation=vertical distance=-120.0, units=1, new value=-120.0, keys=0x0, x=64676,
y=806, client=gtk2.client, wid=4
2017-02-16 10:17:58,092 mousewheel: sent 1 wheel events to the server for distance=-120, remainder=0
2017-02-16 10:17:58,099 mousewheel: orientation=vertical distance=-120.0, units=1, new value=-120.0, keys=0x0, x=64676,
y=806, client=gtk2.client, wid=4
2017-02-16 10:17:58,101 mousewheel: sent 1 wheel events to the server for distance=-120, remainder=0
2017-02-16 10:17:58,108 mousewheel: orientation=vertical distance=-120.0, units=1, new value=-120.0, keys=0x0, x=64676,
y=806, client=gtk2.client, wid=4
2017-02-16 10:17:58,108 mousewheel: sent 1 wheel events to the server for distance=-120, remainder=0
2017-02-16 10:17:58,115 mousewheel: orientation=vertical distance=-120.0, units=1, new value=-120.0, keys=0x0, x=64676,
y=806, client=gtk2.client, wid=4
2017-02-16 10:17:58,117 mousewheel: sent 1 wheel events to the server for distance=-120, remainder=0
2017-02-16 10:17:58,124 mousewheel: orientation=vertical distance=-240.0, units=2, new value=-240.0, keys=0x0, x=64676,
Last edited 6 weeks ago by maxmylyn (previous) (diff)

comment:26 Changed 6 weeks ago by maxmylyn

The 2.X r15030 client works fine.

comment:27 Changed 6 weeks ago by Antoine Martin

Priority: criticalblocker

The fix to test for this multi monitor bug is in the 2.0 branch.

If wheel events no longer work at all in the 1.x branch then this is serious issue - so raising to blocker. That's surprising seeing that your log samples indicate that the wheel events have been sent to the server.

And I have just tested 1.0.3 and I see the events:

  • client side with "-d mouse"
  • server side with "-d mouse"
  • applications respond
  • xev shows the button events (buttons 4 and 5)

comment:28 Changed 6 weeks ago by maxmylyn

NOTE:

The 1.0.X client only stops scrolling if there more than monitor attached to the client machine - after disconnecting the second monitor, scrolling works again.

comment:29 Changed 6 weeks ago by maxmylyn

Also:

Setting XPRA_WHEEL=0 gets scrolling to work again with both monitors plugged in.

Last edited 6 weeks ago by maxmylyn (previous) (diff)

comment:30 Changed 6 weeks ago by Antoine Martin

Priority: blockermajor

Right, so the 1.0.x client is still affected by this bug - which is totally expected since the fix wasn't in there yet.
I was thinking that something broke in 1.x - stand down red alert.

I have uploaded a beta 1.0.4 with this change backported (r15100 + r15104, backport was made more difficult by the switch to ctypes #678): http://xpra.org/beta/windows.

comment:31 Changed 5 weeks ago by maxmylyn

Latest 1.0.X build (r15129) fixed it - connecting against a 2.0 r15129 Fedora 25 server scrolling works fine with multimonitor in a number of configurations.

I'll hold on to this for a few hours to double check nothing else broke, and then close it if everything checks out.

comment:32 Changed 5 weeks ago by maxmylyn

Resolution: fixed
Status: newclosed

Yep, everything looks good - closing.

Note: See TracTickets for help on using tickets.