Xpra: Ticket #41: wishlist: Support multiple clients connected at the same time

aka: session shadowing?

ref: http://code.google.com/p/partiwm/issues/detail?id=42

It would be useful to support multiple clients connected at the same time. This could allow window sharing between multiple desktops (perhaps with sync'ed location? - so that synergy can 'move' windows between desktops)

Tue, 15 Nov 2011 15:43:30 GMT - Antoine Martin: priority changed; milestone set

I don't understand the link with synergy...

Cloning would be possible I think, but probably quite a lot of work to get right. (concurrent multi-user is even harder - read-only clone would be the first step)

For example: handling constantly moving windows around whenever the controlling client decides to move a window would be tricky (think different screen sizes, different positions forced by each client's window manager, etc)

Then there's the problem of managing redraws and requests for window pixel data: probably would have to send all pixel data to both clients irrespective of which client requested it (since this process is asynchronous)

Tue, 15 Nov 2011 16:27:21 GMT - Norman Rasmussen:

read-only clone would be a great start, read-write (maybe only one copy can write at a time), would be nice too.

Basically I'd like to simulate a multi-monitor set up, by being able to drag a window from one X server to another, at the moment I can xpra attach on the 2nd server, and all windows disappear on the first.

Tue, 20 Dec 2011 10:19:02 GMT - Antoine Martin: owner, status changed

Another useful feature discussed on IRC would be the ability to have different screen update rates for each client so that a slow client does not slow down everyone: it would drop most screen updates and just get a full screen refresh at the appropriate rate.

Sat, 26 May 2012 22:43:56 GMT - Norman Rasmussen:

I spent a few hours on this today, and the code in XpraServer is really tangled. Some of it is for the single server instance, and some is for the currently connected client. The first step is to untangle these two sets of logic. Do we need a new ServerConnection class, that would have ._protocol and ._server_source? or can we move this per-connection code into ServerSource?

As a crazy work-around, could two copies of xpra start be run connecting to the save xvfb?

Sun, 27 May 2012 14:48:23 GMT - Antoine Martin:

Totally agree on the need to entangle the server from the connection stuff. Even if we don't support multiple connections, this will make the code more maintainable and easier to read.

The crazy workaround would not work: xpra is the compositing window manager and there can only be one at a time...

Mon, 27 Aug 2012 18:16:14 GMT - Antoine Martin:

Closed duplicate ticket #169 (my bad), there has been a lot of progress in this area (especially since r1407).

I have managed to connect multiple clients and see something on screen... it's a start.

What is left to do:

Tue, 28 Aug 2012 05:56:32 GMT - Antoine Martin:

Wed, 29 Aug 2012 03:05:02 GMT - Antoine Martin:

I've posted some pretty graphs comparing performance before and after this patch to make sure it does not adversely affect it, you can find them here

Wed, 29 Aug 2012 15:19:33 GMT - Antoine Martin:

r1434 fixes clipboard: only the first client to request it will get it (doing more than this will be quite difficult and race prone)

Wed, 29 Aug 2012 19:31:18 GMT - Norman Rasmussen:

This is looking great!

For moving windows (and clipboard), why not keep track of 'last client to send keystroke/mouse click' and use that as the primary client. I think you use this already for tracking mouse movements (I did a quick test with xeyes). This would allow someone to use two clients at almost the same time, and generally "the right thing" would just happen.

btw, when you're syncing window position, it would be nice to be able to specify an offset on the client side. So I can run client on workstation & laptop, and window position is sync'ed but windows only appear on one display at a time, eg workstation at +0+0, and laptop at -1440+0.

Thu, 30 Aug 2012 10:36:52 GMT - Antoine Martin:

the clipboard would be very difficult to manage with more than one client (it is already difficult enough as it is!).

I'm not sure I understand the concept of "last client to send keystroke/mouse click": what would I use this for? What is the current behaviour that needs fixing?

As for syncing window position: this still needs doing... and we should *not* allow windows to be placed in different locations on each client, as this will lead to bugs: many applications and toolkits rely on knowing where the windows are for placing new windows/widgets/etc, if we lie about it, the results won't be good! That said, since each client may well decide to place windows in slightly different locations (window snap to grid, custom rules, etc..), we may already have a problem on our hands! (and finding a solution that is not racy will be difficult!)

Thu, 30 Aug 2012 14:04:05 GMT - Norman Rasmussen:

My idea of "last client to send keystroke/mouse click" is to help out with cases where multiple clients send conflicting information like: who 'owns' the clipboard, where do I position this window, etc.

The current clipboard implementation means that only one client will have clipboard interaction, where-as if we implement a dynamic primary client (using last user-interacting event), then we have a easy way to tell which client the user is currently interacting with.

Doing this dynamically would allow the user to physically migrate between clients and have a first class experience at all of them.

btw, re: window positioning: I meant a client-side option that would be applied to all windows the same. Windows would still be in the same place relative to each other.

Thu, 30 Aug 2012 15:13:40 GMT - Antoine Martin:

Hah, gotcha, both very good ideas! I'll see what I can do, but this is all lower priority than fixing the obvious bugs, like: try to minimize the window on one client... still shown on the other (but unusable), etc. Feel free to post changes. Note: the way the clipboard stuff is hooked at present is a bit nasty (just wanted to get something working).

Mon, 24 Sep 2012 23:31:22 GMT - Jon Griffiths:

I've been using this since you first released it, and it's working really nicely for me: thank you.

Before you did (or announced anyway) any more towards the window placement synchronisation issue, I thought I'd say that for my use case, I hardly ever notice any issue from it. Furthermore, the behaviour as it exists now: allowing different clients to position windows individually, is very useful. My clients have hugely varying screen sizes and layouts, and it'd be unfortunate if every client had to scrunch all the shared xpra windows in to the top-left corner or such to respect the smallest connected client.

In fact, without the window position syncing, all the GTK apps (and some old xlib only ones) I've used have been fine and have responded to mouse clicks in the right places etc. If I move a window on one client, it keeps responding properly on all the others (the issue you described with minimising aside).

The only problem I've seen is with initial window placement. One of my clients has a gnome top panel that extends to the top-left corner of the display, and all the rest don't (because they have multiple screens, and the panels happen to not be on the left-most screens). Initial placement of new windows is always in the top-left of each client, but obeying window manager rules to not overlap the panel. Every new window on the clients without the panel respond to mouse clicks in a position offset by the depth of the panel on the one client that has it, until they're moved for the first time. Presumably after the initial placement in the top-left, the panel'd client's window manager moves the window to avoid the panel, and that's what creates the issue?

I can elaborate further if you'd like to consider fixing this particular case separately to a more general solution; I'm sorry I'm not able to find my way around the code well enough to address it myself (I tried somewhat, and I haven't entirely given up yet). I wondered if each client simulating moving a window to its client-defined current position each time it wants to send any keyboard/mouse events would be enough to fix it? Maybe that's exactly the same as Norman was suggesting?

Anyway, mostly I was just wondering if you'd consider making the window position syncing optional if and when you find time to implement it please?

Sun, 10 Feb 2013 07:15:14 GMT - Antoine Martin:

New issue: maybe system tray forwarding should be disabled for secondary clients. r2690 fixes system-tray compatibility bugs when clients not supporting the feature, or those that disable it with --no-system-tray, connect to a server which has trays. Problem is that this does not take multi-client into account at all...

In dealing with hidden windows, we can deal with #47

Wed, 05 Feb 2014 13:49:57 GMT - Antoine Martin:

I was looking at this code for unrelated problems, and thought: instead of keeping a "shown" flag in the DesktopManager, this could be a WeakSet (Python 2.7 only, sigh) of all the ServerSources on which the window is shown. Or maybe we should have a manager per source which could keep track of window state independently: iconic, actual workspace (see #774), etc.. When we get the "contents-changed" signal, we can call damage only on the active sources. Same when deciding if the window is shown or not: take into account who is asking. More difficult is the ownership_election code which parks the window on the root window when not visible? Could create some focus problems I think.. See also r8307

You are right about what needs to be done to handle client events properly: we need to keep where the window got mapped to on each client (almost like having a DesktopManager per client), and force this before processing events. This will not play well with #212..

Another relatively easy fix is for transient-for windows: instead of sending plain coordinates for the new window, we can send them relative to the parent window.

New problems: found a race in acknowledge_changes, we call it multiple times (one for each source), and the later ones may cause the earlier ones to miss some screen updates. This should be a one-off function only, fired by whichever WindowSource happens to process the damage data first.

Note: some recent changes may help a bit here (testing needed), see #765

Mon, 13 Jul 2015 05:31:03 GMT - rektide: cc set

Wed, 12 Aug 2015 02:36:52 GMT - Antoine Martin:

See also #943 and #990.

Fri, 06 Nov 2015 00:29:17 GMT - J. Max Mena:

Have done some playing around with Sharing on a r11145 Fedora Client(1080p screen), a r11122 Win8.1 Client(1440p screen), and a r11145 Fedora 21 Server(...no physical screen):

I've found the server will move and resize the windows according to whichever client moved the window last. (I've also found switching desktops on the fedora client triggers this behavior for all windows) This does some weird things, as the server and client do not track mouse properly. Meaning, mouse clicks can go in all the wrong spots. I've found that the client will send the server mouse inputs (clicks, dragging, etc) from where the mouse is located on that client, without knowing that the window (on the server) is actually somewhere else; as the inactive client doesn't bother to move or adjust the window frames, just the paints within each frame. I chronicled this with a series of screenshots.

Firstly, I opened Firefox, and then minimized it on the Fedora machine while keeping it up on the Windows machine. Even while minimized, it covered up an Xterm I had open(on the server), meaning that right clicking on the xterm sent the right click to Firefox

Secondly, I closed Firefox and did some playing around with Gedit, leaving the Windows client with the 'proper' window size and location. To sum up my experiments, I've found that Xpra doesn't offset the mouse location per client and as such, the clicks on the machine without the 'proper' window size and location will not be able to use the mouse correctly as it'll always be slightly offset.

There'll probably need to be some logic figured out as to when each client gets the 'proper' window location and size....and if we'll want to move windows involuntarily. (might be easier than trying to keep track of window locations and offsetting mouse inputs and outputs; or it might not be)

Mon, 09 Nov 2015 17:25:00 GMT - Antoine Martin: milestone changed

I've found the server will move and resize the windows according to whichever client moved the window last.

That's a known issue, #990 did some preparatory work on this: we will need to keep track of who "owns" the window, and either:

Or maybe a mixture of the two: until a client focuses a window or clicks in it, maybe we should not bother sending mouse hover events?

The main difficulty is that the code only has a single "desktop manager" for all clients, and we will need to have one per client connection... (and deal with minimizing, etc)

Tue, 12 Jul 2016 16:51:50 GMT - Antoine Martin: milestone changed

Milestone renamed

Sun, 21 Aug 2016 09:55:49 GMT - Antoine Martin: milestone changed

Milestone renamed

Mon, 12 Sep 2016 11:15:07 GMT - Antoine Martin:

See also r13669: we now have an offset client-side for OR windows in some cases..

Thu, 15 Sep 2016 16:14:42 GMT - Norman Rasmussen:

Cool. Will that also eventually fix the issue with two clients connected, and "pointer events don't line up until the window is moved". Currently the last client to move the window is the canonical source of window location, so pointer events on the other window arrive at the wrong location.

Tue, 01 Nov 2016 04:40:22 GMT - Antoine Martin:

Still TODO, some or all of:

Wed, 02 Nov 2016 06:24:42 GMT - Antoine Martin: status changed

Wed, 02 Nov 2016 06:25:19 GMT - Antoine Martin: owner changed

@norman: does that work for you?

Wed, 02 Nov 2016 16:56:03 GMT - Norman Rasmussen:

I'm running Trusty still :-(, so can't test immediately, but that's looks like it addresses my concerns from #comment:24

I think the last would-be-nice-to-have (from #comment:10) would be a way to sync window position between clients, and be able to specify an offset at the client. This would make it possible to simulate having two monitors connected to the same computer; when in fact they're two different machines, with xpra syncing the windows. (i.e. When you move a window on the one monitor towards the other, it would appear to move between the two monitors.)

Wed, 09 Nov 2016 08:02:42 GMT - Antoine Martin:

Minor fixes in r14400.

@norman: there are beta packages with those changes. (those packages will be called "release candidate" within a few days)

Sat, 19 Nov 2016 10:26:40 GMT - Antoine Martin: owner changed

Follow up in #1368. Related fix in r14460 for #1358 - which looks like it could have been caused by these changes. r14458 + r14461 makes it possible to toggle some settings via dbus or "xpra control", ie: to enable sharing after starting the server:

xpra control :100 toggle sharing on

@afarr: not heard back from @norman, so feel free to just close this one. (though, I won't complain if you do test it!)

Sat, 19 Nov 2016 13:58:50 GMT - Norman Rasmussen:

I tried the beta package, but not with two clients at the same time yet. The only pending enhancement from my side is position sync (#comment:28), which could be tracked as a new ticket if you want.

Sat, 19 Nov 2016 14:34:23 GMT - Antoine Martin:

@norman: yes please, "position sync" definitely won't make the cut for v1.0: this will require a more changes to actually work (ie: the virtual screen size needs to be big enough for both screens, etc).

Mon, 21 Nov 2016 22:27:43 GMT - Norman Rasmussen:

Split out to #1369

Mon, 21 Nov 2016 22:46:56 GMT - Norman Rasmussen:

Brief testing shows that the new position sync code works well. Tried moving windows around independently and there were no issues. ui-driver works okay, having an extra click before interact means initial mouse-over doesn't work, but I saw no issues with that.

I have seen a regression with gvim where the window tries to size 1 line high. I assume that there's an issue with the height feedback.

Also one of my client's didn't detect window resize events correctly (the one using loopback & has opengl enabled). I just used an xeyes window to test.

Mon, 21 Nov 2016 22:49:01 GMT - Norman Rasmussen:

gvim height issues may be related to DPI scaling, as it only occurs when the last connected client has a DPI that causes scaling.

Mon, 21 Nov 2016 22:51:22 GMT - Norman Rasmussen:

The 'remote cursor' cross-hair was only visible on one client (loopback w/opengl). Not sure if it's not displaying or the other client is not sending position correctly. (All this testing was with v1.0-r14449)

Tue, 22 Nov 2016 09:06:54 GMT - Antoine Martin: attachment set

try to catch the hidden motion events

Tue, 22 Nov 2016 09:16:05 GMT - Antoine Martin: owner, status changed

Also one of my client's didn't detect window resize events correctly (the one using loopback & has opengl enabled). I just used an xeyes window to test.

What is loopback? xeyes is notoriously difficult to handle - it doesn't really behave like a normal application. (so not a huge priority for us)

The 'remote cursor' cross-hair was only visible on one client (loopback w/opengl).

I've added the 'remote cursor' to the non-opengl backend in r14468. You're probably running on an Intel chipset (greylisted) or a GPU that doesn't support enough opengl at all.

Taking this ticket back as I am seeing an issue with resized windows: we should forward the resizing to the other clients... but it seems to go MIA. Please file the gvim issue separately unless it is triggered by sharing. I'm not seeing anything wrong with it (apart from the resizing..)

PS: it seems that Firefox intercepts the mouse motion events by placing another input-only window on top of its window... (fixing that would be very very very hard - the lost-motion patch above tried all sorts of things, no luck so far)

Thu, 24 Nov 2016 14:25:40 GMT - Antoine Martin: owner, status changed

The resizing is fixed in r14486.

It's not perfect by any means, but it is definitely a lot more usable than it has ever been. So this will do for this release.

@afarr: feel free to test or just close.

Mon, 28 Nov 2016 21:04:22 GMT - J. Max Mena: owner changed

Alright, doing some testing with a Fedora 24 trunk r14493 server, and a Fedora 24 trunk r14493 client, and a Windows trunk r14492 client:

Tue, 29 Nov 2016 07:08:06 GMT - Antoine Martin: status, milestone changed; resolution set

This will do for this release, will follow up in #1373, #1368, #972, #1716

Sat, 23 Jan 2021 04:43:56 GMT - migration script:

this ticket has been moved to: https://github.com/Xpra-org/xpra/issues/41