Xpra: Ticket #670: Subpar performance when running gvim or emacs

Reproduction steps:

where executable might be gvim or emacs when configured for GUI use

I've talked to totaam on #winswitch and we agreed this looks like a real problem, so he recommended me to open this ticket here.

Fri, 05 Sep 2014 02:53:58 GMT - Antoine Martin: owner, status, description changed

Fri, 05 Sep 2014 05:14:15 GMT - Antoine Martin: description changed

On a 1080p screen with a gvim window taking up half the screen, with a fairly large file open, a single click on the scrollbar causes 320 damage events! All but 5 of those repaints are exactly 1 line in height (15 pixels high), many are one or two characters wide.. The 5 odd ones do contain an almost-full window repaint, right at the beginning. 69 repaints are a single character wide (8x15)

What a really odd way of painting the screen!

So the good news is that I have a workaround:

XPRA_FORCE_BATCH=1 xpra start ...

Will start the server with force-batch mode, which means that even when things are going well (no pixel backlog or network congestion) we will wait before encoding the pixels, which allows us to batch them up together. On my system, it makes things usable again.

cesarkawakami: does that improve things for you?

As for a more proper fix for this: I'm not too sure how to go about it. Should we always enable batching by default? (vnc does) We would lose a tiny bit of latency in some cases (well behaved applications) just to workaround applications with pathological behaviour...

Another approach would be to enable batching when we detect a damage storm like this one. Which is probably the better option, but may not be suitable for a 0.14.x backport.

Fri, 05 Sep 2014 05:39:15 GMT - Antoine Martin: owner, status changed

The slightly better solution implemented in r7516 is to turn on damage batching if we find more than 5 damage events in the last 10 milliseconds. The downside is that those 5 events will still get sent unbatched before the rest, but at least we don't penalize well behaved application too much (well, doing the checking has its own cost... but hopefully not too bad)

cesarkawakami: do both solutions work for you? Any other ideas?

FYI: using other encodings, even video encodings, seems to work just as well: the small updates get sent as rgb anyway.

Note: emacs just sitting idle will repaint the cursor even when it isn't focused or blinking, causing 4 damage events as it repaints each line around the hollow cursor individually!

Fri, 05 Sep 2014 17:41:58 GMT - cesarkawakami2:

[For some reason I can neither login with my previous account nor get my username/email combination accepted in the "reset password" screen. Weird.]

Unfortunately, I just now noticed my xpra server was already batching updates, so none of the workarounds above work in my case.

Sat, 06 Sep 2014 04:04:55 GMT - Antoine Martin: owner changed

For some reason I can neither login with my previous account..

Sorry about that, there were many spam accounts created yesterday, and I must have deleted your account by mistake.

my xpra server was already batching updates, so none of the workarounds above work in my case

That's odd, they make a very noticeable difference on my test setup. How did you verify this? Can you post xpra info of when things are bad and the server's -d encoding log?

Mon, 08 Sep 2014 03:38:39 GMT - Antoine Martin:

Many problems identified:

That's twice as much time figuring out what to do (rectangles calculations) than actually doing it! (encode loop)

Mon, 08 Sep 2014 17:15:12 GMT - cesarkawakami2: attachment set

-d encoding, large screen

Mon, 08 Sep 2014 17:30:44 GMT - cesarkawakami2:

With large screen, the rectangles computation issue is more pronounced:

Basic info:
    Sampling interval: 0.010 s
    Number of samples: 8208
    Overhead per sample: 0.0001 s
    Overhead percentage: 1.3%
    Thread IDs:
    Total CPU time in samples: 14.4418 s
|- ./xpra-profile:66:<module> (10.99, 76.1%)
| \- /usr/lib/python2.7/dist-packages/xpra/scripts/main.py:103:main (10.99, 76.1%)
|   \- /usr/lib/python2.7/dist-packages/xpra/scripts/main.py:761:run_mode (10.99, 76.1%)
|     \- /usr/lib/python2.7/dist-packages/xpra/scripts/server.py:997:run_server (10.99, 76.1%)
|       \- /usr/lib/python2.7/dist-packages/xpra/server/server_base.py:403:run (10.99, 76.1%)
|         \- /usr/lib/python2.7/dist-packages/xpra/server/server_core.py:290:run (10.99, 76.1%)
|           \- /usr/lib/python2.7/dist-packages/xpra/server/gtk_server_base.py:51:do_run (10.99, 76.1%)
|             \- /usr/lib/python2.7/dist-packages/xpra/gtk_common/gtk2common.py:15:gtk2main (10.99, 76.1%)
|               |- /usr/lib/python2.7/dist-packages/xpra/x11/gtk_x11/composite.py:229:do_xpra_damage_event (6.89, 47.7%)
|               | \- /usr/lib/python2.7/dist-packages/xpra/x11/gtk_x11/window.py:395:_forward_contents_changed (6.56, 45.4%)
|               |   \- /usr/lib/python2.7/dist-packages/xpra/x11/server.py:614:_contents_changed (6.18, 42.8%)
|               |     \- /usr/lib/python2.7/dist-packages/xpra/server/server_base.py:1625:_damage (6.13, 42.5%)
|               |       \- /usr/lib/python2.7/dist-packages/xpra/server/source.py:1557:damage (6.13, 42.5%)
|               |         \- /usr/lib/python2.7/dist-packages/xpra/server/window_source.py:671:damage (5.96, 41.3%)
|               |           \- /usr/lib/python2.7/dist-packages/xpra/server/region.py:157:add_rectangle (5.83, 40.4%)
|               |             \- /usr/lib/python2.7/dist-packages/xpra/server/region.py:60:intersects_rect (4.98, 34.5%)
|               |               |- /usr/lib/python2.7/dist-packages/xpra/server/region.py:49:intersects (1.32, 9.2%)
|               |               \- /usr/lib/python2.7/dist-packages/xpra/server/region.py:53:intersects (0.83, 5.8%)
|               |- /usr/lib/python2.7/dist-packages/xpra/x11/gtk_x11/composite.py:226:do_xpra_damage_event (1.21, 8.4%)
|               \- /usr/lib/python2.7/dist-packages/xpra/log.py:160:__call__ (0.84, 5.8%)
\- /usr/lib/python2.7/threading.py:783:__bootstrap (3.46, 23.9%)
  \- /usr/lib/python2.7/threading.py:810:__bootstrap_inner (3.46, 23.9%)
    \- /usr/lib/python2.7/threading.py:763:run (3.42, 23.7%)
      \- /usr/lib/python2.7/dist-packages/xpra/server/source.py:1615:encode_loop (2.91, 20.1%)
        \- /usr/lib/python2.7/dist-packages/xpra/server/window_source.py:1057:make_data_packet_cb (2.42, 16.7%)
          \- /usr/lib/python2.7/dist-packages/xpra/server/window_source.py:1333:make_data_packet (2.33, 16.1%)
            \- /usr/lib/python2.7/dist-packages/xpra/server/window_source.py:1344:rgb_encode (2.33, 16.1%)
              \- /usr/lib/python2.7/dist-packages/xpra/server/picture_encode.py:164:rgb_encode (2.33, 16.1%)
                \- /usr/lib/python2.7/dist-packages/xpra/net/compression.py:195:compressed_wrapper (0.82, 5.7%)

Attached, the {-d encoding} log of my server.

Also, I've extracted a damage log from output.log and updated test_region.py with it. It seems that even though we have less rectangles, add_rectangle is having a harder time with them:

$ python test_region.py  # the original
created 365 rectangles in 0.23ms
add_rectangle 365 rectangles in 1.09ms
created 142 rectangles in 0.09ms
add_rectangle 142 rectangles in 0.50ms
$ python test_region_2.py  # modified with the new damage log from the larger screen
created 135 rectangles in 0.08ms
add_rectangle 135 rectangles in 2.02ms


Mon, 08 Sep 2014 17:33:13 GMT - cesarkawakami2: attachment set

xpra info

Mon, 08 Sep 2014 17:33:30 GMT - cesarkawakami2:

I've also attached xpra info for this session.

Mon, 08 Sep 2014 18:52:23 GMT - Antoine Martin:

Can you please try to see how much difference the patch in ticket:620#comment:18 makes? I haven't merged it because it didn't make enough of a difference.. but I've been proven wrong already.

Tue, 09 Sep 2014 16:51:26 GMT - Antoine Martin:

The patch breaks "video regions" (which needs to modify the region), and cesarkawakami also reported "random spinfreezes"... so I think we should forget about this one. It's just not worth the trouble.

Fri, 26 Dec 2014 08:10:14 GMT - Antoine Martin:


Tue, 06 Jan 2015 06:47:53 GMT - Antoine Martin: attachment set

shows gvim repainting using rgb24 (orange colour)

Tue, 06 Jan 2015 06:50:19 GMT - Antoine Martin: owner changed

Thanks to the work on #760, I fixed a bug in the damage region merging code (fixed in r8374). This is included in the upcoming 0.14.16

The repaints now look like this: shows gvim repainting using rgb24 (orange colour)

If you can reproduce the problem, please include a screenshot using:

XPRA_OPENGL_PAINT_BOX=1 xpra attach ...

As per #760. This may show us what is going on in more detail.

Sat, 24 Jan 2015 07:23:38 GMT - Antoine Martin: status changed; resolution set

Not heard back, I assume this is fixed. Closing.

Sat, 23 Jan 2021 05:02:31 GMT - migration script:

this ticket has been moved to: https://github.com/Xpra-org/xpra/issues/670