Xpra: Ticket #23: fast refresh (ie: fast scrolling) causes high cpu usage and no screen updates

Not sure why the screen does not update (I don't get that behaviour here), but in any case we need to deal with fast updates better.

Maybe we can detect when a window is causing bursts of damage requests and buffer them for a bit, by the time we get around to processing them we may be able to remove many duplicates for the same region, or if the total surface of the damage requests is close to the full window size then just do one full refresh instead.

The damage requests from the server end come via _contents_changed in server.py (from CompositeHelper and BaseWindowModel.setup()). Currently this fires _protocol.source_has_more() which will write the packet if the write queue is empty. Could it be that very large packets make the write queue appear empty (it is) when in fact there is still a large chunk of data still to be sent in _write_thread_loop? Or do the packets go out faster than client can deal with?

More testing needed. Ideas and suggestions welcome.



Fri, 23 Sep 2011 14:34:40 GMT - Antoine Martin: status changed; resolution set

r179 deals with this, we may improve on that and automatically adjust the batch delay based on better heuristics than just the number of damage requests but this will do for now. closing.


Mon, 21 Nov 2011 09:23:54 GMT - Timo Juhani Lindfors:

while true; do

echo "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"

done

still causes xpra svn 266 client to spend all its cpu time and not update any windows. Sending ctrl-c is also not possible.


Mon, 21 Nov 2011 09:27:40 GMT - Antoine Martin: status changed; resolution deleted


Mon, 21 Nov 2011 12:36:57 GMT - Timo Juhani Lindfors:

Both client and server are relatively fast x86 systems connected over 100 Mbps ethernet. I'm simply using

xpra start :7

and

xpra attach ssh:server:7


Mon, 21 Nov 2011 20:48:36 GMT - Antoine Martin: attachment set

better batching: send whole screen to save rectangles and speedup/slow down according to client consumption rate


Mon, 21 Nov 2011 20:49:16 GMT - Antoine Martin: owner, status changed

Can you please try the attached patch and most the server log output if you still do get lag.


Wed, 23 Nov 2011 07:38:51 GMT - Timo Juhani Lindfors:

The patch does not apply to svn 266 or svn 279.

$ patch -p1 < adaptive-damage.patch
patching file server.py
Hunk #1 FAILED at 131.
Hunk #2 FAILED at 141.
Hunk #3 FAILED at 166.
Hunk #4 FAILED at 240.
Hunk #5 FAILED at 287.
Hunk #6 FAILED at 397.
Hunk #7 FAILED at 1069.
Hunk #8 FAILED at 1083.
Hunk #9 FAILED at 1154.
Hunk #10 FAILED at 1244.
Hunk #11 FAILED at 1362.
Hunk #12 FAILED at 1419.
12 out of 12 hunks FAILED -- saving rejects to file server.py.rej
patching file client.py
Hunk #1 FAILED at 360.
Hunk #2 FAILED at 651.
Hunk #3 FAILED at 741.
Hunk #4 FAILED at 781.
4 out of 4 hunks FAILED -- saving rejects to file client.py.rej

Wed, 23 Nov 2011 07:41:42 GMT - Timo Juhani Lindfors:

Oh well, it does apply. I just have to use

cd src && patch -p0 < ../adaptive-damage.patch

instead of the usualy

patch -p1 < adaptive-damage.patch


Wed, 23 Nov 2011 07:54:47 GMT - Timo Juhani Lindfors:

After I apply this patch to svn 279 I see no updates to windows at all. Server prints

lindi1:~$ xpra start --no-daemon --use-display :7
found /home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty
Xlib:  extension "RANDR" missing on display ":7.0".
Xlib:  extension "RANDR" missing on display ":7.0".
Xlib:  extension "RANDR" missing on display ":7.0".
Randr not supported: X server does not support required extension Randr
randr enabled: False
using notification forwarder
xpra is ready.
New connection received
Handshake complete; enabling connection
client resolution is [1280, 1024], current server resolution is 3840x2560
setting key repeat rate from client: 500 / 33
guessing keyboard layout='pc+fi+inet'
['setxkbmap', 'pc+fi+inet'] successfully applied
['xkbcomp', '-', ':7'] successfully applied
['xmodmap', '-'] successfully applied
damage(4, 0, 0, 1, 1) client is keeping up, reduced batch delay to: 66
damage(4, 0, 0, 1, 1) recent event list is too small, not batching
damage(4, 0, 0, 1, 1) sending now with sequence 0
Unhandled error while processing packet from peer
Traceback (most recent call last):
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/protocol.py", line 272, in _process_packet
    self._process_packet_cb(self, decoded)
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 1504, in process_packet
    self._packet_handlers[packet_type](self, proto, packet)
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 1189, in _process_hello
    self._send_new_or_window_packet(window)
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 970, in _send_new_or_window_packet
    self._damage(window, 0, 0, w, h)
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 951, in _damage
    self._protocol.source.damage(id, window, x, y, width, height)
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 238, in damage
    return damage_now()
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 201, in damage_now
    add_damage_to_region(region, damage_info)
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 192, in add_damage_to_region
    now_full = maybe_send_full_window()
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 185, in maybe_send_full_window
    (_, _, ww, wh) = self._desktop_manager.window_geometry(window)
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 77, in window_geometry
    return self._models[model].geom
KeyError: <OverrideRedirectWindowModel object at 0x285c140 (wimpiggy+window+OverrideRedirectWindowModel at 0x290c0a0)>
resize_window to 1270x945, desktop manager set it to 1270x945
damage(1, 0, 0, 1270, 945) client is keeping up, reduced batch delay to: 50
damage(1, 0, 0, 1270, 945) recent event list is too small, not batching
damage(1, 0, 0, 1270, 945) sending now with sequence 1
Unhandled error while processing packet from peer
Traceback (most recent call last):
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/protocol.py", line 272, in _process_packet
    self._process_packet_cb(self, decoded)
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 1504, in process_packet
    self._packet_handlers[packet_type](self, proto, packet)
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 1280, in _process_map_window
    self._damage(window, 0, 0, width, height)
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 951, in _damage
    self._protocol.source.damage(id, window, x, y, width, height)
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 238, in damage
    return damage_now()
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 202, in damage_now
    self._protocol.source_has_more()
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/protocol.py", line 127, in source_has_more
    self._maybe_queue_more_writes()
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/protocol.py", line 131, in _maybe_queue_more_writes
    self._flush_one_packet_into_buffer()
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/protocol.py", line 151, in _flush_one_packet_into_buffer
    packet, self._source_has_more = self.source.next_packet()
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 267, in next_packet
    packet = self.next_damage_packet()
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 299, in next_damage_packet
    window.acknowledge_changes()
TypeError: acknowledge_changes() takes exactly 5 arguments (1 given)
resize_window to 1272x946, desktop manager set it to 1272x946
resize_window to 1272x946, desktop manager set it to 1272x946
damage(2, 0, 0, 1272, 946) recent event list is too small, not batching
damage(2, 0, 0, 1272, 946) sending now with sequence 2
Unhandled error while processing packet from peer
Traceback (most recent call last):
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/protocol.py", line 272, in _process_packet
    self._process_packet_cb(self, decoded)
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 1504, in process_packet
    self._packet_handlers[packet_type](self, proto, packet)
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 1280, in _process_map_window
    self._damage(window, 0, 0, width, height)
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 951, in _damage
    self._protocol.source.damage(id, window, x, y, width, height)
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 238, in damage
    return damage_now()
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 202, in damage_now
    self._protocol.source_has_more()
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/protocol.py", line 127, in source_has_more
    self._maybe_queue_more_writes()
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/protocol.py", line 131, in _maybe_queue_more_writes
    self._flush_one_packet_into_buffer()
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/protocol.py", line 151, in _flush_one_packet_into_buffer
    packet, self._source_has_more = self.source.next_packet()
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 267, in next_packet
    packet = self.next_damage_packet()
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 299, in next_damage_packet
    window.acknowledge_changes()
TypeError: acknowledge_changes() takes exactly 5 arguments (1 given)
damage(3, 0, 0, 1272, 946) recent event list is too small, not batching
damage(3, 0, 0, 1272, 946) sending now with sequence 3
Unhandled error while processing packet from peer
Traceback (most recent call last):
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/protocol.py", line 272, in _process_packet
    self._process_packet_cb(self, decoded)
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 1504, in process_packet
    self._packet_handlers[packet_type](self, proto, packet)
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 1280, in _process_map_window
    self._damage(window, 0, 0, width, height)
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 951, in _damage
    self._protocol.source.damage(id, window, x, y, width, height)
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 238, in damage
    return damage_now()
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 202, in damage_now
    self._protocol.source_has_more()
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/protocol.py", line 127, in source_has_more
    self._maybe_queue_more_writes()
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/protocol.py", line 131, in _maybe_queue_more_writes
    self._flush_one_packet_into_buffer()
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/protocol.py", line 151, in _flush_one_packet_into_buffer
    packet, self._source_has_more = self.source.next_packet()
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 267, in next_packet
    packet = self.next_damage_packet()
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 299, in next_damage_packet
    window.acknowledge_changes()
TypeError: acknowledge_changes() takes exactly 5 arguments (1 given)
next_damage_packet is empty: <GdkRegion at 0x2c0b670>
(xpra:25032): atk-bridge-WARNING **: AT_SPI_REGISTRY was not started at session startup.
(xpra:25032): atk-bridge-WARNING **: IOR not set.
(xpra:25032): atk-bridge-WARNING **: Could not locate registry
Gtk-Message: Failed to load module "gail-gnome": libgail-gnome.so: cannot open shared object file: No such file or directory
Gtk-Message: Failed to load module "gail-gnome": libgail-gnome.so: cannot open shared object file: No such file or directory

Wed, 23 Nov 2011 09:31:53 GMT - Antoine Martin:

Sorry about that, the patch required some changes that were not in trunk yet..

Please see the changelog in r284: there are a lot of new heuristics for detecting when either end is unable to cope with damage requests. This should solve your problems. For debugging, if necessary, change:

log("update_batch_delay: ...

to

log.info("update_batch_delay: ...

You should then see something like this:

update_batch_delay: damage data queue overflow: 6, factor=3.60735492206, delta=0, new batch delay=18
update_batch_delay: damage packet queue is full, factor=1.42426406871, delta=0, new batch delay=25
update_batch_delay: client up to date, factor=1.0, delta=1, new batch delay=24
update_batch_delay: damage data queue overflow: 4, factor=3.12192809489, delta=0, new batch delay=74
update_batch_delay: client up to date, factor=1.0, delta=1, new batch delay=73
...
update_batch_delay: client up to date, factor=1.0, delta=2, new batch delay=21
update_batch_delay: client 7 damage packets behind, factor=3.0, delta=0, new batch delay=63
update_batch_delay: client 7 damage packets behind, factor=1.2, delta=0, new batch delay=75

If you find a scenario where the batch_delay is not adjusted correctly, we can tweak the maths..

Please confirm so I can close this ticket and make a new release.


Wed, 23 Nov 2011 11:15:43 GMT - Timo Juhani Lindfors:

I still see nothing in the windows but white background:

$ scratch/xpra-do-run.sh 286 start --no-daemon --use-display :7
found /home/lindi/xpra-install/amd64/svn286_v0.0.7.22-202-gcadf6f0/
Xlib:  extension "RANDR" missing on display ":7.0".
Xlib:  extension "RANDR" missing on display ":7.0".
Xlib:  extension "RANDR" missing on display ":7.0".
Randr not supported: X server does not support required extension Randr
randr enabled: False
using notification forwarder
xpra is ready.
New connection received
Handshake complete; enabling connection
client resolution is [1280, 1024], current server resolution is 3840x2560
setting key repeat rate from client: 500 / 33
guessing keyboard layout='pc+fi+inet'
['setxkbmap', 'pc+fi+inet'] successfully applied
['xkbcomp', '-', ':7'] successfully applied
['xmodmap', '-'] successfully applied
Unhandled exception in thread started by <bound method ServerSource.damage_to_data of <xpra.server.ServerSource object at 0x1133ad0>>
Traceback (most recent call last):
  File "/home/lindi/xpra-install/amd64/svn286_v0.0.7.22-202-gcadf6f0/lib/python/xpra/server.py", line 311, in damage_to_data
    (_, _, ww, wh) = self._desktop_manager.window_geometry(window)
  File "/home/lindi/xpra-install/amd64/svn286_v0.0.7.22-202-gcadf6f0/lib/python/xpra/server.py", line 85, in window_geometry
    return self._models[model].geom
KeyError: <OverrideRedirectWindowModel object at 0xdbe460 (wimpiggy+window+OverrideRedirectWindowModel at 0xe76480)>
(xpra:768): atk-bridge-WARNING **: AT_SPI_REGISTRY was not started at session startup.
(xpra:768): atk-bridge-WARNING **: IOR not set.
(xpra:768): atk-bridge-WARNING **: Could not locate registry
Gtk-Message: Failed to load module "gail-gnome": libgail-gnome.so: cannot open shared object file: No such file or directory
Gtk-Message: Failed to load module "gail-gnome": libgail-gnome.so: cannot open shared object file: No such file or directory

Wed, 23 Nov 2011 11:22:08 GMT - Antoine Martin:

for the error just above, please see #44 (fixed by r287 - better fix since in r300)


Tue, 29 Nov 2011 18:24:16 GMT - Antoine Martin: status changed; resolution set


Mon, 20 Feb 2012 19:57:48 GMT - Antoine Martin: version set


Sat, 23 Jan 2021 04:43:28 GMT - migration script:

this ticket has been moved to: https://github.com/Xpra-org/xpra/issues/23