Not sure why the screen does not update (I don't get that behaviour here), but in any case we need to deal with fast updates better.
Maybe we can detect when a window is causing bursts of damage requests and buffer them for a bit, by the time we get around to processing them we may be able to remove many duplicates for the same region, or if the total surface of the damage requests is close to the full window size then just do one full refresh instead.
The damage requests from the server end come via _contents_changed
in server.py
(from CompositeHelper
and BaseWindowModel.setup()
).
Currently this fires _protocol.source_has_more()
which will write the packet if the write queue is empty.
Could it be that very large packets make the write queue appear empty (it is) when in fact there is still a large chunk of data still to be sent in _write_thread_loop
?
Or do the packets go out faster than client can deal with?
More testing needed. Ideas and suggestions welcome.
r179 deals with this, we may improve on that and automatically adjust the batch delay based on better heuristics than just the number of damage requests but this will do for now. closing.
while true; do
echo "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"
done
still causes xpra svn 266 client to spend all its cpu time and not update any windows. Sending ctrl-c is also not possible.
Both client and server are relatively fast x86 systems connected over 100 Mbps ethernet. I'm simply using
xpra start :7
and
xpra attach ssh:server:7
better batching: send whole screen to save rectangles and speedup/slow down according to client consumption rate
Can you please try the attached patch and most the server log output if you still do get lag.
The patch does not apply to svn 266 or svn 279.
$ patch -p1 < adaptive-damage.patch patching file server.py Hunk #1 FAILED at 131. Hunk #2 FAILED at 141. Hunk #3 FAILED at 166. Hunk #4 FAILED at 240. Hunk #5 FAILED at 287. Hunk #6 FAILED at 397. Hunk #7 FAILED at 1069. Hunk #8 FAILED at 1083. Hunk #9 FAILED at 1154. Hunk #10 FAILED at 1244. Hunk #11 FAILED at 1362. Hunk #12 FAILED at 1419. 12 out of 12 hunks FAILED -- saving rejects to file server.py.rej patching file client.py Hunk #1 FAILED at 360. Hunk #2 FAILED at 651. Hunk #3 FAILED at 741. Hunk #4 FAILED at 781. 4 out of 4 hunks FAILED -- saving rejects to file client.py.rej
Oh well, it does apply. I just have to use
cd src && patch -p0 < ../adaptive-damage.patch
instead of the usualy
patch -p1 < adaptive-damage.patch
After I apply this patch to svn 279 I see no updates to windows at all. Server prints
lindi1:~$ xpra start --no-daemon --use-display :7 found /home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty Xlib: extension "RANDR" missing on display ":7.0". Xlib: extension "RANDR" missing on display ":7.0". Xlib: extension "RANDR" missing on display ":7.0". Randr not supported: X server does not support required extension Randr randr enabled: False using notification forwarder xpra is ready. New connection received Handshake complete; enabling connection client resolution is [1280, 1024], current server resolution is 3840x2560 setting key repeat rate from client: 500 / 33 guessing keyboard layout='pc+fi+inet' ['setxkbmap', 'pc+fi+inet'] successfully applied ['xkbcomp', '-', ':7'] successfully applied ['xmodmap', '-'] successfully applied damage(4, 0, 0, 1, 1) client is keeping up, reduced batch delay to: 66 damage(4, 0, 0, 1, 1) recent event list is too small, not batching damage(4, 0, 0, 1, 1) sending now with sequence 0 Unhandled error while processing packet from peer Traceback (most recent call last): File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/protocol.py", line 272, in _process_packet self._process_packet_cb(self, decoded) File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 1504, in process_packet self._packet_handlers[packet_type](self, proto, packet) File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 1189, in _process_hello self._send_new_or_window_packet(window) File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 970, in _send_new_or_window_packet self._damage(window, 0, 0, w, h) File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 951, in _damage self._protocol.source.damage(id, window, x, y, width, height) File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 238, in damage return damage_now() File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 201, in damage_now add_damage_to_region(region, damage_info) File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 192, in add_damage_to_region now_full = maybe_send_full_window() File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 185, in maybe_send_full_window (_, _, ww, wh) = self._desktop_manager.window_geometry(window) File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 77, in window_geometry return self._models[model].geom KeyError: <OverrideRedirectWindowModel object at 0x285c140 (wimpiggy+window+OverrideRedirectWindowModel at 0x290c0a0)> resize_window to 1270x945, desktop manager set it to 1270x945 damage(1, 0, 0, 1270, 945) client is keeping up, reduced batch delay to: 50 damage(1, 0, 0, 1270, 945) recent event list is too small, not batching damage(1, 0, 0, 1270, 945) sending now with sequence 1 Unhandled error while processing packet from peer Traceback (most recent call last): File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/protocol.py", line 272, in _process_packet self._process_packet_cb(self, decoded) File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 1504, in process_packet self._packet_handlers[packet_type](self, proto, packet) File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 1280, in _process_map_window self._damage(window, 0, 0, width, height) File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 951, in _damage self._protocol.source.damage(id, window, x, y, width, height) File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 238, in damage return damage_now() File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 202, in damage_now self._protocol.source_has_more() File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/protocol.py", line 127, in source_has_more self._maybe_queue_more_writes() File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/protocol.py", line 131, in _maybe_queue_more_writes self._flush_one_packet_into_buffer() File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/protocol.py", line 151, in _flush_one_packet_into_buffer packet, self._source_has_more = self.source.next_packet() File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 267, in next_packet packet = self.next_damage_packet() File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 299, in next_damage_packet window.acknowledge_changes() TypeError: acknowledge_changes() takes exactly 5 arguments (1 given) resize_window to 1272x946, desktop manager set it to 1272x946 resize_window to 1272x946, desktop manager set it to 1272x946 damage(2, 0, 0, 1272, 946) recent event list is too small, not batching damage(2, 0, 0, 1272, 946) sending now with sequence 2 Unhandled error while processing packet from peer Traceback (most recent call last): File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/protocol.py", line 272, in _process_packet self._process_packet_cb(self, decoded) File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 1504, in process_packet self._packet_handlers[packet_type](self, proto, packet) File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 1280, in _process_map_window self._damage(window, 0, 0, width, height) File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 951, in _damage self._protocol.source.damage(id, window, x, y, width, height) File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 238, in damage return damage_now() File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 202, in damage_now self._protocol.source_has_more() File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/protocol.py", line 127, in source_has_more self._maybe_queue_more_writes() File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/protocol.py", line 131, in _maybe_queue_more_writes self._flush_one_packet_into_buffer() File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/protocol.py", line 151, in _flush_one_packet_into_buffer packet, self._source_has_more = self.source.next_packet() File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 267, in next_packet packet = self.next_damage_packet() File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 299, in next_damage_packet window.acknowledge_changes() TypeError: acknowledge_changes() takes exactly 5 arguments (1 given) damage(3, 0, 0, 1272, 946) recent event list is too small, not batching damage(3, 0, 0, 1272, 946) sending now with sequence 3 Unhandled error while processing packet from peer Traceback (most recent call last): File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/protocol.py", line 272, in _process_packet self._process_packet_cb(self, decoded) File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 1504, in process_packet self._packet_handlers[packet_type](self, proto, packet) File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 1280, in _process_map_window self._damage(window, 0, 0, width, height) File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 951, in _damage self._protocol.source.damage(id, window, x, y, width, height) File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 238, in damage return damage_now() File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 202, in damage_now self._protocol.source_has_more() File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/protocol.py", line 127, in source_has_more self._maybe_queue_more_writes() File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/protocol.py", line 131, in _maybe_queue_more_writes self._flush_one_packet_into_buffer() File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/protocol.py", line 151, in _flush_one_packet_into_buffer packet, self._source_has_more = self.source.next_packet() File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 267, in next_packet packet = self.next_damage_packet() File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 299, in next_damage_packet window.acknowledge_changes() TypeError: acknowledge_changes() takes exactly 5 arguments (1 given) next_damage_packet is empty: <GdkRegion at 0x2c0b670> (xpra:25032): atk-bridge-WARNING **: AT_SPI_REGISTRY was not started at session startup. (xpra:25032): atk-bridge-WARNING **: IOR not set. (xpra:25032): atk-bridge-WARNING **: Could not locate registry Gtk-Message: Failed to load module "gail-gnome": libgail-gnome.so: cannot open shared object file: No such file or directory Gtk-Message: Failed to load module "gail-gnome": libgail-gnome.so: cannot open shared object file: No such file or directory
Sorry about that, the patch required some changes that were not in trunk yet..
Please see the changelog in r284: there are a lot of new heuristics for detecting when either end is unable to cope with damage requests. This should solve your problems. For debugging, if necessary, change:
log("update_batch_delay: ...
to
log.info("update_batch_delay: ...
You should then see something like this:
update_batch_delay: damage data queue overflow: 6, factor=3.60735492206, delta=0, new batch delay=18 update_batch_delay: damage packet queue is full, factor=1.42426406871, delta=0, new batch delay=25 update_batch_delay: client up to date, factor=1.0, delta=1, new batch delay=24 update_batch_delay: damage data queue overflow: 4, factor=3.12192809489, delta=0, new batch delay=74 update_batch_delay: client up to date, factor=1.0, delta=1, new batch delay=73 ... update_batch_delay: client up to date, factor=1.0, delta=2, new batch delay=21 update_batch_delay: client 7 damage packets behind, factor=3.0, delta=0, new batch delay=63 update_batch_delay: client 7 damage packets behind, factor=1.2, delta=0, new batch delay=75
If you find a scenario where the batch_delay is not adjusted correctly, we can tweak the maths..
Please confirm so I can close this ticket and make a new release.
I still see nothing in the windows but white background:
$ scratch/xpra-do-run.sh 286 start --no-daemon --use-display :7 found /home/lindi/xpra-install/amd64/svn286_v0.0.7.22-202-gcadf6f0/ Xlib: extension "RANDR" missing on display ":7.0". Xlib: extension "RANDR" missing on display ":7.0". Xlib: extension "RANDR" missing on display ":7.0". Randr not supported: X server does not support required extension Randr randr enabled: False using notification forwarder xpra is ready. New connection received Handshake complete; enabling connection client resolution is [1280, 1024], current server resolution is 3840x2560 setting key repeat rate from client: 500 / 33 guessing keyboard layout='pc+fi+inet' ['setxkbmap', 'pc+fi+inet'] successfully applied ['xkbcomp', '-', ':7'] successfully applied ['xmodmap', '-'] successfully applied Unhandled exception in thread started by <bound method ServerSource.damage_to_data of <xpra.server.ServerSource object at 0x1133ad0>> Traceback (most recent call last): File "/home/lindi/xpra-install/amd64/svn286_v0.0.7.22-202-gcadf6f0/lib/python/xpra/server.py", line 311, in damage_to_data (_, _, ww, wh) = self._desktop_manager.window_geometry(window) File "/home/lindi/xpra-install/amd64/svn286_v0.0.7.22-202-gcadf6f0/lib/python/xpra/server.py", line 85, in window_geometry return self._models[model].geom KeyError: <OverrideRedirectWindowModel object at 0xdbe460 (wimpiggy+window+OverrideRedirectWindowModel at 0xe76480)> (xpra:768): atk-bridge-WARNING **: AT_SPI_REGISTRY was not started at session startup. (xpra:768): atk-bridge-WARNING **: IOR not set. (xpra:768): atk-bridge-WARNING **: Could not locate registry Gtk-Message: Failed to load module "gail-gnome": libgail-gnome.so: cannot open shared object file: No such file or directory Gtk-Message: Failed to load module "gail-gnome": libgail-gnome.so: cannot open shared object file: No such file or directory
for the error just above, please see #44 (fixed by r287 - better fix since in r300)
this ticket has been moved to: https://github.com/Xpra-org/xpra/issues/23