xpra icon
Bug tracker and wiki

Opened 9 years ago

Closed 9 years ago

Last modified 9 years ago

#23 closed defect (worksforme)

fast refresh (ie: fast scrolling) causes high cpu usage and no screen updates

Reported by: Antoine Martin Owned by: Antoine Martin
Priority: critical Milestone: 0.0.7.x
Component: client Version: 0.0.7.32
Keywords: Cc:

Description

Not sure why the screen does not update (I don't get that behaviour here), but in any case we need to deal with fast updates better.

Maybe we can detect when a window is causing bursts of damage requests and buffer them for a bit, by the time we get around to processing them we may be able to remove many duplicates for the same region, or if the total surface of the damage requests is close to the full window size then just do one full refresh instead.

The damage requests from the server end come via _contents_changed in server.py (from CompositeHelper and BaseWindowModel.setup()).
Currently this fires _protocol.source_has_more() which will write the packet if the write queue is empty.
Could it be that very large packets make the write queue appear empty (it is) when in fact there is still a large chunk of data still to be sent in _write_thread_loop?
Or do the packets go out faster than client can deal with?

More testing needed. Ideas and suggestions welcome.

Attachments (1)

adaptive-damage.patch (14.9 KB) - added by Antoine Martin 9 years ago.
better batching: send whole screen to save rectangles and speedup/slow down according to client consumption rate

Download all attachments as: .zip

Change History (14)

comment:1 Changed 9 years ago by Antoine Martin

Resolution: fixed
Status: newclosed

r179 deals with this, we may improve on that and automatically adjust the batch delay based on better heuristics than just the number of damage requests but this will do for now. closing.

comment:2 Changed 9 years ago by Timo Juhani Lindfors

while true; do

echo "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"

done

still causes xpra svn 266 client to spend all its cpu time and not update any windows. Sending ctrl-c is also not possible.

comment:3 Changed 9 years ago by Antoine Martin

Resolution: fixed
Status: closedreopened

comment:4 Changed 9 years ago by Timo Juhani Lindfors

Both client and server are relatively fast x86 systems connected over 100 Mbps ethernet. I'm simply using

xpra start :7

and

xpra attach ssh:server:7

Changed 9 years ago by Antoine Martin

Attachment: adaptive-damage.patch added

better batching: send whole screen to save rectangles and speedup/slow down according to client consumption rate

comment:5 Changed 9 years ago by Antoine Martin

Owner: changed from Antoine Martin to Antoine Martin
Status: reopenedaccepted

Can you please try the attached patch and most the server log output if you still do get lag.

comment:6 Changed 9 years ago by Timo Juhani Lindfors

The patch does not apply to svn 266 or svn 279.

$ patch -p1 < adaptive-damage.patch 
patching file server.py
Hunk #1 FAILED at 131.
Hunk #2 FAILED at 141.
Hunk #3 FAILED at 166.
Hunk #4 FAILED at 240.
Hunk #5 FAILED at 287.
Hunk #6 FAILED at 397.
Hunk #7 FAILED at 1069.
Hunk #8 FAILED at 1083.
Hunk #9 FAILED at 1154.
Hunk #10 FAILED at 1244.
Hunk #11 FAILED at 1362.
Hunk #12 FAILED at 1419.
12 out of 12 hunks FAILED -- saving rejects to file server.py.rej
patching file client.py
Hunk #1 FAILED at 360.
Hunk #2 FAILED at 651.
Hunk #3 FAILED at 741.
Hunk #4 FAILED at 781.
4 out of 4 hunks FAILED -- saving rejects to file client.py.rej

comment:7 Changed 9 years ago by Timo Juhani Lindfors

Oh well, it does apply. I just have to use

cd src && patch -p0 < ../adaptive-damage.patch

instead of the usualy

patch -p1 < adaptive-damage.patch

comment:8 Changed 9 years ago by Timo Juhani Lindfors

After I apply this patch to svn 279 I see no updates to windows at all. Server prints

lindi1:~$ xpra start --no-daemon --use-display :7
found /home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty
Xlib:  extension "RANDR" missing on display ":7.0".
Xlib:  extension "RANDR" missing on display ":7.0".
Xlib:  extension "RANDR" missing on display ":7.0".
Randr not supported: X server does not support required extension Randr
randr enabled: False
using notification forwarder

xpra is ready.
New connection received
Handshake complete; enabling connection
client resolution is [1280, 1024], current server resolution is 3840x2560
setting key repeat rate from client: 500 / 33
guessing keyboard layout='pc+fi+inet'
['setxkbmap', 'pc+fi+inet'] successfully applied
['xkbcomp', '-', ':7'] successfully applied
['xmodmap', '-'] successfully applied
damage(4, 0, 0, 1, 1) client is keeping up, reduced batch delay to: 66
damage(4, 0, 0, 1, 1) recent event list is too small, not batching
damage(4, 0, 0, 1, 1) sending now with sequence 0
Unhandled error while processing packet from peer
Traceback (most recent call last):
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/protocol.py", line 272, in _process_packet
    self._process_packet_cb(self, decoded)
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 1504, in process_packet
    self._packet_handlers[packet_type](self, proto, packet)
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 1189, in _process_hello
    self._send_new_or_window_packet(window)
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 970, in _send_new_or_window_packet
    self._damage(window, 0, 0, w, h)
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 951, in _damage
    self._protocol.source.damage(id, window, x, y, width, height)
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 238, in damage
    return damage_now()
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 201, in damage_now
    add_damage_to_region(region, damage_info)
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 192, in add_damage_to_region
    now_full = maybe_send_full_window()
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 185, in maybe_send_full_window
    (_, _, ww, wh) = self._desktop_manager.window_geometry(window)
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 77, in window_geometry
    return self._models[model].geom
KeyError: <OverrideRedirectWindowModel object at 0x285c140 (wimpiggy+window+OverrideRedirectWindowModel at 0x290c0a0)>
resize_window to 1270x945, desktop manager set it to 1270x945
damage(1, 0, 0, 1270, 945) client is keeping up, reduced batch delay to: 50
damage(1, 0, 0, 1270, 945) recent event list is too small, not batching
damage(1, 0, 0, 1270, 945) sending now with sequence 1
Unhandled error while processing packet from peer
Traceback (most recent call last):
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/protocol.py", line 272, in _process_packet
    self._process_packet_cb(self, decoded)
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 1504, in process_packet
    self._packet_handlers[packet_type](self, proto, packet)
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 1280, in _process_map_window
    self._damage(window, 0, 0, width, height)
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 951, in _damage
    self._protocol.source.damage(id, window, x, y, width, height)
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 238, in damage
    return damage_now()
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 202, in damage_now
    self._protocol.source_has_more()
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/protocol.py", line 127, in source_has_more
    self._maybe_queue_more_writes()
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/protocol.py", line 131, in _maybe_queue_more_writes
    self._flush_one_packet_into_buffer()
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/protocol.py", line 151, in _flush_one_packet_into_buffer
    packet, self._source_has_more = self.source.next_packet()
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 267, in next_packet
    packet = self.next_damage_packet()
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 299, in next_damage_packet
    window.acknowledge_changes()
TypeError: acknowledge_changes() takes exactly 5 arguments (1 given)
resize_window to 1272x946, desktop manager set it to 1272x946
resize_window to 1272x946, desktop manager set it to 1272x946
damage(2, 0, 0, 1272, 946) recent event list is too small, not batching
damage(2, 0, 0, 1272, 946) sending now with sequence 2
Unhandled error while processing packet from peer
Traceback (most recent call last):
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/protocol.py", line 272, in _process_packet
    self._process_packet_cb(self, decoded)
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 1504, in process_packet
    self._packet_handlers[packet_type](self, proto, packet)
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 1280, in _process_map_window
    self._damage(window, 0, 0, width, height)
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 951, in _damage
    self._protocol.source.damage(id, window, x, y, width, height)
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 238, in damage
    return damage_now()
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 202, in damage_now
    self._protocol.source_has_more()
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/protocol.py", line 127, in source_has_more
    self._maybe_queue_more_writes()
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/protocol.py", line 131, in _maybe_queue_more_writes
    self._flush_one_packet_into_buffer()
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/protocol.py", line 151, in _flush_one_packet_into_buffer
    packet, self._source_has_more = self.source.next_packet()
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 267, in next_packet
    packet = self.next_damage_packet()
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 299, in next_damage_packet
    window.acknowledge_changes()
TypeError: acknowledge_changes() takes exactly 5 arguments (1 given)
damage(3, 0, 0, 1272, 946) recent event list is too small, not batching
damage(3, 0, 0, 1272, 946) sending now with sequence 3
Unhandled error while processing packet from peer
Traceback (most recent call last):
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/protocol.py", line 272, in _process_packet
    self._process_packet_cb(self, decoded)
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 1504, in process_packet
    self._packet_handlers[packet_type](self, proto, packet)
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 1280, in _process_map_window
    self._damage(window, 0, 0, width, height)
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 951, in _damage
    self._protocol.source.damage(id, window, x, y, width, height)
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 238, in damage
    return damage_now()
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 202, in damage_now
    self._protocol.source_has_more()
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/protocol.py", line 127, in source_has_more
    self._maybe_queue_more_writes()
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/protocol.py", line 131, in _maybe_queue_more_writes
    self._flush_one_packet_into_buffer()
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/protocol.py", line 151, in _flush_one_packet_into_buffer
    packet, self._source_has_more = self.source.next_packet()
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 267, in next_packet
    packet = self.next_damage_packet()
  File "/home/lindi/xpra-install/amd64/svn279_v0.0.7.22-195-g8e04984-dirty/lib/python/xpra/server.py", line 299, in next_damage_packet
    window.acknowledge_changes()
TypeError: acknowledge_changes() takes exactly 5 arguments (1 given)
next_damage_packet is empty: <GdkRegion at 0x2c0b670>

(xpra:25032): atk-bridge-WARNING **: AT_SPI_REGISTRY was not started at session startup.

(xpra:25032): atk-bridge-WARNING **: IOR not set.

(xpra:25032): atk-bridge-WARNING **: Could not locate registry
Gtk-Message: Failed to load module "gail-gnome": libgail-gnome.so: cannot open shared object file: No such file or directory
Gtk-Message: Failed to load module "gail-gnome": libgail-gnome.so: cannot open shared object file: No such file or directory


comment:9 Changed 9 years ago by Antoine Martin

Sorry about that, the patch required some changes that were not in trunk yet..

Please see the changelog in r284: there are a lot of new heuristics for detecting when either end is unable to cope with damage requests. This should solve your problems.
For debugging, if necessary, change:

log("update_batch_delay: ...

to

log.info("update_batch_delay: ...

You should then see something like this:

update_batch_delay: damage data queue overflow: 6, factor=3.60735492206, delta=0, new batch delay=18
update_batch_delay: damage packet queue is full, factor=1.42426406871, delta=0, new batch delay=25
update_batch_delay: client up to date, factor=1.0, delta=1, new batch delay=24
update_batch_delay: damage data queue overflow: 4, factor=3.12192809489, delta=0, new batch delay=74
update_batch_delay: client up to date, factor=1.0, delta=1, new batch delay=73
...
update_batch_delay: client up to date, factor=1.0, delta=2, new batch delay=21
update_batch_delay: client 7 damage packets behind, factor=3.0, delta=0, new batch delay=63
update_batch_delay: client 7 damage packets behind, factor=1.2, delta=0, new batch delay=75

If you find a scenario where the batch_delay is not adjusted correctly, we can tweak the maths..

Please confirm so I can close this ticket and make a new release.

comment:10 Changed 9 years ago by Timo Juhani Lindfors

I still see nothing in the windows but white background:

$ scratch/xpra-do-run.sh 286 start --no-daemon --use-display :7
found /home/lindi/xpra-install/amd64/svn286_v0.0.7.22-202-gcadf6f0/
Xlib:  extension "RANDR" missing on display ":7.0".
Xlib:  extension "RANDR" missing on display ":7.0".
Xlib:  extension "RANDR" missing on display ":7.0".
Randr not supported: X server does not support required extension Randr
randr enabled: False
using notification forwarder

xpra is ready.
New connection received
Handshake complete; enabling connection
client resolution is [1280, 1024], current server resolution is 3840x2560
setting key repeat rate from client: 500 / 33
guessing keyboard layout='pc+fi+inet'
['setxkbmap', 'pc+fi+inet'] successfully applied
['xkbcomp', '-', ':7'] successfully applied
['xmodmap', '-'] successfully applied
Unhandled exception in thread started by <bound method ServerSource.damage_to_data of <xpra.server.ServerSource object at 0x1133ad0>>
Traceback (most recent call last):
  File "/home/lindi/xpra-install/amd64/svn286_v0.0.7.22-202-gcadf6f0/lib/python/xpra/server.py", line 311, in damage_to_data
    (_, _, ww, wh) = self._desktop_manager.window_geometry(window)
  File "/home/lindi/xpra-install/amd64/svn286_v0.0.7.22-202-gcadf6f0/lib/python/xpra/server.py", line 85, in window_geometry
    return self._models[model].geom
KeyError: <OverrideRedirectWindowModel object at 0xdbe460 (wimpiggy+window+OverrideRedirectWindowModel at 0xe76480)>

(xpra:768): atk-bridge-WARNING **: AT_SPI_REGISTRY was not started at session startup.

(xpra:768): atk-bridge-WARNING **: IOR not set.

(xpra:768): atk-bridge-WARNING **: Could not locate registry
Gtk-Message: Failed to load module "gail-gnome": libgail-gnome.so: cannot open shared object file: No such file or directory
Gtk-Message: Failed to load module "gail-gnome": libgail-gnome.so: cannot open shared object file: No such file or directory


comment:11 Changed 9 years ago by Antoine Martin

for the error just above, please see #44 (fixed by r287 - better fix since in r300)

Last edited 9 years ago by Antoine Martin (previous) (diff)

comment:12 Changed 9 years ago by Antoine Martin

Resolution: worksforme
Status: acceptedclosed

comment:13 Changed 9 years ago by Antoine Martin

Version: 0.0.7.32
Note: See TracTickets for help on using tickets.