xpra icon
Bug tracker and wiki

Opened 3 years ago

Closed 2 weeks ago

#1320 closed enhancement (fixed)

lossless scrolling

Reported by: Antoine Martin Owned by: Antoine Martin
Priority: minor Milestone: 3.0
Component: encodings Version: trunk
Keywords: scrolling Cc:

Description

Follow up from #1232: we could remove the need for auto-refresh packets when we do scrolling and the original data is lossless.
The same rectangle transformations could be applied to the list of regions needed a refresh.

Attachments (1)

lossless-scrolling.patch (4.8 KB) - added by Antoine Martin 5 months ago.
work in progress patch

Download all attachments as: .zip

Change History (7)

comment:1 Changed 7 months ago by Antoine Martin

Milestone: future3.0
Status: newassigned

Changed 5 months ago by Antoine Martin

Attachment: lossless-scrolling.patch added

work in progress patch

comment:2 Changed 5 months ago by Antoine Martin

Updates:

  • r22188 better rectangle hash: prevent collisions, already backported
  • r22189 bug fix (almost impossible to trigger), already backported
  • r22190 boost non-scroll quality, more so if we have a high match percentage (less to send) - could be backported but not strictly a bug fix

Those fixes combined with the patch above and scrolling already works a lot better than I expected. Giving smooth lossless scrolling in most cases.

Still TODO:

  • fix race conditions: we modify the refresh list from a network thread..
  • the list of rectangles could grow too big, and we traverse it twice (O(N^2)): give up past a certain size?
  • maybe try harder to re-merge contiguous rectangles?

One unintended benefit of this ticket is that we should be able to see bugs in the scroll paint code more easily because those would not get refreshed. So far so good.

Last edited 5 months ago by Antoine Martin (previous) (diff)

comment:3 Changed 5 months ago by Antoine Martin

Owner: changed from Antoine Martin to Jonathan Anthony
Status: assignednew

Changes:

  • r22205 + r22206: reduce race condition window, try to only add to the refresh list until we consume it
  • r22207 more correct pixel accounting when adding regions to the refresh list (side note: the cython rectangle code is much faster with python2.. not sure why)
  • r22208 "lossless scrolling", the core changes

It works very well.

Caveats:

  • it may still be possible for the server to lose track of some regions that need a refresh, but the window of opportunity for this race condition is very small and unlikely - adding locking to prevent this race condition would be too costly.
  • a video may still look weird when scrolling since it runs in parallel, with its own scheduling.
  • I trust the html5 client scroll paint code less than the python client (ie: #1637), though it seems to work fine when I tried.

For testing, I had to force the quality lower to ensure that the non-scroll areas would get sent using a lossy encoding and later refreshed correctly (ie: jpeg or webp in lossy mode is what we want).
To make sure that the server doesn't choose rgb for those, I configured my client to not use rgb at all: --encodings=webp,png,jpeg,h264

To be able to see what is being sent to the client:

  • use -d compress on the server (and add -d refresh,scroll for even more logging
  • use paint boxes on the client: #760 (or the "paint" debug for the html5 client)

I can scroll up and down frantically using a browser as a client application and I never see any visual artifacts. And it's smooth too.

@encodedEntropy: this is a major improvement over the previous version of the scrolling code.

comment:4 Changed 3 weeks ago by Smo

Owner: changed from Jonathan Anthony to Smo

comment:5 Changed 2 weeks ago by Smo

Owner: changed from Smo to Antoine Martin

Tested with server and client 3.0-r23413

server

xpra --bind-tcp=0.0.0.0:14000 --start=xterm --no-daemon -d compress,refresh,scroll start :15

client

XPRA_OPENGL_PAINT_BOX=1 xpra --encodings=webp,png,jpeg,h264 attach tcp:localhost:14000

Also tested with google-chrome and the html5 client.

In both cases the scrolling was working as expected and was extremely smooth.
Very good!

comment:6 Changed 2 weeks ago by Antoine Martin

Resolution: fixed
Status: newclosed

@smo: why override the encodings?
Not having rgb in that list is going to hurt performance, why bother?

Note: See TracTickets for help on using tickets.