Xpra: Ticket #756: multi delta

At the moment, we only use delta compression for non video screen updates using the last suitable region we had. Multiple updates of differing sizes means we will constantly change the delta buffer, and never match anything.

We could let the client tell the server how many buffers it will accept, and of what size. Then the server can keep more delta regions in memory, keyed using their dimensions, and is more likely to make use of them.

This will help with things like unnecessary repaints (#670). The only difficulty will be with UDP and dropped packets #639: we will need a way of requesting a reset / refresh when we are missing a reference region.

Thu, 25 Dec 2014 16:50:29 GMT - Antoine Martin: owner, status changed

Implemented in r8285, see commit message for details.

Remaining tasks:

Sun, 28 Dec 2014 06:33:56 GMT - Antoine Martin: attachment set

frees buckets as needed - not sure this is worth doing

Sun, 28 Dec 2014 10:32:04 GMT - Antoine Martin: owner, status changed

More improvements in r8315, r8311, r8309: we now re-stride the data, and directly into the target buffer in Cython, so now much faster. I don't think we want to apply the delta-free patch above: too much complexity and extra code in the hot path, all for small memory savings. If we want to constrain the client memory usage, a better approach would be to let the client specify its own limit for the size of delta buffers. That's on top of being able to choose the number of delta buffers already via:

XPRA_DELTA_BUCKETS=10 xpra attach ..

Ready for testing, #760 may be helpful here to visualize where we use delta regions (and see if we miss any).

Fri, 23 Jan 2015 21:00:26 GMT - alas: owner changed

Seems to be working as expected (aside from the windows client opengl paint box issue mentioned in #760).

I would note that setting XPRA_DELTA_BUCKETS=11 does seem to lead to increased blurriness, even on a small window on a 1080p display.

Sat, 24 Jan 2015 01:54:11 GMT - Antoine Martin: owner changed

I would note that setting XPRA_DELTA_BUCKETS=11 does seem to lead to increased blurriness, even on a small window on a 1080p display.

It is not meant to.

I should explain: delta means that we substract the previous contents of a region from its current contents before compression. So if you end up showing more or less the same contents, you end up with something empty as delta - which compresses very well. Video compressors already do this by default (that's why they compress so well - and also why the size of the video cannot change without a full encoder re-initialization), we use delta compression on lossless picture encodings (rgb24/rgb32+lz4 and png) to benefit from the same trick.

Sat, 21 Feb 2015 01:56:21 GMT - alas:

Testing some more with 0.15.0 r8647 win32 client against 0.15.0 r8661 fedora 20 server... it looks like the blurriness when scrolling is resolving itself promptly (perhaps previous tests hit some extra latency... or maybe I was just using a monitor that periodically seems to actually be becoming blurry itself... not certain).

However... testing this I'm suddenly seeing that unicode error from #786 again (in this case 2015-02-20 17:31:10,731 unexpected data type <type 'unicode'> in logging packet: u'using audio codec: MPEG 1 Audio, Layer 3 (MP3)'), and I'm also seeing another message that I'm also seeing in a number of other places (not sure it's related specifically to this, thought it was a webkit browser related error until just now): 2015-02-20 17:25:13,779 failed to set group leader: 'module' object has no attribute 'SHGetPropertyStoreForWindow'.

In case it is liable to make matters clear, launched the server with the following:

[jimador@zapopan ~]$  XPRA_OPENGL_PAINT_BOX=1 dbus-launch xpra --no-daemon --bind-tcp= --start-child=xterm --start-child=xterm start :23 --remote-logging=on -d client

And the client with the following:

C:\Program Files (x86)\Xpra>SET XPRA_OPENGL_PAINT_BOX=1
C:\Program Files (x86)\Xpra>xpra_cmd.exe attach tcp:
2015-02-20 17:05:38,502 xpra client version 0.15.0
2015-02-20 17:05:38,986 OpenGL_accelerate module loaded
2015-02-20 17:05:38,986 Using accelerated ArrayDatatype
2015-02-20 17:05:39,095 detected keyboard: layout=us
2015-02-20 17:05:39,095 desktop size is 5120x2160 with 1 screen(s):
2015-02-20 17:05:39,095   '1\WinSta-Default' (1354x571 mm - DPI: 96x96) workarea
: 5120x2120
2015-02-20 17:05:39,095     DISPLAY1 3840x2160 at 1280x0 (621x341 mm - DPI: 157x
160) workarea: 3840x2120
2015-02-20 17:05:39,095     DISPLAY2 1280x720 (597x336 mm - DPI: 54x54) workarea
: 1280x680
2015-02-20 17:05:39,408 enabled remote logging, see server log file for output

And, with the client also logging to the server, the server initialization output this initially:

2015-02-20 17:05:47,941 Python/Gtk2 Microsoft Windows 8 client version 0.15.0 connected from 'hasenpfeffer' as 'hassenpfeffer' ('hassenpfeffer')
2015-02-20 17:05:47,942 using h264 as primary encoding, also available: vp8, png, png/P, png/L, webp, rgb24, jpeg, rgb32
2015-02-20 17:05:47,948 client root window size is 5120x2160 with 1 displays:
2015-02-20 17:05:47,949   '1\WinSta-Default' (1354x571 mm - DPI: 96x96) workarea: 5120x2120
2015-02-20 17:05:47,949     DISPLAY1 3840x2160 at 1280x0 (621x341 mm - DPI: 157x160) workarea: 3840x2120
2015-02-20 17:05:47,949     DISPLAY2 1280x720 (597x336 mm - DPI: 54x54) workarea: 1280x680
2015-02-20 17:05:48,098 server virtual display now set to 5120x2560 (best match for 5120x2160)
2015-02-20 17:05:48,100 setting key repeat rate from client: 500ms delay / 33ms interval
2015-02-20 17:05:48,101 setting keyboard layout to 'us'
2015-02-20 17:05:48,234 server: Linux Fedora 20 Heisenbug, Xpra version 0.15.0 (r8661)
2015-02-20 17:05:48,308 DPI set to 96 x 96
2015-02-20 17:05:48,310 sent updated screen size to 1 clients: 5120x256I 0 (max 8192x4096)
2015-02-20 17:05:48,313 Attached to tcp: (press Control-C to detach)
2015-02-20 17:05:48,333 failed to set group leader: 'module' object has no attribute 'SHGetPropertyStoreForWindow'
2015-02-20 17:05:48,347 using Pulseaudio device 'Monitor of Null Output'
2015-02-20 17:05:48,361 starting sound stream capture using Pulseaudio source
2015-02-20 17:05:48,362 failed to set group leader: 'module' object has no attribute 'SHGetPropertyStoreForWindow'
2015-02-20 17:05:50,472 codec: MPEG-1 Layer 3 (MP3)
2015-02-20 17:05:50,494 using audio codec: MPEG 1 Audio, Layer 3 (MP3)
2015-02-20 17:05:50,495 unexpected data type <type 'unicode'> in logging packet: u'using audio codec: MPEG 1 Audio, Layer 3 (MP3)'

Using firefox to test for delta processing, I'm periodically (mostly when opening a new tab, but not exactly correspondingly) seeing the failed to set group leader: 'module' object has no attribute 'SHGetPropertyStoreForWindow' error repeating.

Not sure if it is a combination of the global variable settings that is triggering these messages or if it is just a matter of my now aging test versions... any insights where to start (I mention here just before we close this delta ticket, because the combination of the global variables and the client debug forwarding seems to have triggered those messages which I was no longer seeing when testing #786 without all the other variables and log forwarding in place)... before I try going through the bits and pieces randomly until I find a combination-culprit?

Sat, 21 Feb 2015 06:02:56 GMT - Antoine Martin:

testing this I'm suddenly seeing that unicode error from #786 again

Tracked in #786.

failed to set group leader: 'module' object has no attribute 'SHGetPropertyStoreForWindow'

This is tracked in #799. If you can reproduce this problem with one of my later builds, please add to that ticket.

As for actual delta compression, I think we can close this ticket?

Mon, 23 Feb 2015 20:13:22 GMT - alas: status changed; resolution set

Testing again for #799, I couldn't reproduce the error(s).

Agreed regarding the actual delta compression.


Tue, 07 Apr 2015 09:20:18 GMT - Antoine Martin: status changed; resolution deleted

Whilst testing vp9 (#832) and having decoder problems (or maybe encoder problems even..), I think I've hit a bug related to multi-delta:

2015-04-05 19:49:05,883 paint_with_video_decoder: \
    wid=1, vp9 decompression error on 44 bytes of picture data for 499x316 pixels using vpx.Decoder(vp9), \
    options={'speed': 83, 'frame': 2, 'quality': 67, 'csc': 'YUV444P', 'encoding': 'vp9'}
2015-04-05 19:49:06,251 draw error
Traceback (most recent call last):
  File "/usr/lib64/python2.7/site-packages/xpra/client/ui_client_base.py", line 1962, in _do_draw
    window.draw_region(x, y, width, height, coding, data, rowstride, packet_sequence, options, [record_decode_time])
  File "/usr/lib64/python2.7/site-packages/xpra/client/client_window_base.py", line 423, in draw_region
    self._backing.draw_region(x, y, width, height, coding, img_data, rowstride, options, callbacks)
  File "/usr/lib64/python2.7/site-packages/xpra/client/window_backing_base.py", line 472, in draw_region
    self.paint_with_video_decoder(VIDEO_DECODERS.get(coding), coding, img_data, x, y, width, height, options, callbacks)
  File "/usr/lib64/python2.7/site-packages/xpra/client/window_backing_base.py", line 376, in paint_with_video_decoder
    img = self._video_decoder.decompress_image(img_data, options)
  File "xpra/codecs/vpx/decoder.pyx", line 300, in xpra.codecs.vpx.decoder.Decoder.decompress_image (xpra/codecs/vpx/decoder.c:4578)
  File "/usr/lib64/python2.7/site-packages/xpra/codecs/codec_constants.py", line 61, in get_subsampling_divs
    raise Exception("invalid pixel format: %s" % pixel_format)
Exception: invalid pixel format: bucket

Tue, 07 Apr 2015 12:57:35 GMT - Antoine Martin: status changed; resolution set

comment:8 can be ignored, the problem was a dangling pointer caused by a bug in the vpx decoder, fixed in r8940.

Thu, 16 Apr 2015 17:23:19 GMT - Antoine Martin:

r9028 adds a new debug flag for delta compression: -d delta.

Sun, 19 Apr 2015 11:35:15 GMT - Antoine Martin:

r9063 ensures that we expire all delta buckets after 20 hits to prevent xor decay where the bucket just contains unhelpful noise. The number of hits can be configured using:

XPRA_MAX_DELTA_HITS=20 xpra start ...

Tue, 19 May 2015 08:30:41 GMT - Antoine Martin:

This caused a bug: #861. (fixed in r9449)

Thu, 25 Apr 2019 06:28:11 GMT - Antoine Martin:

Related: #2248 (more aggressive use of scroll encoding)

Sat, 23 Jan 2021 05:05:03 GMT - migration script:

this ticket has been moved to: https://github.com/Xpra-org/xpra/issues/756