xpra icon
Bug tracker and wiki

Opened 2 years ago

Closed 2 years ago

Last modified 2 years ago

#756 closed task (fixed)

multi delta

Reported by: Antoine Martin Owned by: alas
Priority: major Milestone: 0.15
Component: core Version: trunk
Keywords: Cc:

Description

At the moment, we only use delta compression for non video screen updates using the last suitable region we had. Multiple updates of differing sizes means we will constantly change the delta buffer, and never match anything.

We could let the client tell the server how many buffers it will accept, and of what size.
Then the server can keep more delta regions in memory, keyed using their dimensions, and is more likely to make use of them.

This will help with things like unnecessary repaints (#670).
The only difficulty will be with UDP and dropped packets #639: we will need a way of requesting a reset / refresh when we are missing a reference region.

Attachments (1)

delta-free-buckets.patch (6.0 KB) - added by Antoine Martin 2 years ago.
frees buckets as needed - not sure this is worth doing

Download all attachments as: .zip

Change History (13)

comment:1 Changed 2 years ago by Antoine Martin

Owner: changed from Antoine Martin to Antoine Martin
Status: newassigned

Implemented in r8285, see commit message for details.

Remaining tasks:

  • we should re-stride the pixel data, at the moment we copy 2000*height pixels... which can be expensive
  • try to prefer a delta encoding if the damage area matches one of the delta regions
  • tell the client to free its delta bucket when we do it server-side

Changed 2 years ago by Antoine Martin

Attachment: delta-free-buckets.patch added

frees buckets as needed - not sure this is worth doing

comment:2 Changed 2 years ago by Antoine Martin

Owner: changed from Antoine Martin to alas
Status: assignednew

More improvements in r8315, r8311, r8309: we now re-stride the data, and directly into the target buffer in Cython, so now much faster.
I don't think we want to apply the delta-free patch above: too much complexity and extra code in the hot path, all for small memory savings. If we want to constrain the client memory usage, a better approach would be to let the client specify its own limit for the size of delta buffers. That's on top of being able to choose the number of delta buffers already via:

XPRA_DELTA_BUCKETS=10 xpra attach ..

Ready for testing, #760 may be helpful here to visualize where we use delta regions (and see if we miss any).

comment:3 Changed 2 years ago by alas

Owner: changed from alas to Antoine Martin

Seems to be working as expected (aside from the windows client opengl paint box issue mentioned in #760).

I would note that setting XPRA_DELTA_BUCKETS=11 does seem to lead to increased blurriness, even on a small window on a 1080p display.

comment:4 Changed 2 years ago by Antoine Martin

Owner: changed from Antoine Martin to alas

I would note that setting XPRA_DELTA_BUCKETS=11 does seem to lead to increased blurriness, even on a small window on a 1080p display.


It is not meant to.

I should explain: delta means that we substract the previous contents of a region from its current contents before compression.
So if you end up showing more or less the same contents, you end up with something empty as delta - which compresses very well.
Video compressors already do this by default (that's why they compress so well - and also why the size of the video cannot change without a full encoder re-initialization), we use delta compression on lossless picture encodings (rgb24/rgb32+lz4 and png) to benefit from the same trick.

Last edited 2 years ago by Antoine Martin (previous) (diff)

comment:5 Changed 2 years ago by alas

Testing some more with 0.15.0 r8647 win32 client against 0.15.0 r8661 fedora 20 server... it looks like the blurriness when scrolling is resolving itself promptly (perhaps previous tests hit some extra latency... or maybe I was just using a monitor that periodically seems to actually be becoming blurry itself... not certain).

However... testing this I'm suddenly seeing that unicode error from #786 again (in this case 2015-02-20 17:31:10,731 unexpected data type <type 'unicode'> in logging packet: u'using audio codec: MPEG 1 Audio, Layer 3 (MP3)'), and I'm also seeing another message that I'm also seeing in a number of other places (not sure it's related specifically to this, thought it was a webkit browser related error until just now): 2015-02-20 17:25:13,779 failed to set group leader: 'module' object has no attribute 'SHGetPropertyStoreForWindow'.

In case it is liable to make matters clear, launched the server with the following:

[jimador@zapopan ~]$  XPRA_OPENGL_PAINT_BOX=1 dbus-launch xpra --no-daemon --bind-tcp=0.0.0.0:1201 --start-child=xterm --start-child=xterm start :23 --remote-logging=on -d client

And the client with the following:

C:\Program Files (x86)\Xpra>SET XPRA_OPENGL_PAINT_BOX=1

C:\Program Files (x86)\Xpra>xpra_cmd.exe attach tcp:10.0.32.53:1201
2015-02-20 17:05:38,502 xpra client version 0.15.0
2015-02-20 17:05:38,986 OpenGL_accelerate module loaded
2015-02-20 17:05:38,986 Using accelerated ArrayDatatype
2015-02-20 17:05:39,095 detected keyboard: layout=us
2015-02-20 17:05:39,095 desktop size is 5120x2160 with 1 screen(s):
2015-02-20 17:05:39,095   '1\WinSta-Default' (1354x571 mm - DPI: 96x96) workarea
: 5120x2120
2015-02-20 17:05:39,095     DISPLAY1 3840x2160 at 1280x0 (621x341 mm - DPI: 157x
160) workarea: 3840x2120
2015-02-20 17:05:39,095     DISPLAY2 1280x720 (597x336 mm - DPI: 54x54) workarea
: 1280x680
2015-02-20 17:05:39,408 enabled remote logging, see server log file for output

And, with the client also logging to the server, the server initialization output this initially:

2015-02-20 17:05:47,941 Python/Gtk2 Microsoft Windows 8 client version 0.15.0 connected from 'hasenpfeffer' as 'hassenpfeffer' ('hassenpfeffer')
2015-02-20 17:05:47,942 using h264 as primary encoding, also available: vp8, png, png/P, png/L, webp, rgb24, jpeg, rgb32
2015-02-20 17:05:47,948 client root window size is 5120x2160 with 1 displays:
2015-02-20 17:05:47,949   '1\WinSta-Default' (1354x571 mm - DPI: 96x96) workarea: 5120x2120
2015-02-20 17:05:47,949     DISPLAY1 3840x2160 at 1280x0 (621x341 mm - DPI: 157x160) workarea: 3840x2120
2015-02-20 17:05:47,949     DISPLAY2 1280x720 (597x336 mm - DPI: 54x54) workarea: 1280x680
2015-02-20 17:05:48,098 server virtual display now set to 5120x2560 (best match for 5120x2160)
2015-02-20 17:05:48,100 setting key repeat rate from client: 500ms delay / 33ms interval
2015-02-20 17:05:48,101 setting keyboard layout to 'us'
2015-02-20 17:05:48,234 server: Linux Fedora 20 Heisenbug, Xpra version 0.15.0 (r8661)
2015-02-20 17:05:48,308 DPI set to 96 x 96
2015-02-20 17:05:48,310 sent updated screen size to 1 clients: 5120x256I 0 (max 8192x4096)
2015-02-20 17:05:48,313 Attached to tcp:10.0.32.53:1201 (press Control-C to detach)
2015-02-20 17:05:48,333 failed to set group leader: 'module' object has no attribute 'SHGetPropertyStoreForWindow'
2015-02-20 17:05:48,347 using Pulseaudio device 'Monitor of Null Output'
2015-02-20 17:05:48,361 starting sound stream capture using Pulseaudio source
2015-02-20 17:05:48,362 failed to set group leader: 'module' object has no attribute 'SHGetPropertyStoreForWindow'
2015-02-20 17:05:50,472 codec: MPEG-1 Layer 3 (MP3)
2015-02-20 17:05:50,494 using audio codec: MPEG 1 Audio, Layer 3 (MP3)
2015-02-20 17:05:50,495 unexpected data type <type 'unicode'> in logging packet: u'using audio codec: MPEG 1 Audio, Layer 3 (MP3)'

Using firefox to test for delta processing, I'm periodically (mostly when opening a new tab, but not exactly correspondingly) seeing the failed to set group leader: 'module' object has no attribute 'SHGetPropertyStoreForWindow' error repeating.

Not sure if it is a combination of the global variable settings that is triggering these messages or if it is just a matter of my now aging test versions... any insights where to start (I mention here just before we close this delta ticket, because the combination of the global variables and the client debug forwarding seems to have triggered those messages which I was no longer seeing when testing #786 without all the other variables and log forwarding in place)... before I try going through the bits and pieces randomly until I find a combination-culprit?

comment:6 Changed 2 years ago by Antoine Martin

testing this I'm suddenly seeing that unicode error from #786 again


Tracked in #786.


failed to set group leader: 'module' object has no attribute 'SHGetPropertyStoreForWindow'


This is tracked in #799. If you can reproduce this problem with one of my later builds, please add to that ticket.


As for actual delta compression, I think we can close this ticket?

comment:7 Changed 2 years ago by alas

Resolution: fixed
Status: newclosed

Testing again for #799, I couldn't reproduce the error(s).

Agreed regarding the actual delta compression.

Closing.

comment:8 Changed 2 years ago by Antoine Martin

Resolution: fixed
Status: closedreopened

Whilst testing vp9 (#832) and having decoder problems (or maybe encoder problems even..), I think I've hit a bug related to multi-delta:

2015-04-05 19:49:05,883 paint_with_video_decoder: \
    wid=1, vp9 decompression error on 44 bytes of picture data for 499x316 pixels using vpx.Decoder(vp9), \
    options={'speed': 83, 'frame': 2, 'quality': 67, 'csc': 'YUV444P', 'encoding': 'vp9'}
2015-04-05 19:49:06,251 draw error
Traceback (most recent call last):
  File "/usr/lib64/python2.7/site-packages/xpra/client/ui_client_base.py", line 1962, in _do_draw
    window.draw_region(x, y, width, height, coding, data, rowstride, packet_sequence, options, [record_decode_time])
  File "/usr/lib64/python2.7/site-packages/xpra/client/client_window_base.py", line 423, in draw_region
    self._backing.draw_region(x, y, width, height, coding, img_data, rowstride, options, callbacks)
  File "/usr/lib64/python2.7/site-packages/xpra/client/window_backing_base.py", line 472, in draw_region
    self.paint_with_video_decoder(VIDEO_DECODERS.get(coding), coding, img_data, x, y, width, height, options, callbacks)
  File "/usr/lib64/python2.7/site-packages/xpra/client/window_backing_base.py", line 376, in paint_with_video_decoder
    img = self._video_decoder.decompress_image(img_data, options)
  File "xpra/codecs/vpx/decoder.pyx", line 300, in xpra.codecs.vpx.decoder.Decoder.decompress_image (xpra/codecs/vpx/decoder.c:4578)
  File "/usr/lib64/python2.7/site-packages/xpra/codecs/codec_constants.py", line 61, in get_subsampling_divs
    raise Exception("invalid pixel format: %s" % pixel_format)
Exception: invalid pixel format: bucket

comment:9 Changed 2 years ago by Antoine Martin

Resolution: fixed
Status: reopenedclosed

comment:8 can be ignored, the problem was a dangling pointer caused by a bug in the vpx decoder, fixed in r8940.

comment:10 Changed 2 years ago by Antoine Martin

r9028 adds a new debug flag for delta compression: -d delta.

comment:11 Changed 2 years ago by Antoine Martin

r9063 ensures that we expire all delta buckets after 20 hits to prevent xor decay where the bucket just contains unhelpful noise.
The number of hits can be configured using:

XPRA_MAX_DELTA_HITS=20 xpra start ...

comment:12 Changed 2 years ago by Antoine Martin

This caused a bug: #861. (fixed in r9449)

Note: See TracTickets for help on using tickets.