xpra icon
Bug tracker and wiki

Opened 11 months ago

Closed 11 months ago

Last modified 10 months ago

#1694 closed task (fixed)

re-add webp codec

Reported by: Antoine Martin Owned by: J. Max Mena
Priority: major Milestone: 2.2
Component: encodings Version: trunk
Keywords: Cc:

Description

It was removed in r14508 as part of a codec cleanup.
The main problems with it were that:

  • the distro libraries were often way too old - which is still the case in Debian land (no surprises there)
  • the performance was inadequate in some cases

After playing with it on Fedora 27 with libwebp 0.6.0, it seems that there are settings we can use that will give us better compression and better performance than png. And we should be able to avoid using it for the cases where it isn't adequate.

We need something better than png to avoid using up all the bandwidth with lossless refresh. (see ticket:999#comment:20 for details)

Attachments (2)

webp-fast.patch (38.5 KB) - added by Antoine Martin 11 months ago.
re-adds webp and makes it fast!
load-pillow-last.patch (2.2 KB) - added by Antoine Martin 11 months ago.
I don't think this is needed as long as pillow doesn't link with libwebp

Download all attachments as: .zip

Change History (10)

comment:1 Changed 11 months ago by Antoine Martin

Status: newassigned

A very quick and dirty test with glxspheres at 1240x900 with speed=100 and quality=100 shows:

  • webp: ~35ms, 8.5% compression ratio (~375KB)
  • png: ~55ms, ~12.5% compression ratio (~540KB)

The pillow webp code is much slower (~100ms), but it also compresses better: the size is just 2.5%! So clearly we can do better with this codec, I'm just not sure how yet. Undoing all the configuration we do, the same image takes 2 seconds to compress! (and the gains really aren't worth it: only 6.5% compression ratio)

Here are the settings that we use when trying to go for maximum lossless compression:

webp.compress used config: {'emulate_jpeg_size': 0, 'filter_sharpness': 0, 'thread_level': 1, \
    'autofilter': 1, 'pass': 1, 'alpha_filtering': 0, 'sns_strength': 0, \
    'target_PSNR': 0.0, 'low_memory': 0, 'show_compressed': 0, 'target_size': 0, 'method': 0, \
    'filter_type': 1, 'preprocessing': 0, 'alpha_compression': 0, 'alpha_quality': 0, \
    'partition_limit': 0, 'lossless': 1, 'partitions': 3, 'filter_strength': 0, 'image_hint': 'graph', 'segments': 2}

The only major problem with this is that my xterm window comes out all black. (but the glxspheres window looks fine!)

Sadly, not many browsers support webp, but chrome does and we should be able to detect webp support.

Last edited 11 months ago by Antoine Martin (previous) (diff)

comment:2 Changed 11 months ago by Antoine Martin

doh: we were forcing speed=100 when quality=100 (lossless).

Hardcoding the "method" to 0 (fast) and lossless mode (quality=100), we can fine tune the time spent compressing by setting webp's "quality" attribute (which has nothing to do with quality in this case..) - but the resulting output size remains pretty much the same!
Setting the method to 1 or higher and the compression becomes too slow (250ms or more), for only 1 percentage point of extra compression!
The reason why Pillow was doing better is that it wasn't doing true lossless as the quality value was clamped to 99 in our code!

Changed 11 months ago by Antoine Martin

Attachment: webp-fast.patch added

re-adds webp and makes it fast!

comment:3 Changed 11 months ago by Antoine Martin

Owner: changed from Antoine Martin to J. Max Mena
Status: assignednew
  • r17467 adds support for it in the html5 client,
  • r17468 re-adds the "webp" encoding (+r17469 build fixup for win32), and since the speed and quality settings are now much improved (no thanks to the complete lack of documentation on what those knobs do), we will use webp ahead of png in most cases. This should help with #999.
  • r17471 + r17483: use webp more
  • r17473 + r17474: build pillow without webp on centos7 to prevent conflicts between the host libwebp library and our private library - note: we no longer use private libraries for webp on Fedora (the spec file refuses to build there)

In debug mode, webp is shown using a pink contour colour as before.
All builds should have the correct dependencies added, including macos and win32 builds.

@maxmylyn: mostly a FYI, testing wouldn't hurt but feel free to just close.

Last edited 11 months ago by Antoine Martin (previous) (diff)

Changed 11 months ago by Antoine Martin

Attachment: load-pillow-last.patch added

I don't think this is needed as long as pillow doesn't link with libwebp

comment:4 Changed 11 months ago by Antoine Martin

Updates:

  • r17483: can use webp at high res
  • r17492: add threshold so we use webp in lossless mode for quality above 75%

Interesting thing I spotted during bandwidth tuning for #999, webp can compress the xterm in lossless mode 40 times better than when using lossy compression!

compress:  10.5ms for  499x316  pixels at    0,0    for wid=1     using   webp \
    with ratio  23.0%  (  615KB to   141KB), sequence    74, \
    client_options={'quality': 99, 'speed': 97, 'rgb_format': 'RGBX'}
compress:   2.6ms for  499x316  pixels at    0,0    for wid=1     using   webp \
    with ratio   0.6%  (  615KB to     3KB), sequence    75, \
    client_options={'quality': 100, 'speed': 50, 'rgb_format': 'RGBX'}

Time for some window hints to make better guesses at what compression to use: #1699.

comment:5 Changed 11 months ago by Antoine Martin

r17541: libwebp 0.6.1

comment:6 Changed 11 months ago by J. Max Mena

Resolution: fixed
Status: newclosed

Noted. I'll also make a note to myself to add it to the test box.

Closing.

comment:7 Changed 10 months ago by Antoine Martin

Found a bug with the non-opengl rendering: the red and blue channels would end up swapped.
That's because the webp encoder compresses BGRA (or BGRX) as input, so if it was fed "RGBA" (or "RGBX") then we have to swap the channels when decoding.
The correct fix is a bit long but it does make sense: r17688.

comment:8 Changed 10 months ago by Antoine Martin

r17697 is a more complete end-to-end fix: the client tells the server which RGB modes it can handle for webp.

This fixes the potential problem where the HTML5 client would be asked to decode a swapped-channels-webp image. This could happen if we had a server that fed "RGBX" (or "RGBA") to the encoder instead of "BGRX" (or "BGRA"). ie: maybe on a little endian arch?

Note: See TracTickets for help on using tickets.