Xpra: Ticket #487: webp artifacts with transparency

Split from ticket:385#comment:20

This affects all platforms, with or without opengl rendering. See screenshot: /raw-attachment/ticket/487/webp-transparency-artifacts.png

Probably a regression since I must have tested transparency when it was added to webp, but 0.10.x is also affected.



Thu, 09 Jan 2014 06:48:47 GMT - Antoine Martin: attachment set

shows the artifacts on transparent windows when using webp


Thu, 09 Jan 2014 06:49:35 GMT - Antoine Martin: owner, status, description changed


Thu, 09 Jan 2014 12:33:14 GMT - Antoine Martin:

Transparency support was added in r3522, and works ok as I expected. r3884, r4245 and r5091 don't change any of the actual encoder code, so the problem must come from somewhere else..

Bisecting:

So the problem comes from r4151, likely to be in the argb code, I tested byteswapping on RGB components, but maybe it's the (un)premultiply codepath..


Thu, 09 Jan 2014 15:55:39 GMT - Antoine Martin: milestone changed

This is a regression, needs to be in 0.11

Taking out the line:

return byte_buffer_to_buffer(unpremultiply_argb(img_data))

And returning the data as it is instead (premultiplied), the picture looks OK (albeit, with too much in the alpha channel)

What I don't understand is why this affects webp and not the other encodings, which also use the same codepath!? (ie: png via PIL also gives us a string buffer)


Fri, 10 Jan 2014 07:53:30 GMT - Antoine Martin: milestone changed

Since webp is buggy and not going to be used by default (and maybe we should even disable it?), re-scheduling this ticket until after #491


Sun, 09 Feb 2014 08:25:00 GMT - Antoine Martin: milestone changed

re-scheduling all webp stuff


Sat, 17 May 2014 14:19:06 GMT - Antoine Martin:

The problem: webp's simple API returns unpremultiplied alpha, which is not what we want and when we unpremultiply it again, it goes out of range... not sure about Pillow (the version in Fedora 20 does not seem to be able to decode webp..)

So, we have to:

Note: the corruption is not directly visible with the opengl backing because we now use the premultiplied alpha directly during rendering (it may just end up oversaturated instead)


Sun, 18 May 2014 10:10:30 GMT - Antoine Martin:

And... to make matters worse, webp decides if the picture has an alpha channel not using any API flags, but by inspecting the alpha component of every pixel in the input and deciding that if any values are not 0xFF then it is on.. WTH? And whose bright idea was this?


Sun, 18 May 2014 11:16:29 GMT - Antoine Martin: attachment set

clamp rgb values when unpremultiplying to prevent overflow


Sun, 18 May 2014 12:37:25 GMT - Antoine Martin:

Implemented a new "advanced API" webp decoder in r6509:

I'm not going to bother fixing the codepath for the other decoders (I have not tested to see if Pillow uses premultiplied alpha or not), you will just get the wrong transparency levels with those. (and eventually, maybe we should just drop them)


Sun, 18 May 2014 13:26:02 GMT - Antoine Martin:

r5510 clears out the alpha channel before calling the webp Encode function, not very efficient but it works.


Sun, 18 May 2014 14:18:37 GMT - Antoine Martin:

Ready for testing. Related to #419.

afarr: please check that webp works as primary encoding:

Important note: because of #419, the actual encoding used for each region of pixels we send may end up being different from the one we specify. The only way to verify what encoding is used is to turn on compression debugging with -d compress on the server and to manually verify what comes out.

Note: I am not applying the attachment/ticket/487/argb-clamp.patch because it would actually hide any overflow, and we want to see it and fix it instead.


Sun, 18 May 2014 14:18:53 GMT - Antoine Martin: owner, status changed

(re-assign)


Tue, 20 May 2014 20:27:29 GMT - J. Max Mena:

Tested with Windows 7 - 64 bit, Windows 8.1 - 32 bit, and Fedora 20 - 64 bit:

Of note, using both Windows and Fedora clients, running a high definition youtube video full screen with webp causes the framerate to slow down to a crawl. Otherwise, everything worked as expected.


Wed, 21 May 2014 13:33:08 GMT - Antoine Martin:

Thanks for testing, this reminds me of just how slow webp can be (see #419): it falls off a cliff in lossless mode, especially at low speed and at high resolutions (1080p and up). So I'll run some more tests tomorrow to ensure that we never use it as a temporary non-video encoding with those problematic combinations of options (this should already be the case, but best to triple check).

What was the resolution you tested at when fullscreen? 1080p? If so, unless you have other options set, it should have switched to high speed which should still be faster than png. Having xpra info of when it is really slow may help me figure things out.


Thu, 22 May 2014 14:24:21 GMT - Antoine Martin: status changed; resolution set

Re-tested some more today, and made some more tweaks (r6528, r6534, r6535), fixes (r6529, r6531) and improvements to the test code (r6530, r6532, r6533).

Closing.


Mon, 26 May 2014 05:29:38 GMT - Antoine Martin:

Note: in 0.14, the old python-webm is completely removed in favor of the new webp advanced codec, see r6555.


Sat, 23 Jan 2021 04:57:05 GMT - migration script:

this ticket has been moved to: https://github.com/Xpra-org/xpra/issues/487