decode jpeg to YUV

[Antoine Martin]

Split from #1423.

For some unknown reason, the YUV data loses some precision and so we decompress to RGB.
The opengl backend could use the YUV data directly. (the code is commented out as of r15062)

[Antoine Martin]

• r17543 + r17544 + r17545 + r17546 + r17547: minor updates, cleanups + decoding jpeg to YUV???P can be enabled with XPRA_JPEG_YUV=1
• r17548: decoding webp (#1694) to YUV can be enabled with XPRA_WEBP_YUV=1, defaults to 0 because we always compress from RGB and libwebp only currently supports YUV420P: the loss of quality is visible, even at very low quality settings

The type of colour shade that goes MIA is seen here: ticket:1423#comment:3.
webp doesn't have this problem (only the subsampling issue), so the paint code is unlikely to be the problem.

[Antoine Martin]

jpeg uses BT.601, but using full range values. Y, Cb and Cr all use the full range of 0-255, with 128 being "0" for Cb and Cr: ​Full-range Y’CbCr in JPEG: If you are ever concerned about correctness, the easiest way is to simply never think about full-range Y’CbCr?

So we'll need to use a different shader for those frames, and a new flag of some sort.

[Antoine Martin]

source for clamped shader

[Antoine Martin]

source for full range shader

[Antoine Martin]

Finally fixed in r18330. Decoding to YUV is now the default in r18331.
Thanks to ​wikipedia : YCbCr, we now use those authoritative matrix values in r18332.

Had to:

• compile the yuv16.cg shader source to verify that we can re-generate the shaders found in browser/xpra/trunk/src/xpra/client/gl/gl_colorspace_conversions.py
• then reverse engineer the values used in that file - found the correct matrix in ​Video Demystified:

  R = Y + 1.371 * (Cr - 128)
  G = Y - 0.698 * (Cr - 128) - 0.336 * (Cb - 128)
  B = Y + 1.732 * (Cb - 128)
  

That was hard because those values are divided by 219 in the shader we have, and I had to figure that out... 219 is the limited range used by this encoding: 235-16!
Also, modulo minor floating point number inaccuracies, they must have used values with more precision?

• now we can create a new shader, and use 255 to divide instead since the values we get use the full range

To compile those shaders, find the nvidia cdc compiler somewhere, then run:

cgc -profile arbfp1 -o yuv.pso yuv.cg
cgc -profile arbfp1 -o yuv16.pso yuv16.cg

[Antoine Martin]

edit title: we're not dealing with webp yuv decoding, see #1764

[Antoine Martin]

Caused a regression: #1769