Colourspace conversion step - CSC

Before passing the pixels to the video encoder, we may or may not include a colourspace conversion step. (some newer versions of x264 support BGRA pixels as input directly)

Information and Diagnostics

The pixel format used as input as well as the CSC module chosen to do this conversion work can be found via "xpra info". These should be chosen automatically based on the desired speed/quality settings. Note that the CSC step may degrade the quality of the picture (YUV444P mode does not, but YUV422P and YUV420P do).

CSC Modules

On the server side we currently support three modules for doing this step:

• ​csc_swscale which uses ​FFmpeg's swscale
• ​csc_opencl which uses ​PyOpenCL
• ​csc_nvcuda which uses ​PyCUDA

On the client side, if a CSC step is needed, it will use swscale. The OpenGL mode does not need a CSC step as it can display YUV pixels directly on screen.

Choosing a CSC module

The best way to choose a CSC module is to test them all and compare them.

Specify a module or mode

To choose a specific CSC module, use:

XPRA_CSC_TYPE=name xpra ...

Where name is one of the csc modules above.

One can also force the use a specific CSC mode:

XPRA_FORCE_CSC_MODE=cscmode xpra ...

Where cscmode is one of: YUV420P, YUV422P, YUV444P.

Some modules can be configured further, for example the csc_opencl module can select which device type is used as backend using:

XPRA_OPENCL_DEVICE_TYPE=devicetype xpra start ...

Where devicetype is either GPU or CPU.
Or choosing the OpenCL implementation to use when more than one are installed:

XPRA_OPENCL_PLATFORM="Intel" xpra ...

Or even which specific OpenCL device is used when more than one is available:

XPRA_OPENCL_DEVICE_NAME="GeForce GTX 760" xpra ...

Scaling

As part of the colourspace conversion step, we can also downscale the frame if the XPRA_SCALING environment variable is unset or set to "1", and either:

• the application requests scaling using the _XPRA_SCALING X11 window property (specified as a 32-bit fraction, 16-bits for each)
• the XPRA_SCALING_HARDCODED environment variable is set on the server, using one of those 2 forms (the second one is preferred):
  • XPRA_SCALING_HARDCODED=N to downscale by N
  • XPRA_SCALING_HARDCODED=M:N to downscale by the fraction M/N
• automatically, using some heuristics (the bigger the picture size and the lower the quality + the higher the speed - the more likely it is that it will be enabled)

Note: if the automatic heuristics turn on frame downscaling, this may introduce a colourspace conversion step where none was necessary before (when the encoder was previously handling RGB directly).

The scaling can be handled by some encoders natively, nvenc does.