Colourspace conversion step - CSC

Before passing the pixels to the video encoder, we may or may not include a colourspace conversion step. (some newer versions of x264 support BGRA pixels as input directly)

Information and Diagnostics

The pixel format used as input as well as the CSC module chosen to do this conversion work can be found via "xpra info". These should be chosen automatically based on the desired speed/quality settings. Note that the CSC step may degrade the quality of the picture (YUV444P mode does not, but YUV422P and YUV420P do).

CSC Modules

On the server side we currently support three modules for doing this step:

• ​csc_swscale which uses ​FFmpeg's swscale
• ​csc_opencl which uses ​PyOpenCL
• ​csc_nvcuda which uses ​PyCUDA

See below for some performance samples.

On the client side, if a CSC step is needed, it will use swscale. The OpenGL mode does not need a CSC step as it can display YUV pixels directly on screen.

Specify a module or mode

To choose a specific CSC module, use:

XPRA_CSC_TYPE=name xpra ...

Where name is one of the csc modules above.

One can also force the use a specific CSC mode:

XPRA_FORCE_CSC_MODE=cscmode xpra ...

Where cscmode is one of: YUV420P, YUV422P, YUV444P.

Some modules can be configured further, for example the csc_opencl module can select which device type is used as backend using:

XPRA_OPENCL_DEVICE_TYPE=devicetype xpra start ...

Where devicetype is either GPU or CPU.
Or even which specific OpenCL device is used:

XPRA_OPENCL_DEVICE_NAME="GeForce GTX 760" xpra ...

Scaling

As part of the colourspace conversion step, we can also downscale the frame if the XPRA_SCALING environment variable is unset or set to "1", and either:

• the application requests scaling using the _XPRA_SCALING X11 window property (specified as a 32-bit fraction, 16-bits for each)
• the XPRA_SCALING_HARDCODED environment variable is set on the server, using one of those 2 forms (the second one is preferred):
  • XPRA_SCALING_HARDCODED=N to downscale by N
  • XPRA_SCALING_HARDCODED=M:N to downscale by the fraction M/N
• automatically, if the frame size is bigger than 1MPixels and the quality is low whilst the speed is high

Note: if the automatic heuristics turn on frame downscaling, this may introduce a colourspace conversion step where none was necessary before (when the encoder was previously handling RGB directly).

CSC Performance

You can get your own figures by running the tests:

• xpra.codecs.test_csc_opencl (you can use environment variables to choose the OpenCL backend to use)
• xpra.codecs.test_csc_swscale
• xpra.codecs.test_csc_nvcuda.py

Measured with r4272 (results are in MPixels/s):

• 1920x1080 RGB to YUV???P:

Module	CPU/GPU	YUV420P	YUV422P	YUV444P
swscale	AMD FX 8150	142	182	151
swscale	AMD X4 945	120	165	131
swscale	AMD X2 260	124	170	140
swscale	Intel Core i3-3110M	164	229	181
swscale	2xIntel Xeon E5-2670	215	322	253
CUDA-Nvidia	AMD X4 945 + GTS 450	366	341	290
CUDA-Nvidia	2xIntel Xeon E5-2670 / 2xK1	173	177	160
OpenCL-Nvidia	AMD FX8150 + GTX 760	345	303	254
OpenCL-Nvidia	AMD X4 945 + GTS 450	357	303	260
OpenCL-Nvidia	2xIntel Xeon E5-2670 / 2xK1	210	211	192
OpenCL-Nvidia	Intel Xeon E5-2620 / GTX 650ti	502	457	399
OpenCL-Intel	AMD FX 8150	129	114	119
OpenCL-Intel	Intel Core i3-3110M	141	92	53
OpenCL-Intel	2xIntel Xeon E5-2670	472	412	263
OpenCL-Intel	Intel Xeon E5-2620	254	213	131
OpenCL-AMD	AMD FX 8150 + Radeon HD5450	110	49	42
OpenCL-AMD	AMD FX 8150	93	79	76
OpenCL-AMD	AMD X4 945	63	54	53
OpenCL-AMD	AMD M300	14	12	12
OpenCL-AMD	AMD X2 + Radeon HD5450	151	61	57
OpenCL-AMD	AMD X2	15	14	11
OpenCL-AMD	Intel Core i3-3110M	71	58	63
OpenCL-Apple	Intel Core2Duo P8600 + GeForce? 320	22	28	22

• 1920x1080 RGB to GBR (simple byte swapping):

Module	CPU/GPU	MPixels/s
swscale	AMD FX 8150	718
swscale	AMD X4 945	524
swscale	AMD X2 260	582
swscale	Intel Core i3-3110M	550
swscale	2xIntel Xeon E5-2670	758

• 1920x1080 YUV???P to BGR(X):

Module	CPU/GPU	YUV420P	YUV422P	YUV444P
swscale	AMD FX 8150	381	406	416
swscale	AMD X4 945	369	323	237
swscale	AMD X2 260	312	255	330
swscale	Intel Core i3-3110M	350	309	310
swscale	2xIntel Xeon E5-2670	177	168	163
CUDA-Nvidia	AMD X4 945 + GTS 450	202	191	180
CUDA-Nvidia	2xIntel Xeon E5-2670 / 2xK1	180	155	151
OpenCL-Nvidia	AMD FX 8150 + GTX 760	331	289	257
OpenCL-Nvidia	AMD X4 945 + GTS 450	?	?	?
OpenCL-Nvidia	Intel Xeon E5-2620 / GTX 650ti	458	377	358
OpenCL-Nvidia	2xIntel Xeon E5-2670 / 2xK1	190	165	148
OpenCL-Intel	AMD FX 8150	96	70	67
OpenCL-Intel	Intel Core i3-3110M	82	88	87
OpenCL-Intel	Intel Xeon E5-2620	146	123	116
OpenCL-Intel	2xIntel Xeon E5-2670	265	271	268
OpenCL-AMD	AMD FX 8150 + Radeon HD5450	84	82	70
OpenCL-AMD	AMD FX 8150	60	55	47
OpenCL-AMD	AMD X4 945	54	51	50
OpenCL-AMD	AMD M300	11	9	7
OpenCL-AMD	AMD X2 260 + Radeon HD5450	107	98	98
OpenCL-AMD	AMD X2 260	11	10	7
OpenCL-AMD	Intel Core i3-3110M	60	56	58