The x264 encoding supports many options and tunings. It is the default encoding used when it is available because it offers the best performance and compression out of the box, unless you have NVENC. Hopefully, this will one day be superseded by x265 support (#445), vp9 (#464) or daala (#454). Until then, it is the best encoding you can use with xpra.
Quality/speed and minimum quality/speed can be set via the command line or via the tray menu. Note that even when using the x264 encoding, some small screen updates may get sent as png or rgb24 to save time/bandwidth (not encoding a full frame).
Recent versions of x264 support
RGB pixels directly as input, which removes the need for a CSC step on the server, and the OpenGL rendering in the client also handles
RGB data directly - which allows for an even faster forwarding of pixels to the client's screen.
`Note: at lower quality settings, either automatically or manually selected, a CSC step may be introduced to reduce bandwidth.
When building xpra from source,
x264 is included by default.
For encoding support (server), you only need the
x264 library and headers installed.
Please note: the
libav fork of ffmpeg requires a number of patches to be applied to the source for compatibility - failure to apply them will result in crashes.
Unless you have also installed nvenc,
x264 will be automatically used when you select the
h264 encoding. Alternatively, you can use the
--video-encoders=x264 option to ensure that
x264 is the only video encoder loaded in the server.
The following wiki pages are also relevant:
(see wikipedia h264 profiles)
x264 supports the following profiles:
At present, we use
high or better by default unless the client overrides it.
What this does should be obvious (in x264 speak, this controls the
rc.f_rf_constant parameter), but it is more complicated than you think:
- we support 3 different colourspace modes: YUV420, YUV422 and YUV444 and this affects the quality of the picture too. (YUV420 is used for lowest quality settings).
YUV444uses roughly twice as much bandwidth as
YUV420. Switching from one mode to another is expensive, as we then need to send a new key frame and re-initialize both the encoder and the decoder - so the thresholds for going up to the next mode are not the same as the thresholds for going down to the next mode (prevents a yoyo effect).
- not all modes are supported by all profiles, so we need to switch to a different profile to support
- some builds against older versions of libav/ffmpeg only support
YUV420, see x264-limited-csc.patch
This option, shown as "latency" via the tray menu, controls how hard the encoder is going to work at compressing the picture. Working harder means lower bandwidth, but also higher latency.
x264 supports the following speed settings:
xpra maps the 100% to 0% speed option from
placebo are not particularly useful (diminishing returns: much much slower and without sufficient savings for real-time use).
ultrafast is only available when setting the encoder speed to 100% manually (via the command line or UI) - note that this setting has side-effects which prevents other settings from behaving as they should if the option is later changed.
x264, especially on the decoding side via
avcodec, has provided an almost constant stream of bugs:
x264 encoder threading related memory corruption and crash
client memory leak
Crash when connecting with x264 encoding
draw decode failure when attaching to older remote
fix support for libav 9.8
AVFrameWrapper falling out of scope before being freed by avcodec
Ideas for future Work
- finding the actual encoder dimension limits, which are currently hardcoded since there are no values in the header files: creating encoding contexts is pretty cheap, so we should be able to detect the limit by trial and error
- better tuning: most default options are targeted at video
- detect application profile (
browser) and tune settings
- motion algorithm improvements:
- tweak motion range (see merange)
- use a better motion algorithm where appropriate:
exhaustive search algorithm) is expensive,
diamond search) is a better fit for traditional scrolling
- write one where we can feed the information ourselves (assuming we do get it somehow)
input-range(see X264 Settings: input-range
- allow intra frames when dealing with high fps: buffer one frame, we must schedule a timeout to ensure the buffered frames do get sent. (and force an IDR frame to flush)