xpra icon
Bug tracker and wiki

Version 9 (modified by Antoine Martin, 7 years ago) (diff)

--

Picture Encodings

This page is strongly related to WindowRefresh, the [wiki/ClientRendering client rendering] is also relevant.

Introduction

Xpra supports a number of picture encodings, provided you have the required libraries installed. Even then, the features of each encoding may vary based on the version of the libraries and other dependencies, both client and server side.
Here is the list as of v0.8

  • lossless encodings:
    • rgb24 (+zlib)
    • png
  • lossy encodings:
    • jpeg
    • webm
    • vpx
    • x264

Choosing an Encoding

The best thing to do is to try them all and choose the one that provides the best results. Here are some rough guidelines:

  • on LANs with 100MBit/s or higher, use fast lossy encodings: either rgb24 or png
  • otherwise, choose x264 and tune the speed/quality to suit your needs (see below)

The other encodings are somewhat less useful:

  • vpx is similar to x264 but it does not support speed and quality tuning
  • webm is single image subset of vpx, and therefore lacks intra-frame compression - but latency is good
  • jpeg gives lower size/quality than other lossy encodings

x264 specifics

(see wikipedia H.264) x264 is the encoding that supports the most options and tunings. Quality/speed and minimum quality/speed can be set via the command line (and the latter can also be changed at runtime via the tray applet) Note that even when using the x264 encoding, some small screen updates may get sent as png or rgb24 to save time/bandwidth (not encoding a full frame).

Profiles

(see wikipedia h264 profiles) x264 supports the following profiles: baseline, main, high, high10, high422 and high444.

At present, we use high or better by default unless the client overrides it.

Quality Option

What this does should be obvious (in x264 speak, this controls the rc.f_rf_constant parameter), but it is more complicated than you think:

  • we support 3 different colourspace modes: YUV420, YUV422 and YUV444 and this affects the quality of the picture too. (YUV420 is used for lowest quality settings). YUV444 uses roughly twice as much bandwidth as YUV420. Switching from one mode to another is expensive, as we then need to send a new key frame and re-initialize both the encoder and the decoder - so the thresholds for going up to the next mode are not the same as the thresholds for going down to the next mode (prevents a yoyo effect).
  • not all modes are supported by all profiles, so we need to switch to a different profile to support YUV422 (high422 or high444) and YUV444 (high444 only).
  • some builds against older versions of libav/ffmpeg only support YUV420, see x264-limited-csc.patch

Speed Option

This option, shown as "latency" via the tray menu, controls how hard the encoder is going to work at compressing the picture. Working harder means lower bandwidth, but also higher latency. x264 supports the following speed settings: ultrafast, superfast, veryfast, faster, fast, medium, slow, slower, veryslow and placebo. xpra maps the 100% to 0% speed option from superfast to slower only. veryslow and placebo are not particularly useful (diminishing returns: much much slower and without sufficient savings for real-time use). ultrafast is not available because setting the encoder to this setting has side-effects which prevents other settings from behaving as they should if the option is later changed. And in theory, superfast is almost as good. To test ultrafast, try x264-allow-ultrafast.patch

Tuning

The best way to choose the right options is through wiki/Testing. Though you may find some good illustrations online to give you an idea of the trade offs. (ie: fps vs noise, fps vs size) Be aware that the lossless auto-refresh will trigger a new frame at 95% quality, which often means re-initializing the whole codec, and this is expensive so increasing this delay may help.

Notes

When comparing performance, make sure that you use the right metrics... The number of updates per second is not always a good one (if there are more small regions, this can be a good or a bad thing), more examples here: Misleading Statistics