xpra icon
Bug tracker and wiki

Opened 11 months ago

Closed 7 months ago

#1257 closed enhancement (worksforme)

better x264 tuning for type of content

Reported by: Antoine Martin Owned by: Antoine Martin
Priority: major Milestone: 1.0
Component: encodings Version: trunk
Keywords: x264 Cc:

Description

Related to #1107, #1232 and #800.

We (almost) already know if the content we are encoding using the video encoders is really real video or just fast moving.
This is what we use to enable b-frames (#800), we should also use this to tune x264 better.

Change History (11)

comment:1 Changed 11 months ago by Antoine Martin

Status: newassigned
  • r13020: use the simplest motion search with non-video content (DIA - diagonal)
  • r13021 + r13022: make it possible to use different x264 "tune" option

For details on tune, see: Explanation of x264 tune.

ie:

XPRA_B_FRAMES=0 \
XPRA_FORCE_CSC_MODE=YUV420P \
XPRA_X264_PROFILE=high10 \
XPRA_X264_TUNE=film \
    xpra --bind-tcp=0.0.0.0:10000 start --speed=100 -d x264

Will use YUV420P csc mode, high10 profile, film tune with ultrafast preset (speed).
In this case, using "film" without b-frames is not particularly useful!

The main question here is to figure out what is the best setting to use, both for video and non-video content.
There are going to be many quality vs cpu vs bandwidth vs latency trade offs to choose from, so maybe using x264 directly from the command line would be quicker.

comment:2 Changed 10 months ago by Antoine Martin

As of r13051, we are also more likely to use colour subsampling for video regions, and less likely to use it for other areas (the rest of the window).

comment:3 Changed 10 months ago by Antoine Martin

As of r13052, we will use "film" tuning (intended for high-bitrate/high-quality movie content. Lower deblocking is used here) for video regions, and "zerolatency" (optimization for fast encoding and low latency streaming) for everything else.

Here's the "-d x264" output for a video region:

using default profile=high444
x264 context=0x7fabf40497c0,    BGRX  300x300  preset=faster, profile=high444, tune=film
 me=HEX, me_range=16, mv_range=-1, opencl=False, b-frames=1, max delayed frames=1
 vfr-input=0, lookahead=0, sync-lookahead=-1, mb-tree=1, bframe-adaptive=FAST
 threads=auto, sliced-threads=True
x264 encode 1 delayed frames after 0
x264 encode frame 0 as  IDR slice with 7 nals, total   15737 bytes, keyframe=True , delayed=1
x264 encode frame 1 as    P slice with 4 nals, total   12211 bytes, keyframe=False, delayed=1
x264 encode frame 2 as    B slice with 4 nals, total    7849 bytes, keyframe=False, delayed=1
x264 encode frame 3 as    P slice with 4 nals, total   12390 bytes, keyframe=False, delayed=1
x264 encode frame 4 as    B slice with 4 nals, total    7695 bytes, keyframe=False, delayed=1

And for non-video (no b-frames):

using default profile=high444
x264 context=0x7efc480151c0,    BGRX  498x316  quality=99, speed=74, source=unknown
 preset=faster, profile=high444, tune=zerolatency
 me=DIA, me_range=16, mv_range=-1, opencl=False, b-frames=0, max delayed frames=0
 vfr-input=0, lookahead=-1, sync-lookahead=0, mb-tree=0, bframe-adaptive=FAST
 threads=auto, sliced-threads=True
x264 encode frame     0 as  IDR slice with 7 nals, total  124041 bytes, keyframe=True , delayed=0
x264 encode frame     1 as    P slice with 4 nals, total     154 bytes, keyframe=False, delayed=0
x264 encode frame     2 as    P slice with 4 nals, total     278 bytes, keyframe=False, delayed=0
x264 encode frame     3 as    P slice with 4 nals, total     109 bytes, keyframe=False, delayed=0

This should be tested together with #800 as there is a lot of overlap.

comment:4 Changed 10 months ago by Antoine Martin

r13056 allows us to detect the whole window as a video region, so that we will correctly use colour subsampling and more aggressive compression settings for full screen video (or anything full screen with high fps, like glxgears).

If this makes us wrongly detect the whole window as video area, we may need to tweak the video region detection code #967 (#410 - see also #1135, #1060)

r13058 also exposes the tune option via xpra info:

$ xpra info | grep tune
window.2.encoder.tune=film
Last edited 10 months ago by Antoine Martin (previous) (diff)

comment:5 Changed 10 months ago by Antoine Martin

Owner: changed from Antoine Martin to alas
Status: assignednew

Ready for testing, including b-frames from #800.

We want to:

  • verify that when we have a video region, we use "film" tuning and enable b-frames, non-video areas that still use h264 should still be using "zerolatency" (those should be rare)
  • ideally, measure the bandwidth savings and cpu costs of those new settings - do we need to adjust the speed (or even quality) setting to compensate?
  • verify that this doesn't negatively impact av-sync (#835 #1164)

Note: if we end up with a blurry window and delayed paints because the whole window is mistakenly identified as a video region then this needs to go in #967, not here.

comment:6 Changed 10 months ago by maxmylyn

Owner: changed from alas to Antoine Martin

Using a Fedora 23 trunk r13086 built from source server and client:

  • When watching a YouTube? video, or something with VLC, I verified that the film tuning is being used. However, typing quickly into a text box (like Trac tickets...) does trigger an h264 region, also using film, not zerolatency.
  • AV Sync does not appear to be affected. It's still very good - under 20ms off from what I can tell.


Note: if we end up with a blurry window and delayed paints because the whole window is mistakenly identified as a video region then this needs to go in #967, not here.


I do see this, will leave a comment in #967.



ideally, measure the bandwidth savings and cpu costs of those new settings - do we need to adjust the speed (or even quality) setting to compensate?


I'll talk about this with Smo and see what he thinks. Last I heard he was still setting up his testing environment. We also got a new hire and she should be able to do this once she's all set up.

comment:7 Changed 10 months ago by Antoine Martin

Owner: changed from Antoine Martin to maxmylyn

However, typing quickly into a text box (like Trac tickets...) does trigger an h264 region, also using film, not zerolatency.


Anything that is mistakenly detected as a video region will end up using film.
You must have had a video region detected at that point, you can verify with xpra info | grep region.
So unless this is not the case, it does not belong here (x264 tuning) but in #967 (video region detection).



Last I heard he was still setting up his testing environment.


Note: you will need to run one or more sets of tests with extra environment variables to compare with and without b-frames and tuning.

comment:8 Changed 9 months ago by Antoine Martin

r13436 further lowers the quality of "true" video (more than 10 frames per second, and at least 20 frames have been processed using the given video region)
As of r13439, we also only enable "b-frames" and the "video" encoder tuning for "true" video.
See also #1232: scrolling should now be detected and compressed better. There is now more information in

$ xpra info | grep subregion
$ xpra info | grep -e "window\.[0-9]*\.encoder.*"

comment:9 Changed 9 months ago by Antoine Martin

See ticket:1232#comment:16 which now exposes the "video-mode" flag for "true" video. (>10fps, etc..)

As of r13775, we can switch to and from "true" video on the fly, see ticket:800#comment:25

Last edited 8 months ago by Antoine Martin (previous) (diff)

comment:10 Changed 7 months ago by maxmylyn

Owner: changed from maxmylyn to Antoine Martin

Alright, so I think it's time to make a decision on this one.

It looks like we (afarr and myself) aren't gonna get someone who can write the tests, and aren't going to be given the time to write them ourselves.

So, @antoine - what would you like to do with this ticket? The x264 tuning works well in my experience, and I haven't had any issues with it. We can hand it to Smo, in the hopes that he will have time to include it in his tests? Probably not the best idea as he's already busy.

comment:11 Changed 7 months ago by Antoine Martin

Resolution: worksforme
Status: newclosed

Closing and hoping for the best :/

Note: See TracTickets for help on using tickets.