Xpra: Ticket #1014: take video size into account for scoring codecs

The codecs are rated according to their speed and quality performance, but it may make sense to change which codec is used based on how big the video is: h265 and vp9 become more competitive at higher resolution. See ticket:832#comment:19.

Also means removing changing the "default" encoding to a new "auto" mode so that we can mix vp8/vp9 and h264 in the same window.

Wed, 28 Oct 2015 02:32:29 GMT - Antoine Martin: status, description changed

Sat, 27 Feb 2016 13:33:58 GMT - Antoine Martin: milestone changed


Tue, 12 Jul 2016 16:52:22 GMT - Antoine Martin: milestone changed

Milestone renamed

Sun, 11 Sep 2016 17:39:50 GMT - Antoine Martin: owner, status changed

Testing shows that x264 is still used for "small" dimensions, vp8 starts to win at higher resolutions. I'm not switching to vp9 because although it compresses better and decompresses faster, it is still slower than vp8 on the fastest setting. Note: vp8/vp9 doesn't have b-frames. (#800)

@afarr: you should be seeing vp8 encoding being used at higher video resolutions.

Mon, 12 Sep 2016 22:19:36 GMT - alas: owner changed

Tested with 1.0 r13673 fedora 23 server against 1.0 r13637 osx and win32 clients (hoping new client bits aren't essential).

With a 1440p display as a secondary monitor for the osx client, it was almost impossible to trigger the vp8 encoding, even playing videos full screen... though I was able to trigger by resizing windows near the full display size. I assume 1440p is still "smaller", and that this behavior was expected.

I did notice, though, that portions of the window never got a lossless refresh once the vp8 was triggered (I'll attach a screenshot, and leave it relatively large so some details can be discerned... looks like the video portion is encoding with h264, there are several orange boxed regions being encoded in rgb, but there are several other regions of the window, which seems to be largely a green-boxed vp8 region, which are still displaying lossily even after many minutes... will also attach an xpra info when I initially spotted, and a second taken some 15-20 minutes later with the same region still not refreshed).

With the win32 client on a 4 display, it was considerably easier to trigger the vp8 encoding. Some checking on size just to see if it's behaving about as expected.

Scrolling in the window sizes needed to trigger the vp8, I noticed that (even when the scrolling region ends up being encoded with h264 or jpeg) I'm seeing a fair amount of delay before any lossless refreshes are triggered. On further consideration though, I think I'll just attach one with the vp8 on the 4k display here, and the h264 and jpeg lossy xpra infos in #967.

If there doesn't seem to be anything suspicious in the slow refresh info outputs (or if there's something that should be broken out to another ticket), then I think this can be closed... vp8 is definitely being seen more often than previously and more often the larger the video region detected.

Mon, 12 Sep 2016 22:25:11 GMT - alas: attachment set

screenshot with paint boxes of vp8 encoded window not losslessly refreshed after 15+ minutes, with rgb & h264 regions that have been static as well

Mon, 12 Sep 2016 22:26:32 GMT - alas: attachment set

xpra info to go with screenshot, taken very shortly after page settled (with boxes indicated by screenshot)

Mon, 12 Sep 2016 22:28:03 GMT - alas: attachment set

another xpra info after some 15-20 minutes, window still painted the same (maybe video paused?, but no lossless refresh)

Mon, 12 Sep 2016 22:30:02 GMT - alas: attachment set

xpra info from early tests inducing vp8 on 4K display by scrolling, lossless refreshes taking a long half-second to trigger (expected with such large windows?)

Tue, 13 Sep 2016 04:26:02 GMT - Antoine Martin: owner changed

I assume 1440p is still "smaller", and that this behavior was expected.

The heuristics take various things into account, 1440p is near the limit in most cases. (x264 still scores higher for quality and speed).

I think we can close this ticket. FYI: as of r13684, the "auto" encoding option is now visible in the tray menu, the command line help, etc..

Scrolling should be using the new "scroll" encoding: #1232 and not video.

Refresh issues are unlikely to be caused by the changes in this ticket: the refresh logic does not differentiate between vp8 and h264 video encodings. Please include the "-d regionrefresh" for video refresh issues only, "-d refresh" for refresh issues in other areas of the page. (in a separate ticket unless going back to a pre-r13662 version fixes the issue).

Wed, 28 Sep 2016 01:13:13 GMT - alas: status changed; resolution set

Both details noted (and hopefully will be remembered when the time comes.

I'll take the liberty of closing this.

Sat, 23 Jan 2021 05:12:32 GMT - migration script:

this ticket has been moved to: https://github.com/Xpra-org/xpra/issues/1014