xpra icon
Bug tracker and wiki

Opened 5 years ago

Closed 5 years ago

#796 closed enhancement (fixed)

YUV444 and lossless mode with NVENC4

Reported by: Antoine Martin Owned by: alas
Priority: major Milestone: 0.15
Component: encodings Version: trunk
Keywords: nvenc Cc:

Description

Split from #466 and #653.

Like it says on the tin.

Attachments (3)

nvenc-yuv444p-v2.patch (30.7 KB) - added by Antoine Martin 5 years ago.
work in progress
quality-speed-refactoring.patch (16.6 KB) - added by Antoine Martin 5 years ago.
in order to be able to re-initialize the encoder, we need both speed and quality updated at the same time
quality-speed-refactoring-v2.patch (27.6 KB) - added by Antoine Martin 5 years ago.
now split init functions so we can re-use them

Download all attachments as: .zip

Change History (11)

Changed 5 years ago by Antoine Martin

Attachment: nvenc-yuv444p-v2.patch added

work in progress

comment:1 Changed 5 years ago by Antoine Martin

Status: newassigned

I just don't see what we're doing wrong.. But only the "Y" plane gets encoded.

The ffmpeg nvenc code does pretty much the same thing: the input buffer is YUV444 with a dstPitch*dst_h offset.

comment:2 Changed 5 years ago by Antoine Martin

Took a while but the fix was simple (forgot to set the buffer format when registering it - wasn't set in previous NVENC SDK versions).

YUV444 works correctly as of r8571.

Still TODO:

  • some client decoding errors on the first frame, ie:
    avcodec error decoding 48538 bytes of data with \
        options={'quality': 74, 'encoding': 'h264', 'speed': 39, 'frame': 0, \
                 'csc': 'YUV444P', 'pts': 0} (frame 0, step 1 of 1)
    
  • lossless mode
  • compare performance with NV12 mode
  • we cannot switch from NV12 to YUV444 without doing an encoder re-init..
Last edited 5 years ago by Antoine Martin (previous) (diff)

comment:3 Changed 5 years ago by Antoine Martin

lossless mode support added in r8604.

BTW, there's a new codec supported (maybe h265 which is meant to be supported in SDK v5 - but which is completely undocumented?):

[0] H264 : 6BC82762-4E63-4CA4-AA85-1E50F321F6BF
[1] None : 790CDC88-4522-4D7B-9425-BDA9975F7603

Changed 5 years ago by Antoine Martin

in order to be able to re-initialize the encoder, we need both speed and quality updated at the same time

Changed 5 years ago by Antoine Martin

now split init functions so we can re-use them

comment:4 Changed 5 years ago by Antoine Martin

As of r8606 we can do an encoder re-init on the fly, which only takes a few milliseconds. r8608 allows us to use lossless mode properly, and also now from the system tray menu for h264.

From my limited testing: YUV444 performance is very good, only 10 to 15% slower than NV12, lossless mode is also impressive: fast and we just use more space.

Note: you need the "new" set of license keys to be able to use more than 2 encoding sessions on the same card... and we should probably detect that this is the case and tune the load balancing accordingly.

comment:5 Changed 5 years ago by Antoine Martin

Owner: changed from Antoine Martin to Smo
Status: assignednew

The decoding problems are fixed in r8620 (was caused by B frames - we will deal with this better eventually, see #800).

Ready for testing. Some defaults to be aware of:

  • we switch to YUV444 over 85% quality (but with edge resistance of 10 points both ways) - is this the right value?
    XPRA_NVENC_YUV444_THRESHOLD=85
    
  • we use lossless at 100% quality (which means you have to change this value or use the command line or tray to be able to use lossless mode):
    XPRA_NVENC_LOSSLESS_THRESHOLD=100
    

Lossless mode isn't as useful as before, since auto-refresh is now meant to do a decent job of eventually fixing lossy paints. But since it is reasonably cheap to use, maybe we should use it more?

The current pixel format and lossless flag can be seen with (including thresholds as of r8621):

xpra info | grep encoder

@smo: please re-assign to afarr once you've got it setup?

comment:6 Changed 5 years ago by Smo

Owner: changed from Smo to alas

I have a reasonable setup now that afarr can test this on.

Re-assigning to him.

comment:7 Changed 5 years ago by alas

Smo's setup against a windows client 0.15.0 r8002 works as expected.

When stretching a firefox window as some video was initiating, there was a second with a yellow and red misrendering, but it quickly solved itself and I couldn't reproduce.

Setting XPRA_NVENC_LOSSLESS_THRESHOLD=100 server side, then setting quality to lossless I was able to toggle lossless to on (window[2].encoder.lossless=1) ... it took several seconds for the heuristics to adjust, but once it had done so, the lossless (and the x264 encoder when I changed the quality to merely Best) rendered exceptionally, even when I stretched the window size to gargantuan proportions on a 4K monitor (window[2].size=(2446, 1883)).

Testing with an osx client, 0.15.0 r8006, it was a little harder to set the quality to 100/lossless (had to use the control channel, since osx menus don't have any hooks for adjusting quality). Again, it took several seconds to adjust, with some odd green and black misrendering and a few seconds more in which only a subregion was responsive to scrolling and such... but once it adjusted itself it also behaved very well.

comment:8 Changed 5 years ago by alas

Resolution: fixed
Status: newclosed

Ok, this seems to be good enough for current purposes. Closing.

Note: See TracTickets for help on using tickets.