xpra icon
Bug tracker and wiki

Opened 11 months ago

Closed 4 months ago

Last modified 6 weeks ago

#2048 closed enhancement (worksforme)

nvenc startup is slow - send replacement jpeg frame

Reported by: Antoine Martin Owned by: Nathan Hallquist
Priority: major Milestone: 3.0
Component: encodings Version: 2.4.x
Keywords: Cc:

Description

Even with decent hardware and a single card, it can take ~200ms to initialize the nvenc context (some examples in #2022).

Just like we already do with delayed frames (r13047), we can send a jpeg before initializing the video encoder and then skip the first video frame that comes out of it (the client still needs to decode it but doesn't need to display it).

This will probably require a new "slow-start" codec attribute.

Attachments (1)

threaded-encoder-init.patch (11.7 KB) - added by Antoine Martin 5 months ago.
"works" but cleanup causes a segfault

Download all attachments as: .zip

Change History (9)

comment:1 Changed 11 months ago by Antoine Martin

Status: newassigned

Because the video encoder initialization has to run in the encode thread, we would still incur a 200ms delay, but at least the first frame would arrive quickly. (more important when the window is locked on video encoding)

A smoother solution to this problem would be to:

  • set a flag before starting the video encoder initialization
  • send a low quality jpeg (fast encode)
  • if we end up in the damage loop again, don't queue up the request and process frames immediately with "jpeg" from the worker thread until the video encoder is ready (locking?)

comment:2 Changed 11 months ago by Antoine Martin

Milestone: 2.53.1

I think that this can only work well if we move video encoding to a separate thread so that the normal flow isn't disrupted. We have to ensure that we don't update that video area with non-video screen updates, as those could arrive out of sequence.
That's exactly what the video region exclusion code does... but video encoders can be used without video regions.

On the plus side, the "video-clean" code wouldn't have to wait for the encode queue.

comment:3 Changed 8 months ago by Antoine Martin

See also ticket:2113#comment:7.

The threaded initialization could use video_fallback until the video encoder sets a ready flag.
This is_ready() encoder method could return an exception or error code if the initialization failed, otherwise we would need a callback so that the encoder can tell the window-video-source to try fail this encoder and try another one.

Another option would be to pre-allocate encoder sessions, but this would waste precious encoder contexts. Or maybe just pre-allocate the CUDA context, assuming that this is the most expensive part.

comment:4 Changed 7 months ago by Antoine Martin

Milestone: 3.14.0

Milestone renamed

Changed 5 months ago by Antoine Martin

Attachment: threaded-encoder-init.patch added

"works" but cleanup causes a segfault

comment:5 Changed 5 months ago by Antoine Martin

Owner: changed from Antoine Martin to Nathan Hallquist
Status: assignednew

Done in r22744 - details in commit message (minor related change in r22746)

r22745 makes it possible to test this code on "faster" cards that would initialize so quickly that we don't have time to trigger the fallback code:

XPRA_NVENC_SLOW_DOWN_INIT=2 python3 /usr/bin/xpra start -d compress,nvenc

@nathan_lstc: I think you will be interested in this.
The stuttering should now be completely gone,

Last edited 5 months ago by Antoine Martin (previous) (diff)

comment:6 Changed 5 months ago by Antoine Martin

Milestone: 4.03.0

comment:7 Changed 4 months ago by Antoine Martin

Resolution: worksforme
Status: newclosed

Not heard back.

comment:8 Changed 4 months ago by Antoine Martin

r22932 fixup for the vpx encoder.

See also #2414, #2415.

Last edited 6 weeks ago by Antoine Martin (previous) (diff)
Note: See TracTickets for help on using tickets.