xpra icon
Bug tracker and wiki

Opened 8 years ago

Last modified 4 months ago

#202 accepted enhancement

hardware accelerated video decoding

Reported by: Antoine Martin Owned by: Antoine Martin
Priority: major Milestone: 4.1
Component: core Version: trunk
Keywords: Cc: rektide@…

Description (last modified by Antoine Martin)

Following #147, we should go one step further and offload x264 video decoding to the graphics card using VAAPI / libva

Some pointers:

Why VAAPI? (lwn.net) Because this is the most widely compatible/available API, with glue libraries available for both vdpau (nvidia) and xvba (ati).
If really needed we could also talk to those directly, but that is best avoided.

Change History (9)

comment:1 Changed 8 years ago by Antoine Martin

Description: modified (diff)
Status: newaccepted

comment:2 Changed 8 years ago by ahuillet

I don't think we should be using VAAPI directly for video decoding acceleration. libav is a pain, but it has a VAAPI backend, and it will probably be better to use it. Rationale: we'll need video decoding acceleration on Windows (possibly even more than on Linux). We'll want to do it through libva as well.

comment:3 Changed 8 years ago by Antoine Martin

Description: modified (diff)

comment:5 Changed 7 years ago by Antoine Martin

For DXVA see #346

comment:8 Changed 5 years ago by rektide

Cc: rektide@… added

comment:12 Changed 3 years ago by Antoine Martin

From the comments section on VA-API Video Acceleration On The Linux Desktop Is Nearly Ready For Chrome:

  • Then there's the ability to output hardware decoded video to an X11 window or a pixmap, Radeon VAAPI does have that, but this is good mainly for direct display (like a dedicated video player), not for integration into a complex UI that is a web page. It's also tied to X, obviously, so not ideal also from this perspective.
  • If you want to use the opengl video output with hardware decoding on Radeon, VDPAU is probably the better option.

comment:14 Changed 3 years ago by Antoine Martin

Milestone: 0.93.1

comment:15 Changed 20 months ago by Antoine Martin

Milestone: 3.14.0

Milestone renamed

comment:16 Changed 4 months ago by Antoine Martin

Milestone: 4.04.1

Now that encoding via vaapi works (#451), can we use the same solution here? (no)

The data we feed into the decoder is relatively small (so no problem uploading it to the GPU), but the decoded data is big (full size pixel frames) and so it's a waste if we have to download it to CPU memory only to re-upload to the GPU for rendering on screen.
It probably negates most of the savings from decoding on the GPU - the server usually selects fast presets which aren't very taxing for a CPU software decoder.
Then there's also the extra decoder complexity (except on macos with h264_vda which is a drop in replacement for h264?) and usual hardware flakiness (lack of hardware for testing, differences in drivers / OS, etc).

Staying on the GPU, as we do encoding side with nvfbc #1317, will require direct use of GPU buffers (dmabuf / pixmaps on Linux, whatever else on other platforms).
This is harder, and will need to be done for each supported platform separately. PITA.

This is looking unlikely for 4.1

Note: See TracTickets for help on using tickets.