xpra icon
Bug tracker and wiki

Opened 5 weeks ago

Last modified 11 days ago

#1816 assigned enhancement

decode mpeg1 in html5 client

Reported by: Antoine Martin Owned by: Antoine Martin
Priority: major Milestone: 2.4
Component: html5 Version: 2.2.x
Keywords: Cc:

Description

We are having problems enabling H264 video decoding (#1463), but maybe we can use mpeg1 instead via jsmpeg.
It's not going to be as efficient as H264, but still better than jpeg for video. And the API is clean and supports streaming via websockets: DECODE IT LIKE IT'S 1999

Attachments (2)

html5-mpeg1.patch (13.4 KB) - added by Antoine Martin 13 days ago.
mpeg1 encoder, decoder and stub html5 implementation
html5-mpeg1-v2.patch (3.3 KB) - added by Antoine Martin 12 days ago.
hook up mpeg1 decoding, not painting yet

Download all attachments as: .zip

Change History (6)

comment:1 Changed 3 weeks ago by Antoine Martin

Milestone: 3.02.4
Status: newassigned

Changed 13 days ago by Antoine Martin

Attachment: html5-mpeg1.patch added

mpeg1 encoder, decoder and stub html5 implementation

comment:2 Changed 13 days ago by Antoine Martin

Preparatory work:

  • r19296 minor fix for delayed frames (mpeg1 and mpeg2 always buffer one frame?)
  • r19297 support mpeg1 and mpeg2 decoding in python client

The patch attached implements encoding but still needs work:

  • it breaks muxing (fixable)
  • decoding is not actually implemented in the html5 client (couldn't figure out the jsmpeg api)

This may all be moot since H264 decoding seems to be fixed: #1839.
It could still be useful for very low powered device (ie: some arm boards?) where the decoding complexity of H264 is too much?

Last edited 13 days ago by Antoine Martin (previous) (diff)

Changed 12 days ago by Antoine Martin

Attachment: html5-mpeg1-v2.patch added

hook up mpeg1 decoding, not painting yet

comment:3 Changed 12 days ago by Antoine Martin

Updates:

  • r19306: test video encoders with two frames (since mpeg1 and mpeg2 delay one frame, they would error only on the second)
  • r19312: make muxer optional in ffmpeg encoder
  • r19313: implement mpeg1 and mpeg2 encoding, add python avcodec support for decoding mpeg2, and paint colors ("olive" and "lime")

For the html5 client, the patch above seems to decode OK, we just need to do something with the YUV data we get. Either re-use their WebGLRenderer or do software YUV-to-RGB as a POC. (and maybe ship a subset of jsmpeg1 - no need for the player and websocket bits - audio?)

Last edited 12 days ago by Antoine Martin (previous) (diff)

comment:4 Changed 11 days ago by Antoine Martin

html5 mpeg1 decoder added in r19317. (we won't be doing mpeg2: jsmpeg : Support MPEG2?)

Still TODO:

  • check CPU load and bandwidth: compare it with h264 (#1839 - once asm.js optimization is fixed there) so we can use weights to choose the most appropriate video encoder
  • trim jsmpeg.js file
  • re-use the webgl checks to enable the gl renderer in other places?

Note: the "mpeg1" video encoding is controlled by the "Video" option in the advanced connection options. This enables both h264 and mpeg1. To use only mpeg1, set --video-encoders=ffmpeg on the server side.

Last edited 11 days ago by Antoine Martin (previous) (diff)
Note: See TracTickets for help on using tickets.