xpra icon
Bug tracker and wiki

Opened 3 months ago

Last modified 2 months ago

#1423 new enhancement

native jpeg encoder and decoder

Reported by: Antoine Martin Owned by: alas
Priority: major Milestone: 2.0
Component: encodings Version: trunk
Keywords: Cc:

Description

This will speed things up as we can use zero copy upload to the opengl backend directly as YUV data.
The Python Pillow module just doesn't give us YUV...

Attachments (2)

dec_jpeg.patch (15.8 KB) - added by Antoine Martin 3 months ago.
decode jpeg to RGB24
no-background-colour.png (19.8 KB) - added by Antoine Martin 3 months ago.
wiki rendered using the jpeg encoder

Download all attachments as: .zip

Change History (8)

Changed 3 months ago by Antoine Martin

Attachment: dec_jpeg.patch added

decode jpeg to RGB24

comment:1 Changed 3 months ago by Antoine Martin

Status: newassigned
Summary: native jpeg encode and decodernative jpeg encoder and decoder

Decoder added in r14863 + r14864, including build file updates and packaging.
It paints directly using YUV if the opengl backend is present (otherwise it is unused).
The new module should build out of the box on Fedora and centos. (opensuse will need a specfile update). It builds and runs fine on win32 as mingw-w64 already included everything we need.
OSX has the libturbojpeg.dylib but not the matching pkgconfig file. I've created one by hand, it builds with warnings (probably an older version of libjpeg-turbo that we should update) - but the packaging step forgets it?


The encoder shouldn't be too difficult to add.
I don't think we want to go through the pain of hooking our own CSC module so we'll use the built-in libjpeg-turbo version, which is probably plenty fast already.
We should be getting zero-copy transfers from the XSHM, and we will be able to choose the subsampling based on the speed setting (which pillow did not do for us).

comment:2 Changed 3 months ago by Antoine Martin

Owner: changed from Antoine Martin to alas
Status: assignednew

Lots of fixes and related changes:

  • encoder added, decoder re-written using turbojpeg (much nicer): r14886, later improved in r14895 + r14897
  • zero copy in network layer: r14885

Also found a big bug in opengl backend: r14898 (not sure why it didn't bite before..)

Somewhat related:


Ready for testing: the jpeg decoder and encoder should now be faster. BUT, we were previously always using YUV420P with the pillow based encoder whereas now we choose the pixel subsampling based on the quality setting. So this may actually be slower at higher quality settings. (the opengl backend should paint more quickly as it paints using YUV directly, this may also affect the html5 client.. we may want to lower / adjust the jpeg quality settings for it)

Last edited 3 months ago by Antoine Martin (previous) (diff)

Changed 3 months ago by Antoine Martin

Attachment: no-background-colour.png added

wiki rendered using the jpeg encoder

comment:3 Changed 3 months ago by Antoine Martin

Owner: changed from alas to Antoine Martin
Status: newassigned

Found some visual artifacts as part of testing for #1426: the xpra wiki loses its background colour:
wiki rendered using the jpeg encoder.

comment:4 Changed 3 months ago by Antoine Martin

I suspect that the YUV output uses a different range than what we are used to with the avcodec and vpx output, so r14974 disables the YUV jpeg decoder until I can figure this out.

Last edited 3 months ago by Antoine Martin (previous) (diff)

comment:5 Changed 3 months ago by Antoine Martin

This new codec help uncover 2 very serious issues, fixed in r14990 + r14991.
Also needed r14989 to close the context and avoid a memory leak.
Minor improvements in r15000, r15001 (opengl paint debugging)

Last edited 3 months ago by Antoine Martin (previous) (diff)

comment:6 Changed 2 months ago by Antoine Martin

Owner: changed from Antoine Martin to alas
Status: assignednew

I give up on decoding to YUV: moved to #1438.

As of r15061 + r15062, we use this new decoder to RGB with both opengl and pixmap backings.

@afarr: this is mostly a FYI. This new codec is faster, that is all.
You can see the compression performance server side with "-d compress", client side with:

XPRA_JPEG_LOG_PERF=1 xpra attach --encodings=jpeg -d jpeg

I get:

decompress_to_rgb: size=694x244, subsampling=444, colorspace=YCbCr
decompress jpeg to BGRX:  869 MB/s (   677344 bytes in 0.7ms)
decompress_to_rgb: size=694x244, subsampling=444, colorspace=YCbCr
decompress jpeg to BGRX:  461 MB/s (   677344 bytes in 1.4ms)

~400 to 900 MB/s is quite impressive.

Last edited 2 months ago by Antoine Martin (previous) (diff)
Note: See TracTickets for help on using tickets.