Xpra: Ticket #558: nvenc support for shadowing on win32

Shouldn't be too hard to do. As of r6082, the code compiles. What's left to do:

Once that's done, we can have superfast session shadowing, which would be nice. The biggest remaining bottleneck is likely to be how we capture the screen's pixels.



Tue, 19 Aug 2014 04:11:31 GMT - Antoine Martin: milestone changed

Scheduling for 0.16


Wed, 22 Jul 2015 12:01:36 GMT - Antoine Martin: owner, status changed

Mostly done in r9999, and found some bugs along the way, it seems to work, but:


Sun, 26 Jul 2015 10:18:40 GMT - Antoine Martin: attachment set

patch to make it easier to tweak the cuda vertical output size at runtime using an env var


Sun, 26 Jul 2015 10:20:18 GMT - Antoine Martin:

With the patch above, I can make it encode properly by using the magic value:

set XPRA_NVENC_VADJUST=-68

Which makes it convert to YUV444P using a vertical size of 1020 instead of 1088. No idea why this magic value is "right", yet.


Mon, 27 Jul 2015 11:38:28 GMT - Antoine Martin: attachment set

more hacks to tweak input values to the kernel and encoder


Mon, 27 Jul 2015 11:51:49 GMT - Antoine Martin:

With the patch above, we get the correct size for the output picture, though the VADJUST value does make the input buffers overlap a bit which corrupts the picture near the top. It seems that the video encoder vertical size used is not the one we specify (1080 rounded up to 1088) but it uses 1020 instead.. which is just a really weird number to choose: 255*4. Now, we could just find the next size up that rounds down to what we need, but the exact formulae used is unlikely to be obvious. (255?) And I see no way of knowing when this hack should be applied. Only on win32? Only with some driver versions? It is worth mentioning that these latest drivers seem to have fixed the long standing bug which exposed the wrong values for the input formats. (forcing us to use the value as a mask..) Could this be related? We have always rounded up the width of the input buffer from 1920 to 2048, does this matter?


Thu, 30 Jul 2015 02:57:44 GMT - Antoine Martin:

Simple maths:

So based on these (wrong) assumptions, I could explain the 68 lines adjustment. It is wrong because we round up to 2560, not 2048... But maybe something else rounds like that.


Thu, 06 Aug 2015 13:05:02 GMT - Antoine Martin: attachment set

shows the visual corruption with a Linux server and client


Thu, 06 Aug 2015 13:05:57 GMT - Antoine Martin: priority changed

I've updated my Linux drivers to the beta version 355.06 and I am now seeing the same defects here. I should have known that expecting a stable ABI was too much to ask...

So now we need to build a blacklist, or version driven workaround. Not sure if the driver version number is even available on win32 since we use the kmod version on Linux!


Sat, 08 Aug 2015 07:23:00 GMT - Antoine Martin:

Sure enough, downgrading the drivers on my test win32 box fixes nvenc.

Some options for figuring out the version number on win32 (which may also be able to replace the ugly proc module parsing on Linux in some cases):

Immediate problem that I can see: we're running a 32-bit python interpreter, and the 64 bit "NVML.DLL" and won't load...


Sat, 08 Aug 2015 09:01:14 GMT - Antoine Martin: status changed; resolution set

Implemented a simple cython wrapper for NVAPI in r10240, we just statically link against nvapi.lib (nvapi64.lib on 64-bit - untested). This gives us a version number we can check against.

For the time being we just print a big warning, maybe this should fail the whole nvenc module? This will do for now, we need #389 for this to be truly useful anyway (but the bugs found along the way were worth it!).


Sat, 23 Jan 2021 04:59:20 GMT - migration script:

this ticket has been moved to: https://github.com/Xpra-org/xpra/issues/558