Shouldn't be too hard to do. As of r6082, the code compiles. What's left to do:
Once that's done, we can have superfast session shadowing, which would be nice. The biggest remaining bottleneck is likely to be how we capture the screen's pixels.
Scheduling for 0.16
Mostly done in r9999, and found some bugs along the way, it seems to work, but:
patch to make it easier to tweak the cuda vertical output size at runtime using an env var
With the patch above, I can make it encode properly by using the magic value:
Which makes it convert to YUV444P using a vertical size of 1020 instead of 1088. No idea why this magic value is "right", yet.
more hacks to tweak input values to the kernel and encoder
With the patch above, we get the correct size for the output picture, though the VADJUST value does make the input buffers overlap a bit which corrupts the picture near the top. It seems that the video encoder vertical size used is not the one we specify (1080 rounded up to 1088) but it uses 1020 instead.. which is just a really weird number to choose: 255*4. Now, we could just find the next size up that rounds down to what we need, but the exact formulae used is unlikely to be obvious. (255?) And I see no way of knowing when this hack should be applied. Only on win32? Only with some driver versions? It is worth mentioning that these latest drivers seem to have fixed the long standing bug which exposed the wrong values for the input formats. (forcing us to use the value as a mask..) Could this be related? We have always rounded up the width of the input buffer from 1920 to 2048, does this matter?
So based on these (wrong) assumptions, I could explain the 68 lines adjustment. It is wrong because we round up to 2560, not 2048... But maybe something else rounds like that.
shows the visual corruption with a Linux server and client
I've updated my Linux drivers to the beta version 355.06 and I am now seeing the same defects here. I should have known that expecting a stable ABI was too much to ask...
So now we need to build a blacklist, or version driven workaround. Not sure if the driver version number is even available on win32 since we use the kmod version on Linux!
Sure enough, downgrading the drivers on my test win32 box fixes nvenc.
Some options for figuring out the version number on win32 (which may also be able to replace the ugly proc module parsing on Linux in some cases):
Immediate problem that I can see: we're running a 32-bit python interpreter, and the 64 bit "NVML.DLL" and won't load...
Implemented a simple cython wrapper for NVAPI in r10240, we just statically link against
nvapi64.lib on 64-bit - untested).
This gives us a version number we can check against.
For the time being we just print a big warning, maybe this should fail the whole nvenc module? This will do for now, we need #389 for this to be truly useful anyway (but the bugs found along the way were worth it!).
this ticket has been moved to: https://github.com/Xpra-org/xpra/issues/558