xpra icon
Bug tracker and wiki

Opened 3 months ago

Last modified 8 weeks ago

#1926 new enhancement

replace numpy dependency in websockify

Reported by: Antoine Martin Owned by: J. Max Mena
Priority: major Milestone: 2.4
Component: external Version: 2.3.x
Keywords: Cc:


As per ticket:1913#comment:2, importing numpy wastes a lot of memory and the only thing that websockify uses it for is to do a string XOR!

Let's submit a patch to make this more pluggable, we have our own code which can provide it. (our cyxor cython extension)
The import would need to be deferred since the websockify module imports the sub-modules. (ie: WebSocket constructor could get the xor wrapper function from websockify.xor which we would then be able to patch up)

Alternatively, we could use the modules context hack to temporarily hide "numpy" from the module loader, then patch the _unmask method.
This approach would work with older version of websockify - and they have not made any new releases in years now...

See also #1134.

Change History (2)

comment:1 Changed 3 months ago by Antoine Martin

Owner: changed from Antoine Martin to J. Max Mena

r19989 does this by pretending numpy is not installed and then monkeypatching a cython function to do the work.

This should be faster than the numpy code it replaces:

  • one fewer copy of the buffer data when doing the XOR
  • no longer copying the buffers when reading from the websocket (pass memoryview directly to caller, queue the rest)

The impact should be negligible since the server doesn't do a lot of reading.

importing websockify will now show this warning:

WARNING: no 'numpy' module, HyBi protocol will be slower

This can safely be ignored.

@maxmylyn: please check that this new code does not cause any performance regressions.

comment:2 Changed 8 weeks ago by Antoine Martin

r19989 caused a regression reported in ticket:1941#comment:7 :

disconnect: zlib packet decompression failed must be string or read-only buffer, not memoryview

Fixed in r20229.

Note: See TracTickets for help on using tickets.