Xpra: Ticket #231: queuing packets for sending is very expensive in UI thread
We currently require all calls to send()
to be done in the main thread, but this can be quite expensive - see this pycallgraph diagram:
which was obtained with a server running:
xpra start :10 --start-child="vglrun glxgears"
And the client running
XPRA_OPENGL=0 ./tests/scripts/pycallgraph -i '*' -e ALL -d 2 -r 20 -- \
attach :10 --no-mmap --encoding=x264
Most of the time used in the UI thread is to do with this network stuff, all the other things (decoding, I/O are already in separate threads and therefore do not affect latency much).
Potential solutions:
- relax the threading check to ensure that the network code works when called from any thread
- move network code to its own thread/event based
- ???
Tue, 08 Jan 2013 05:36:22 GMT - Antoine Martin: attachment set
- attachment
set to pycallgraph-expensivetosend.png
shows how expensive it is to be queuing packets from the main thread
Tue, 08 Jan 2013 05:36:58 GMT - Antoine Martin: status, description changed; owner set
- owner
set to Antoine Martin
- status
changed from new to accepted
- description
modified (diff)
Tue, 08 Jan 2013 06:00:40 GMT - Antoine Martin: description changed
- description
modified (diff)
Wed, 09 Jan 2013 10:14:52 GMT - Antoine Martin: attachment set
- attachment
set to pycallgraph-expensivetosend.png
re-generate the graph with better / more reproducible instructions (and replace old one)
Wed, 09 Jan 2013 10:16:21 GMT - Antoine Martin: attachment set
- attachment
set to pycallgraph-expensivetosend2.png
re-generate the graph with better / more reproducible instructions
Wed, 09 Jan 2013 10:19:37 GMT - Antoine Martin: description changed
- description
modified (diff)
Wed, 09 Jan 2013 10:24:18 GMT - Antoine Martin: attachment set
- attachment
set to pycallgraph-expensivetosend3.png
re-generate the graph with better / more reproducible instructions (trying again - not sure why trac doesn't let us see it / preview it)
Wed, 09 Jan 2013 10:25:05 GMT - Antoine Martin: description changed
- description
modified (diff)
Wed, 09 Jan 2013 10:26:38 GMT - Antoine Martin: attachment set
- attachment
set to pycallgraph-fixed-r2492.png
updated graph with r2492 showing a much improved codepath and timing
Wed, 09 Jan 2013 10:38:25 GMT - Antoine Martin: status changed; resolution set
- status
changed from accepted to closed
- resolution
set to fixed
r2492 fixes this (lots of details in the commit message)
Profiling wiki received some import updates for correct profiling (some tricky and misleading issues)
What is immediately obvious by looking at the new callgraph below is how we skip numerous indirection steps by simplifying the code structure:
Wed, 09 Jan 2013 10:40:06 GMT - Antoine Martin: attachment set
- attachment
set to pycallgraph-fixed-r2493.png
updated graph with r2493 showing more improvements
Wed, 09 Jan 2013 10:41:30 GMT - Antoine Martin:
Finally, with r2493, we're down to barebones, probably as fast as it is going to be in python:
Sat, 23 Jan 2021 04:49:05 GMT - migration script:
this ticket has been moved to: https://github.com/Xpra-org/xpra/issues/231