xpra icon
Bug tracker and wiki

Opened 7 years ago

Closed 7 years ago

Last modified 7 years ago

#231 closed enhancement (fixed)

queuing packets for sending is very expensive in UI thread

Reported by: Antoine Martin Owned by: Antoine Martin
Priority: major Milestone: 0.8
Component: core Version: trunk
Keywords: Cc:

Description (last modified by Antoine Martin)

We currently require all calls to send() to be done in the main thread, but this can be quite expensive - see this pycallgraph diagram:
https://www.xpra.org/trac/raw-attachment/ticket/231/pycallgraph-expensivetosend3.png

which was obtained with a server running:

xpra start :10 --start-child="vglrun glxgears"

And the client running

XPRA_OPENGL=0 ./tests/scripts/pycallgraph -i '*' -e ALL -d 2 -r 20 -- \
    attach :10 --no-mmap --encoding=x264

Most of the time used in the UI thread is to do with this network stuff, all the other things (decoding, I/O are already in separate threads and therefore do not affect latency much).

Potential solutions:

  • relax the threading check to ensure that the network code works when called from any thread
  • move network code to its own thread/event based
  • ???

Attachments (6)

pycallgraph-expensivetosend.png (192.1 KB) - added by Antoine Martin 7 years ago.
shows how expensive it is to be queuing packets from the main thread
pycallgraph-expensivetosend.png​ (110.3 KB) - added by Antoine Martin 7 years ago.
re-generate the graph with better / more reproducible instructions (and replace old one)
pycallgraph-expensivetosend2.png​ (110.3 KB) - added by Antoine Martin 7 years ago.
re-generate the graph with better / more reproducible instructions
pycallgraph-expensivetosend3.png (108.3 KB) - added by Antoine Martin 7 years ago.
re-generate the graph with better / more reproducible instructions (trying again - not sure why trac doesn't let us see it / preview it)
pycallgraph-fixed-r2492.png (76.4 KB) - added by Antoine Martin 7 years ago.
updated graph with r2492 showing a much improved codepath and timing
pycallgraph-fixed-r2493.png (70.4 KB) - added by Antoine Martin 7 years ago.
updated graph with r2493 showing more improvements

Download all attachments as: .zip

Change History (12)

Changed 7 years ago by Antoine Martin

shows how expensive it is to be queuing packets from the main thread

comment:1 Changed 7 years ago by Antoine Martin

Description: modified (diff)
Owner: set to Antoine Martin
Status: newaccepted

comment:2 Changed 7 years ago by Antoine Martin

Description: modified (diff)

Changed 7 years ago by Antoine Martin

re-generate the graph with better / more reproducible instructions (and replace old one)

Changed 7 years ago by Antoine Martin

re-generate the graph with better / more reproducible instructions

comment:3 Changed 7 years ago by Antoine Martin

Description: modified (diff)

Changed 7 years ago by Antoine Martin

re-generate the graph with better / more reproducible instructions (trying again - not sure why trac doesn't let us see it / preview it)

comment:4 Changed 7 years ago by Antoine Martin

Description: modified (diff)

Changed 7 years ago by Antoine Martin

Attachment: pycallgraph-fixed-r2492.png added

updated graph with r2492 showing a much improved codepath and timing

comment:5 Changed 7 years ago by Antoine Martin

Resolution: fixed
Status: acceptedclosed

r2492 fixes this (lots of details in the commit message)


Profiling wiki received some import updates for correct profiling (some tricky and misleading issues)


What is immediately obvious by looking at the new callgraph below is how we skip numerous indirection steps by simplifying the code structure:


https://www.xpra.org/trac/raw-attachment/ticket/231/pycallgraph-fixed-r2492.png

Changed 7 years ago by Antoine Martin

Attachment: pycallgraph-fixed-r2493.png added

updated graph with r2493 showing more improvements

comment:6 Changed 7 years ago by Antoine Martin

Finally, with r2493, we're down to barebones, probably as fast as it is going to be in python:

https://www.xpra.org/trac/raw-attachment/ticket/231/pycallgraph-fixed-r2493.png

Note: See TracTickets for help on using tickets.