xpra icon
Bug tracker and wiki

Opened 3 weeks ago

Closed 3 weeks ago

Last modified 3 days ago

#2341 closed enhancement (fixed)

faster server startup

Reported by: Antoine Martin Owned by: Antoine Martin
Priority: major Milestone: 3.0
Component: server Version: 2.5.x
Keywords: Cc: [email protected]

Description

Following the complaint that xpra startup is slow (ie: in this ticket: https://github.com/mviereck/x11docker/issues/167), we can parallelize some of the mixin startup work.

Related to #1838

Attachments (2)

server-start-debug.patch (29.4 KB) - added by Antoine Martin 3 weeks ago.
ugly debug logging patch
server-start-debug-mainloop.patch (4.4 KB) - added by Antoine Martin 3 weeks ago.
shows that the main loop startup is not slow because of anything we do

Download all attachments as: .zip

Change History (13)

comment:1 Changed 3 weeks ago by Antoine Martin

Status: newassigned

Updates:

  • r23025: if opengl is set to noprobe, just skip opengl_probe() - saves almost a second
  • r23026: no need to populate the platform cache before registering the SIGCHLD handler (saves about 100 milliseconds)
  • r23027: move some webcam code to the initialization thread (saves around 50 milliseconds)
  • r23028: change fake-xinerama option so we can find the library path once at build time (saves about 100 milliseconds)

comment:2 Changed 3 weeks ago by Antoine Martin

Remaining places where we could save time:

  • instantiating the server class (loading bindings: can we delay that?): 70ms
  • X11ServerBase.init(opts): 90ms
  • init_randr: 10ms (could use --resize-display=no to skip?)
  • mdns_publish: 20ms (could start from init thread?)
  • init_encodings: 70ms (loading all the bindings: only do this from the init thread?)
  • idle_add takes 90ms the first time we call it (just the main loop starting?)

comment:3 Changed 3 weeks ago by Antoine Martin

  • instantiating the server class is slow because we load so many modules (lots of mixins) and cython extensions (x11 bindings mostly), but there is one place where we can do better: importing xpra.net.net_util takes too long: over 40ms!
  • X11ServerBase.init(opts) is slow because of GTK's selection.acquire() in XSettingsManager - not much we can do about that
  • print_run_info takes a bit of time on Posix because we lookup the user in the password database: move this to a new thread or use the background worker (no need to delay the init thread for this)

Changed 3 weeks ago by Antoine Martin

Attachment: server-start-debug.patch added

ugly debug logging patch

comment:4 Changed 3 weeks ago by Antoine Martin

Updates:

  • r23034 (missed from r23028)
  • r23030 print_run_info via worker thread
  • r23032 use faster socket functions for if_nametoindex and if_indextoname with python3
  • r23037 call mdns_publish from the worker thread
  • r23038 load codecs from threaded setup
  • r23039 don't watch for menu changes when we don't expose them (start-new-commands=no)

With these changes and the following command line:

python3 /usr/bin/xpra start --no-daemon --start=xterm \
   --opengl=noprobe --xsettings=no --no-speaker --no-microphone --html=no

The startup takes at most 350ms on my laptop. (400ms for python2)

Other places where we still spend a little bit too much time:

  • server base import of mixins: ~20ms (nothing we can do)
  • BaseHTTPRequestHandler takes 30ms to import! (can be skipped with --html=off - nothing we can do)
  • main loop startup: over 100ms (not sure what can be done there)

New potential issue: race condition if we are still initializing via the worker thread when we are asked to shutdown.. maybe process the shutdown request via the worker thread to guarantee we complete what has already been queued?

Last edited 3 weeks ago by Antoine Martin (previous) (diff)

comment:5 Changed 3 weeks ago by Antoine Martin

New issues found in https://github.com/mviereck/x11docker/files/3341648/xpraserver.log:

  • the server is started against an existing vfb so we end up trying to locate an existing _XPRA_UINPUT_ID and this takes a while (0.4s):
    2019-06-29 10:10:12,666 Popen(['xprop', '-display', ':101', '-root', '_XPRA_UINPUT_ID'])=(0, '_XPRA_UINPUT_ID:  no such atom on any window.\n', '')
    2019-06-29 10:10:13,038 setup_local_sockets: bind=['auto']
    

Partially dealt with in r23043. I'm not sure we want to spend more time on this particular issue: re-using an existing display is not a standard setup..

  • maybe add an option (env var) to skip the codec self tests? (30ms) meh - now that we do this in the init thread, it doesn't really matter much
Last edited 3 weeks ago by Antoine Martin (previous) (diff)

comment:6 Changed 3 weeks ago by Antoine Martin

See also #2344 and #2347

Last edited 3 weeks ago by Antoine Martin (previous) (diff)

comment:7 Changed 3 weeks ago by mviereck

Cc: [email protected] added

Changed 3 weeks ago by Antoine Martin

shows that the main loop startup is not slow because of anything we do

comment:8 Changed 3 weeks ago by Antoine Martin

Resolution: fixed
Status: assignedclosed

Updates:

  • r23047 fix race condition with worker thread
  • r23048 fix race condition with initialization thread
  • r23049 better worker thread cleanup: faster, safer
  • as per the patch above, the main loop takes 150ms to start, but this is not caused by our code

With all these changes, a server starts on my laptop in under 4 seconds flat - and that's including the slow xvfb startup!
With an existing display, the total startup time (logged) is now around 250ms! (actual time is marginally higher than that since logging starts after a the initial startup code)

Log sample with an existing display:

$ python3 /usr/bin/xpra start --no-daemon --start=xterm \
    --opengl=noprobe --xsettings=no --speaker=no --microphone=no \
    --mdns=no --start-new-commands=no --notifications=no --html=no \
    :10 --use-display
2019-06-30 11:15:27,722 created unix domain socket: /run/user/1000/xpra/linux.home-10
2019-06-30 11:15:27,766 pointer device emulation using XTest
2019-06-30 11:15:27,787 xvfb pid=20443
2019-06-30 11:15:27,959 xpra is ready.
2019-06-30 11:15:27,960 xpra GTK3 X11 version 3.0-r22755 64-bit
2019-06-30 11:15:28,037  uid=1000 (antoine), gid=1000 (antoine)
2019-06-30 11:15:28,038  running with pid 20892 on Linux Fedora 30 Thirty
2019-06-30 11:15:28,038  connected to X11 display :10 with 24 bit colors
2019-06-30 11:15:28,657 found 1 virtual video device for webcam forwarding
2019-06-30 11:15:28,750 7.7GB of system memory

comment:9 Changed 3 weeks ago by Antoine Martin

r23066: don't load xdg menu data unless start-new-commands is enabled - huge savings

comment:10 Changed 2 weeks ago by Antoine Martin

See also:

comment:11 Changed 3 days ago by Antoine Martin

This may have caused a regression: #2355.

Note: See TracTickets for help on using tickets.