xpra icon
Bug tracker and wiki

Opened 3 years ago

Closed 5 months ago

#1527 closed enhancement (worksforme)

win32 system service

Reported by: Antoine Martin Owned by: Antoine Martin
Priority: major Milestone: 4.0
Component: platforms Version: trunk
Keywords: win32 Cc:

Description

Split from #389, similar to #1105 for Linux, we want to have a system wide service running even before the user logs in.
Users can then trigger a login.

Some links already added in ticket:389#comment:23.

Attachments (9)

service-mingw.patch (2.7 KB) - added by Antoine Martin 3 years ago.
patch for building the service example with mingw
start-shadow-from-proxy.patch (8.1 KB) - added by Antoine Martin 3 years ago.
ugly work in progress patch to use on top of r15971
logon.patch (22.6 KB) - added by Antoine Martin 3 years ago.
work in progress logon patch
logon-v2.patch (50.2 KB) - added by Antoine Martin 3 years ago.
logon succeeds but launching the new process does not..
logon.py (17.8 KB) - added by Antoine Martin 3 years ago.
this implementation does not work - but has some useful functions
service-test.patch (3.8 KB) - added by totaamwin32 13 months ago.
tweaks to make it easier to test and debug the service
logon.c (18.9 KB) - added by totaamwin32 13 months ago.
logon.c example from "Starting an Interactive Client Process in C++"
win32-system-proxy.patch (3.8 KB) - added by totaamwin32 5 months ago.
switch back to launching the proxy
sessions.py (7.5 KB) - added by totaamwin32 5 months ago.
various ill-fated attempts to get the list of sessions

Download all attachments as: .zip

Change History (42)

comment:1 Changed 3 years ago by Antoine Martin

Status: newassigned

Blocked by #1528.

comment:2 Changed 3 years ago by Antoine Martin

With the cx_freeze 5 workarounds added in ticket:1528#comment:2 and the service stub added in r15933 +

r15934 (based on cx_Freeze samples: service), a "Xpra-Service.exe" is created - it just doesn't do anything when you run it...

The documentation is non-existent, and the executable doesn't give you any help at all. But I eventually found that we're supposed to install it by running:

Xpra-Service.exe --install NAME [configfile]

Problem is that this fails with:

Service not installed. See log file for details.

What log file you ask? Well "./.log" obviously, NOT. This file contains:

[04380] 2017/05/23 22:46:14.473 starting logging at level ERROR
[04380] 2017/05/23 22:46:14.520 Win32 error 0x5 encountered.
[04380] 2017/05/23 22:46:14.520     Context: cannot open service manager
[04380] 2017/05/23 22:46:14.520     Message: Access is denied.

[04380] 2017/05/23 22:46:14.520 ending logging

Fine, let's run it as administrator:

[02964] 2017/05/23 22:49:55.695 starting logging at level ERROR
[02964] 2017/05/23 22:49:55.757 Win32 error 0x57 encountered.
[02964] 2017/05/23 22:49:55.757     Context: cannot start service
[02964] 2017/05/23 22:49:55.757     Message: The parameter is incorrect.

[02964] 2017/05/23 22:49:55.773 ending logging

At least the service is registered and visible in the "Services" control tool, and the same error message is available in the "eventvwr".
We'll need to get rid of "cx_Logging", it's unmaintained since 2014, undocumented (which filename is used from the service?) and generally not helpful.

Last edited 3 years ago by Antoine Martin (previous) (diff)

comment:3 Changed 3 years ago by Antoine Martin

OK, the reason why this fails with the cryptic error 0x57 is because the path to the service binary is empty (for whatever reason).
Assuming that the service has been installed using:

Xpra-Service.exe --install XPRA-TEST

The path can then be changed using regedit, or using Sc config.
First query the service:

sc qc XpraXPRA-TEST

Change the path:

sc config XpraXPRA-TEST binPath= "C:\Program Files\Xpra\Xpra-Service.exe"

We can now also set it to auto-start:

sc config XpraXPRA-TEST start= auto

With these changes, starting the service takes longer to fail.. but fail it does with: Error 1053: The service did not respond to the start or control request in a timely fashion
We can find the service PID with:

sc queryex XpraXPRA-TEST

Then kill it with:

taskkill /f /pid $PID

We could workaround the broken path installation by running the "sc config" commands after the service registration, still as part of the EXE / MSI

installation process.

But the fact that the process does not respond is more problematic...
Ideas:

  • maybe the paths workarounds need to be applied to the service class (in C)
  • maybe use a simple shim? a simple C service that (re)starts the real xpra proxy server?

The shim would allow us to continue to use cx_freeze 4.x (no need for #1528). Some links for that:

Last edited 3 years ago by Antoine Martin (previous) (diff)

comment:4 Changed 3 years ago by Antoine Martin

Difficulties in building the Complete Service Example with mingw:

Last edited 3 years ago by Antoine Martin (previous) (diff)

Changed 3 years ago by Antoine Martin

Attachment: service-mingw.patch added

patch for building the service example with mingw

comment:5 Changed 3 years ago by Antoine Martin

Stub cx_freeze5 service removed in r15960 (#1528 re-scheduled), replaced in r15962 by a shim implemented in C.

Still TODO:

  • fix authentication problems
  • remove hard-coded command line path: load settings from a config file?
  • innosetup script changes: upgrading windows service using inno setup - maybe just ship the "SvcConfig" and "SvcControl" command line tools instead?
  • start new sessions / locate existing ones and run the shadow under that user account
  • fix event logging: we have to use event codes in the resource file (yuk)
  • re-enable SSL: substitute commonappdata to locate the cert - or just generated the path in the config file

Notes:

Last edited 2 years ago by Antoine Martin (previous) (diff)

Changed 3 years ago by Antoine Martin

ugly work in progress patch to use on top of r15971

Changed 3 years ago by Antoine Martin

Attachment: logon.patch added

work in progress logon patch

comment:6 Changed 3 years ago by Antoine Martin

The ugly and incomplete patch above does:

And adds a small Login-Test.exe utility. Problem is that this works when running from the service context, but not when running directly from the utility - even when running from an administrator shell. This is going to make development and testing tedious.

Some useful links:

Last edited 3 years ago by Antoine Martin (previous) (diff)

Changed 3 years ago by Antoine Martin

Attachment: logon-v2.patch added

logon succeeds but launching the new process does not..

comment:7 Changed 3 years ago by Antoine Martin

With the patch above, I could logon but when trying to start the server, or even a test application like whoami.exe with all of its dlls installed, the event log would show:

Application popup: Xpra_cmd.exe - Application Error : \
The application was unable to start correctly (0xc0000142). Click OK to close the application.

The environment looked suspicious, but since we use the LOGON_WITH_PROFILE flag, it should be OK. (will need to re-check that)
See What is up with "The application failed to initialize properly (0xc0000142)" error?, which has lots of relevant information. (and some dead links too.. sigh)
Also some pointers here: The Perils and Pitfalls of Launching a Process Under New Credentials.
It was just missing the desktop name in the STARTUPINFO.... (not obvious)
Now maybe we also need permission to access that desktop? As per CreateProcessAsUser() windowstations and desktops, because the resulting screen is empty.

comment:8 Changed 3 years ago by Antoine Martin

r15980 adds all the hooks for starting the shadow process from the service.
Still TODO:

  • remove hard-coded command line path: load settings from a config file?
  • innosetup integration
  • add firewall rules: How to use the "netsh advfirewall firewall", ie:
    C:\Windows\system32>netsh advfirewall firewall add rule name="Open Port 14500" d
    ir=in action=allow protocol=TCP localport=14500
    
  • re-enable SSL: substitute commonappdata to locate the cert - or just generated the path in the config file
  • innosetup script changes: ​upgrading windows service using inno setup - maybe just ship the "SvcConfig" and "SvcControl" command line tools instead?
  • don't use a fixed TCP port, use named pipes via the system proxy - or enhance the client to support a redirect and use OTP (which we can supply to the shadow process using env auth)
  • html5 client connection fails, re-tries: we end up creating new shadow server instances... don't retry on fatal error (html5), don't create a new shadow if one exists!
  • fix event logging: we have to use event codes in the resource file (yuk)
  • the desktop we get is empty! why?
  • start new sessions / locate existing ones and run the shadow under that user account
  • integrate with "remote desktop services" #1532
Last edited 2 years ago by Antoine Martin (previous) (diff)

Changed 3 years ago by Antoine Martin

Attachment: logon.py added

this implementation does not work - but has some useful functions

comment:9 Changed 2 years ago by Antoine Martin

Milestone: 2.12.4

See ticket:389#comment:23 for LogonUser links.

Last edited 2 years ago by Antoine Martin (previous) (diff)

comment:10 Changed 2 years ago by Antoine Martin

See Another Windows 10 SKU is on its way, this time for remote desktops: With the new SKU, the multi-session capability is now a part of desktop Windows

comment:11 Changed 18 months ago by Antoine Martin

Milestone: 2.43.0

comment:12 Changed 18 months ago by Antoine Martin

See also: python-win32: runas analog: Starting a subprocess in Windows is somewhat of a black art, I fear.

comment:13 Changed 13 months ago by Antoine Martin

Fixes and updates:

  • r23582 support more SDK paths
  • r23583 cosmetic
  • r23584 prefer 64-bit tools
  • r23585 make it easier to override the log file name (+r23586 tidy up)
  • r23588 adds an "uninstall" command to the service utility

The service somehow fails to find the glib typelib.
Debugging this is tedious, will add yet another shim to make it easier to debug.

Last edited 13 months ago by Antoine Martin (previous) (diff)

comment:14 Changed 13 months ago by totaamwin32

The gi bindings error is an unrelated packaging blocker bug: #2393.

comment:15 Changed 13 months ago by totaamwin32

Updates:

  • r23593 fix named-pipe handling of spurious timeouts (should backport)
  • r23594 don't set CKCON_X11_DISPLAY
  • r23595 typo
  • r23596 don't use roaming profile location for log files
  • r23597 don't create app dir in system profile
  • r23598 skip loading "pwd" module on non-posix systems
  • r23599 fix win32 authenticator with python3 (should backport)
  • r23600 better auth debug logging

Some related pointers:

comment:16 Changed 13 months ago by totaamwin32

Updates:

The shadow we start creates a named-pipe, but the proxy server running as "system" cannot connect to it.
Some pointers:

comment:17 Changed 13 months ago by totaamwin32

Updates:

  • r23620 better logging for connection problems
  • r23621 create a specific named pipe, generate the path from the username we shadow as
  • r23622 add more security API definitions
  • r23623 remove all security on the named pipes when XPRA_NAMED_PIPE_UNRESTRICTED=1
  • r23624 by default, don't log the backtrace for socket timeouts (as we get lots of them on win32..)
  • r23625 remove all security on the proxy named pipe, add sys authentication

With these changes, the proxy manages to start a shadow subprocess, but it doesn't seem to reach a usable state.
Redirecting its output to a log file shows that it goes through server initialization, it stops after loading a few icons.

comment:18 Changed 13 months ago by totaamwin32

Updates:

  • r23626 + r23627 + r23631 + r23633 debug logging tweaks
  • r23629 fix pylint warnings
  • r23634 shadow server now shows as "Xpra-Shadow" in the task manager
  • r23635 better named pipe listener code (large)
  • r23636 catch proxy start errors, ie: bug with ssl sockets: #2399
  • r23638 use a shim for starting the shadow server (+r23641 freeze support)
Last edited 13 months ago by totaamwin32 (previous) (diff)

Changed 13 months ago by totaamwin32

Attachment: service-test.patch added

tweaks to make it easier to test and debug the service

Changed 13 months ago by totaamwin32

Attachment: logon.c added

logon.c example from "Starting an Interactive Client Process in C++"

comment:19 Changed 13 months ago by totaamwin32

We need to either shadow the console (so remote users can login from the login screen) or find a way to create a session programmatically.

More pointers:

Last edited 12 months ago by Antoine Martin (previous) (diff)

comment:20 Changed 12 months ago by Antoine Martin

Milestone: 3.04.0

Too late for v3.

comment:21 Changed 7 months ago by Antoine Martin

Milestone: 4.05.0

comment:22 Changed 6 months ago by Antoine Martin

Progress made using the openssh server (#2711): we can shadow the Winlogon secure desktop using https://docs.microsoft.com/en-us/sysinternals/downloads/psexec.

More pointers:

  • Opening native graphical application from remote shell: On Windows there is instead an entire zoo of kernel objects called terminal sessions, login sessions, windows stations, desktops, and finally windows, some of which have their own access-control lists and associations with processes. In addition, there are security tokens

Also: can bitvise ssh server overcome the session 0 isolation problem?
No, nothing can get around it. What Bitvise does is properly associate a Windows Station (something which exists within a Desktop, which exists within a Session) with programs so that they will manage to run when invoked headlessly when the server is running within an interactive session.

Last edited 6 months ago by Antoine Martin (previous) (diff)

comment:23 Changed 6 months ago by Antoine Martin

Milestone: 5.04.0

r25977 starts the shadow server on the Winlogon secure desktop, it's more useful than the proxy it replaces since we can actually see the desktop.

To make more progress, it would be useful to:

  • make it easier to run commands on the server (emulate xdg menu data - use code from winswitch?)
  • run in debug mode and send to a log file so it can be tailed and grepped via ssh in real time
  • distinguish the Winlogon secure desktop from real sessions

As for the change of desktop, that's going to be harder.
GTK probably can't be taught to re-initialize against the new display, so we would need to spawn a new process and either give it the existing socket connection, or proxy it.

comment:24 Changed 6 months ago by Antoine Martin

r25980: we can now easily start new commands when connected to a shadow server

comment:25 Changed 5 months ago by Antoine Martin

Maybe the blank display problems (comment:17 IIRC), can be fixed by using a SYSTEM privilege, just like -s with psexec, see ticket:2711#comment:1.

Also: CreateProcessAsUser creates blank/black window: The call to LogonUser generates a new session (and associated logon SID) rather than reusing the existing one, so your process does not have access to the desktop, and only has minimal access to the window station

Last edited 5 months ago by Antoine Martin (previous) (diff)

comment:26 Changed 5 months ago by Antoine Martin

r26188 uses paexec (as per #2711) to start the shadow server.

Tested by reverting r25977 and then connecting to the proxy server using:

xpra shadow "ssl://user:password@server:14500/" --ssl-server-verify=none

Progress!
This now allows the shadow server to access the GUI session, even though it is started from a SYSTEM account.
But only if the user is already logged in.


If the user is not already logged in, the shadow command will just fail:

New ssl connection received
 from 'CLIENTIP:44224'
 on '0.0.0.0:14500'
Authentication required by win32 authenticator module 1
sending challenge for username 'windows 10 test' using xor digest
proxy_auth win32.get_sessions()=(0, 0, [], {}, {})
proxy_auth(Protocol(..), {..}, None) found sessions: (0, 0, [], {}, {})
proxy_session: displays=[], start_sessions=True, start-new-session={b'mode': b'shadow', b'display': b''}
start_new_session('windows 10 test', 0, 0, {b'mode': b'shadow', b'display': b''}, [])
start_win32_shadow('windows 10 test', {b'mode': b'shadow', b'display': b''})
exec_command('windows 10 test', ['paexec.exe', '-i', '1', '-s', 'C:\\Program Files\\Xpra\\Xpra-Shadow.exe', '--bind=windows_10_test', '-d', 'proxy,win32'], {..})
lsa_logon_user(..)
logon_msv1_s4u(windows 10 test)=..
creation_info=<xpra.platform.win32.create_process_lib.CREATIONINFO object at 0x0000000000c9ca40>
Popen(['paexec.exe', '-i', '1', '-s', 'C:\\Program Files\\Xpra\\Xpra-Shadow.exe', '--bind=windows_10_test', '-d', 'proxy,win32'])=<xpra.platform.win32.create_process_lib.Popen object at 0x0000000000ceb070>
poll()=4294967287
stdout=b''
stderr=b''
start_server_subprocess failed
Traceback (most recent call last):
  File "E:\Xpra\trunk\src/xpra/server/proxy/proxy_server.py", line 349, in proxy_session
    proc, socket_path, display = self.start_new_session(username, uid, gid, sns, displays)
  File "C:\Program Files\Xpra\lib\xpra\platform\win32\proxy_server.py", line 49, in start_new_session
    return self.start_win32_shadow(username, new_session_dict)
  File "C:\Program Files\Xpra\lib\xpra\platform\win32\proxy_server.py", line 94, in start_win32_shadow
    raise Exception("shadow subprocess failed with exit code %s" % r)
Exception: shadow subprocess failed with exit code 4294967287
Error: failed to start server subprocess:
 shadow subprocess failed with exit code 4294967287
disconnect(server error, ('failed to start a new session',))

So we need some other call to create the GUI session. No idea which one.

Pointers:

Last edited 5 months ago by Antoine Martin (previous) (diff)

comment:27 Changed 5 months ago by totaamwin32

creating window station and windows desktop using c#

Programmatically create and launch and RDP session (without gui)

All of those solutions use windows forms (GUI based), Creating a Remote Desktop Client Application without using Windows Forms (C#), How to use ActiveX component in ClassLibrary without Winforms.

To compile any of those C# based solutions, we need the AxMSTSCLib.dll.
Where is the AxMSTSCLib library located? The answer (not found in that link) is actually here: SharpRDP: If you do not want to use the provided DLLs you will need to .NET SDK to create the AxMSTSCLib.dll DLL. To create it you'll need to run aximp from the SDK on mstscax.dll. %<SDK dir>%\aximp.exe %windir%\system32\mstscax.dll.
(aximp.exe)

To logoff once we're done: WTSLogoffSession function

Eventually, I would like to compile this DLL from the command line (Command-line build with csc.exe), as part of the build process.

comment:28 Changed 5 months ago by totaamwin32

Next difficulty: calling this dotnet code from the python server without using pythonnet since that's not supported under mingw.

Options:

comment:29 Changed 5 months ago by Antoine Martin

Updates in r26196 + r26197 including the VS project to create the DesktopLogon.dll.
Reverting r25977 should work, but paexec is now failing (was working fine before..)

The server log files can be located using xpra info | grep log-file.
On my win10 test system, they end up in: C:\Windows\System32\config\systemprofile\AppData\Local\Xpra.
(both the Xpra-Proxy.log and Xpra-Shadow.log)

Changed 5 months ago by totaamwin32

Attachment: win32-system-proxy.patch added

switch back to launching the proxy

comment:30 Changed 5 months ago by Antoine Martin

With the patch above, the proxy tries to run:

paexec.exe -i 1 -s "C:\Program Files\Xpra\Xpra-Shadow.exe" --bind=Windows_10_Test -d win32,proxy

When trying this command from an ssh session, paexec now fails (used to work?!):

Remote app failed to start.  Returned error:
  Failed to start "C:\Program Files\Xpra\Xpra-Shadow.exe" --bind=Windows_10_Test -d win32,proxy. Access is denied. [Err=0x5, 5]

PAExec returning exit code -9

Then I also got File not found - that one was completely misleading, nothing to do with the path, just the -i argument needed a different session id!

Changed 5 months ago by totaamwin32

Attachment: sessions.py added

various ill-fated attempts to get the list of sessions

comment:31 Changed 5 months ago by totaamwin32

Sort of working after:

  • r26210 : xpra service uses the installation path we place in the registry
  • r26211 : we can't use the systray on the login screen (avoid errors)
  • r26212 : default to using the paexec we bundle (don't rely on psexec being installed)
  • r26213 : don't log the password, also show stdout and stdin when process exited without an error
  • r26214 : python half of the desktoplogon dll
  • r26215 + r26216 : wts ctypes definitions
  • r26219 + r26222 : locate the correct session id to use for the username given (fixes issue from comment:30)
  • r26220 : disable debug
  • r26221 : to be able to access the password and use it again, win32_auth keeps hold of it - needs a better solution
  • r26225 : exit-with-children by default, poll the named-pipe, etc..
  • r26226 : we can't use multiprocessing with the proxy server on win32, so use a threaded proxy instead
  • r26227 : switch back to running the proxy server as the service

TODO:

  • proxy shows up a cmd window
  • local named pipe shadow connection chokes on memoryview:
    Error: write connection \\.\pipe\Xpra\test reset
     bytes or integer address expected instead of memoryview instance: 0
    
  • proxy becomes unresponsive
  • activex Logon doesn't work? (doesn't run on the correct desktop?)
  • nvenc slows down server startup? (causes timeouts?)
  • shadow server errors when the login screen is shown (cursor, capture, etc) - rate limit them?
  • fix multiprocessing:
  • generic way to keep hold of the password (so we can use this with other authentication modules)
  • stop proxy using xpra stop to prevent zombies? (needs peercred, or a private socket)
  • fix event messages using message files
  • add a proxy service config file so we don't need to modify win32_proxy_service.py to change settings (ie: debug), and rename to just "xpra service" since we can then configure "shadow" or "proxy" on demand
  • shadow server exit does not remove tray (not until the pointer hover over it)
  • add icon to proxy service as per How do I add an icon to a mingw-gcc compiled executable?

Other APIs and projects which may still be useful:

Last edited 5 months ago by Antoine Martin (previous) (diff)

comment:32 Changed 5 months ago by Antoine Martin

Updates:

  • r26228 also update default command (oops)
  • r26231 fix named pipe connections handling of memoryview
  • r26232 + r26233: more ctypes API definitions
  • r26234 (wrong commit message): make the proxy server's named-pipe private
  • r26235 try to force redirection of shadow instances output
  • r26237 + r26238 + r26241 (fixup) hide the cmd window when lauching the proxy / paexec
  • r26239 rate limit cursor warnings
  • r26240 handle workarea calculation errors
  • r26242 rename, generalize and stop the proxy using an xpra stop command

New weird bug: the service responds on boot (if auto-start is on), responds if started using the service manager when logged in, but it seems to die when a user logs out? (only if it had started a shadow server? even if that process is now gone.. maybe a SIGPIPE?)

So the DesktopSession.dll hack still doesn't work, but using rdesktop -i $USER -p $PASSWORD $IP does what we need ... Just need to emulate it? Without actually showing anything via RDP, and without terminating the session... because disconnecting RDP stops our server receiving anything - the station / desktop are suspended? (logging in resumes the session!)

New links:

Last edited 5 months ago by Antoine Martin (previous) (diff)

comment:33 Changed 5 months ago by Antoine Martin

Resolution: worksforme
Status: assignedclosed
Summary: win32 system shadow servicewin32 system service

This will have to do for this release, follow up in #2756.

The proxy server does work and can start a shadow server for any user, but it is unable to create the station / desktop if one does not exist already..

Note: See TracTickets for help on using tickets.