python 3.7 support

[Antoine Martin]

Python 3.7 is due to be released soon: ​Python 3.7.0rc1, there are a number of things we can take advantage of, some may allow us to drop existing workarounds.

The most interesting features:

• ​PEP 538 -- Coercing the legacy C locale to a UTF-8 based locale and ​PEP 540 -- Add a new UTF-8 Mode may help sort out the mess of the UTF8 encode / decode dance, or is too late already?
• ​PEP 564 -- Add new time functions with nanosecond resolution: we could replace our monotonic_time with time.monotonic_ns - but again, the damage has been done and this will be difficult to merge with the cython code which uses cimport

[Antoine Martin]

Eventually, it would be nice to be able to compile with the new buffer interface and avoid deprecation warnings like #1884.

In the meantime:

• r19908 silences the deprecation warnings
• r19909 switches to python 3.7 on win32 (now the default with msys2)

New problem: OpenGL_accelerate doesn't build - see #1883.

[Antoine Martin]

There's a major blocker with cx_freeze, once again falling flat on its face.
Looks like it does not support python 3.7 yet, and this also affects ​pyinstaller: zipimport.ZipImportError: can't find module 'encodings'.

$ ./Xpra_cmd.exe attach tcp:192.168.1.5:10000
Fatal Python error: init_sys_streams: can't initialize sys standard streams
zipimport.ZipImportError: can't find module 'io'

Moving the missing modules by hand into the "library.zip" doesn't really help and the resulting executable still fails with:

$ ./Xpra_cmd.exe attach tcp:192.168.1.5:10000
zipimport.ZipImportError: can't find module '__startup__'

BTW, those modules were already hard-coded in setup.py to workaround existing cx_freeze breakage:

            #hopefully, cx_Freeze will fix this horror:
            #(we shouldn't have to deal with DLL dependencies)
            import site
            lib_python = os.path.dirname(site.getsitepackages()[0])
            lib_dynload_dir = os.path.join(lib_python, "lib-dynload")
            add_data_files('', glob.glob("%s/zlib*dll" % lib_dynload_dir))
            for x in ("io", "codecs", "abc", "_weakrefset", "encodings"):
                add_data_files("lib/", glob.glob("%s/%s*" % (lib_python, x)))

That's amongst a few other workarounds (base_prefix, numpy, etc)

[Antoine Martin]

cx_freeze can be fixed with this patch: ​Compilation correct .pyc files for Python 3.7.0

Created a mingw ticket for this: ​cx_Freeze for python3.7 needs a patch - bytecode is unusable

[Antoine Martin]

The MSYS2 package has been updated with the patch and we have workarounds for the rest.

[Antoine Martin]

One of the few noisy deprecation warnings left is from ​https://github.com/pyparsing/ (fixed in 2.2.1 - ​Fix collections ABCs deprecation warning).

[Antoine Martin]

For websockify, see attachment/ticket/1568/websockify-0.8-py37.tar.xz

[migration script]

this ticket has been moved to: ​https://github.com/Xpra-org/xpra/issues/1878