building all the dependencies from source on win32

[Antoine Martin]

Split from #90 (see also #640). Similar to #533 for OSX.

Note: cross compiling is not an option because of the Cython extensions which seem to confuse mingw..

Some links:

• ​Compiling the GTK+ (and Clutter) stack using Visual C++ 2008 and later
• #90 has a lot of information
• Some links in this answer to ​Where can I download precompiled GTK+ 3 binaries or windows installer?
• ​http://www.tarnyko.net/repo/gtk3_build_system/
• ​http://www.gtk.org/download/win32_tutorial.php
• ​http://opensourcepack.blogspot.com/p/pygobject-pygi-aio.html
• ​https://wiki.gnome.org/action/show/Projects/PyGObject
• ​msys2
• ​https://github.com/Alexpux/MINGW-packages
• ​http://tesla.duckdns.org/jewelkit-1-0-released/

[Antoine Martin]

Note: this is required for ticket:263#comment:12.

[Antoine Martin]

(added new links found on the mailing list)

[Antoine Martin]

​GTK+3 build environment for Win32/Win64 - temporary fork for upstreaming

[Antoine Martin]

GTK3 is not looking good.

[Antoine Martin]

All GTK3 things will now be tracked in #640, we still want to do this for GTK2.

[Antoine Martin]

A solution based on hexchat's ​https://github.com/codekiddy2/Visual-Studio-gtkmm/wiki.
Also: ​https://github.com/fanc999/gtk-msvc-projects - MSVC Projects to build GTK+ from Stock Installation of MSVC 2008-2013

See this thread: ​https://www.mail-archive.com/gtkmm-list@gnome.org/msg19515.html.

[Antoine Martin]

Becoming more urgent every day (missing out on fixes and new features added since 0.24.10! that's lots of them), ie: ticket:1131#comment:5

[Antoine Martin]

The latest hexchat scripts can be found here: ​https://github.com/hexchat/gtk-win32, for GTK3 see ticket:640#comment:36

[Smo]

I've gone ahead with getting a machine ready to compile what we can with the hexchat scripts.

Not done yet but maybe we can get a headstart by using the precompiled binaries from hexchat for testing?

​https://hexchat.readthedocs.org/en/latest/building.html

​https://dl.hexchat.net/gtk-win32/vc14/x86/gtk-Win32.7z
​https://dl.hexchat.net/gtk-win32/vc14/x64/gtk-x64.7z

[Antoine Martin]

Building GTK itself is documented here: ​https://github.com/hexchat/gtk-win32/blob/master/README.md.
It does include all the latest versions, which is great.
(re-scheduling)

[Antoine Martin]

​Build GStreamer on Windows: A Guide

[Antoine Martin]

There are 3 tickets that should be consolidated into one: #917, #678 and #300

[Antoine Martin]

We'll probably have to also make 64-bit builds in order to support newer CUDA and NVENC versions for shadowing (#558) as the latest sdks don't support 32-bit..

Preliminary steps:

• ​hexchat build setup instructions, but install python in the more standard location (root of C drive) and choose Python 2.7 and 3.4 (3.5 is not supported yet by cx_freeze or the gi bindings)
• install ​Microsoft Visual C++ Compiler for Python 2.7
• install ​tortoisesvn for accessing the xpra source code (optional)
• install ​pip for python 2.7
• using pip, install most of the python wiki/Dependencies for both python 2.7 and 3.4 (the exact names to use may vary slightly in the ​pypi registry), some special cases:
  • py2exe 0.6.9 installed from exe for python 2.7, not needed with python 3.x
  • pywin32 installed from exe
• problematic dependencies: lz4, lzo, pillow, cryptography
• build x264, libyuv, vpx and ffmpeg and place them in XPRA_WIN32_BUILD_LIB_PREFIX

[Smo]

From the hexchat build setup instructions I installed Visual Studio 2015 Community and didn't end up with what was needed for building.

Should install the 2nd one instead Visual C++ Build Tools 2015

It also installs less stuff and installs much faster.

[Smo]

Also since it only uses msys2 for tools like patch etc.. it is not necessary to have the 64bit version

[Smo]

Following instructions for hexchat I can now build both win32 and x64 builds of gtk and the other libraries needed for Hexchat.

Hexchat's build system does this by using vcproj files to facilitate building from the command line.

We will have to do the same for some of the libraries that you listed as problematic since we'd like to build these all with msvc as well.

[Antoine Martin]

Blocker for #1072 and #1230

[Antoine Martin]

Milestone renamed

[Antoine Martin]

We'll have to make sure everything runs obviously, so this should cover #628.

[Antoine Martin]

Edited a brain fart: ​building the GTK bundles from source

Then I noticed that there are also some files that are supposed to build pygtk2 for win32: ​https://github.com/Alexpux/MINGW-packages/tree/master/mingw-w64-python2-pygtk
And jhbuild was used at least once for cross compiling: ​Cross Compiling Jhbuild.
And there's even a jhbuild for windows: ​http://afuera.me.uk/jhbuild-windows/

Another thing worth considering: do we want to continue to support 32-bit windows? Maybe just leave that for the 1.0.x branch and focus on 64-bit? (so we can use more modern things like AVX acceleration in openssl, NVENC, etc)

[Antoine Martin]

Closed #300 and #917 as duplicates of this ticket.

I now have two machines with everything installed as per hexchat's ​building the GTK bundles from source. One 32-bit VM ("Windows 7 Pro x86") for regular builds and one 64-bit dedicated system ("Windows 7 Pro x64") with an Nvidia card for NVENC and hardware testing.

Note: I had to install "Visual Studio 2015 Community" and not "Visual C++ Build Tools 2015". The latter caused build failures, it seems that the directory layout does not match the expectations of the powershell build script.

Next we need to build a number of extra packages, some of those may be available via MSYS2 or from here: ​https://github.com/Alexpux/MINGW-packages/tree/master/, others may be found here: ​https://github.com/codekiddy2/Visual-Studio-gtkmm/wiki/Packages, OTOH:

• gtkglext + pygtkgl
• yasm / nasm
• openssl
• gmp / mpfr?
• libjpeg-turbo?
• gstreamer, its plugins and their dependencies - we can make progress without this first, and we may be able to get by using the binaries if needed - this may help: ​Build GStreamer on Windows: A Guide
• x264 / ffmpeg / libvpx and their dependencies
• python (probably easier to just call setuptools / pip): cython, rencode, netifaces, python-cryptography, pillow, numpy, lz4, lzo, pyopengl, pycuda, pynvml, gi, py2exe / cx_freeze
• python win32: pywin32, comtypes, wmi
• opencv?

How do we update this build script? fork it? (PITA) or maintain an overlay?
ie: we want a newer libpng.

The packaging scripts will also need to be modified to find (py)GTK in its new location.

[Antoine Martin]

For setting up a pure mingw development environment (here for 32-bit):

pacman -S base-devel msys2-devel mingw-w64-i686-toolchain

Since the packages exist in MSYS2, why can't we just use that? (and those are also built from source if needed - or we can tweak the PKGBUILD file).

Here's a list of versions found as of today here: ​https://github.com/Alexpux/MINGW-packages, looks like things get updated fairly regularly too:

• pygtk2 2.24.0 and all its dependencies - this is already using the latest versions too:
  • gtk 2.24.31
  • glib 2.50.2
  • gdk-pixbuf 2.36.0
  • pygoject2 2.28.6
  • libjpeg-turbo 1.5.1
  • libpng 1.6.26 (needs updating)
• winpthreads (useful for ffmpeg and others)
• orc 0.4.26
• yasm 1.2.0
• x264 r2721.72d53ab
• ffmpeg 3.2: will need tweaking to reduce the dependencies as per ticket:445#comment:4
• opus 1.1.3, flac 1.3.1, wavpack 4.80.0, gmp 6.1.1, lame 3.99.5, libmad 0.15.1b, libmatroska 1.4.5: all present
• gstreamer 1.8.3: we can trim the plugins during the packaging phase
• gobject-introspection bindings 1.50.0
• lz4 1.7.1
• lzo2 2.09
• xxhash 0.6.1
• openssl 1.0.2j: no 1.1 yet, no biggie
• python2 2.7.12: 2.7.13 should land soon I expect
• python3 3.5.2

And it even includes the python packages:

• numpy 1.11.1
• pillow 3.3.0
• pywin32 219
• setuptools 28.6.0

extras we may want to use:

• opencv 3.1.0
• xpdf (tools only) 3.04
• evince 3.22.1
• ghostscript 9.20
• switch to libssh / libssh2 instead of executing putty?

Good tip: ​MSVC and MinGW DLLs, or how we can generate lib / def files.

[Antoine Martin]

With the pure mingw approach:

• looks like we would need the enum vs gflags patch (shouldn't this be upstreamed?): ​python-gobject has wrong types
• pywin32 will have to be replaced: ​751: mingw-w64-python-pywin32 is on git but not in the repos, or install using a wheel package? or using ​unoffical windows binaries

We use it for:

• win32con everywhere..
• win32api.SetConsoleTitle
• win32console in wait for input
• win32api.EnumDisplayMonitors, win32api.MonitorFromWindow, win32api.GetMonitorInfo: gui screen detection
• win32com.propsys (win32_propsys_set_group_leader): window hooks
• win32api.GetSystemMetrics, gdi32.GetDeviceCaps, win32gui.GetDoubleClickTime: settings
• win32api.GetWindowLong / win32gui.SetWindowLong: gui
• win32api.ClipCursor: grabs
• win32gui.FindWindow, win32api.SendMessage in show_desktop
• win32api.MapVirtualKey, win32api.GetAsyncKeyState, win32api.GetKeyState, win32api.GetKeyboardLayoutList, win32api.GetKeyboardLayout, win32gui.SystemParametersInfo: keyboard
• win32com.shell.SHGetFolderPath, win32api.RegOpenKey, win32api.RegQueryValueEx: paths, etc
• win32print.EnumPrinters: printing
• win32process.GetWindowThreadProcessId, win32gui.GetWindowText, win32gui.GetWindowRect, win32gui.EnumWindows, win32gui.GetDesktopWindow, win32gui.GetWindowDC, win32ui.CreateDCFromHandle, etc. for shadow
• winxpgui and win32gui, win32gui.Shell_NotifyIcon, win32api.GetCursorPos: tray
• win32ts.WTSRegisterSessionNotification / win32ts.WTSUnRegisterSessionNotification: events
• comtypes webcam

It is a lot, but managing all the builds provided by MSYS2 ourselves is probably worse..
Most of these calls can be changed to ctypes relatively easily.
It may help to write unit tests for these functions to make sure we re-implement them correctly.

Easily installed with pacman:

• yasm, lz4, lzo2, xxhash, openssl
• python modules: python2-numpy, python2-pillow, cython2 (only at 0.24.1)

Then with setuptools, these python modules:

• rencode, python-lz4, xxhash, cryptography, nvidia-ml-py, netifaces
• comtypes, wmi (will these actually work without pywin32?)

Python modules that need fixing:

• lzo (needs pointer to the C source code)
• py2exe or cx_freeze
• gtkglext + pygtkglext: the big ones, if that's too hard then we can look at GTK3 again (we have the GTK source so we can build cython extension if we need the speed)
• pycuda (needs CUDA installed first)

[Antoine Martin]

Added:

• subversion, cmake
• python2-cx_Freeze
• libvpx, x264, ffmpeg
• gst-plugins-good, gst-plugins-bad, gst-plugins-ugly

Added using their own installer:

• ghostscript
• gsview

Added with easy_install:

• PyOpenGL + PyOpenGL_accelerate

Needs sorting out:

• uglifyjs + yuicompressor
• libyuv (see patch and cmake . && make)
• websockify (installed OK via easy_install, but needs our patches for win32)

With those changes and also some minor fixes to the build file: r14673, r14674, r14675, r14676
I can build xpra (as long as I replace win32con with a handmade copy) using:

python ./setup.py build_ext --inplace --without-enc_x265
python ./setup.py install_exe --install=./install-dir

The x265 plugin crashes hard, probably a bug in our code - we should either disable it or remove it altogether.

The resulting executables are almost usable, apart from the pywin32 bits missing.. (and pygtkglext)

[Antoine Martin]

(incomplete) patch for compiling libyuv with mingw

[Antoine Martin]

gtkglext and pygtkgl

Install:

• gtk-doc
• gobject-introspection
• gtkglext

Download pygtkglext (the download links on the ​gnome gtkglext project site are dead):

git clone https://github.com/GNOME/pygtkglext
cd pygtkglext
#remove sanity check which would fail:
sed -i -e 's/if not pkgc.*/if False:/g' setup.py
#tell the build tools where to find codegen:
export PYTHONPATH=${MINGW_PREFIX}/share/pygobject/2.0/
#fix the pygtk dsextras class to support gcc 6:
wget "https://xpra.org/trac/raw-attachment/ticket/678/dsextras-gcc.patch"
patch -p1 -d ${MINGW_PREFIX} < dsextras-gcc.patch
#build pygtkglext:
python ./setup.py build
#install phase fails on "add_template_option", see #147 for details
#so do it by hand:
rsync -rplogtv build/lib.mingw-2.7/gtk/* ${MINGW_PREFIX}/lib/python2.7/site-packages/gtk-2.0/gtk/

With r14677 (detect mingw and use minor build tweaks) + r14678 (honour install directory for pyopengl modules), I get a working pygtkgl installation!

[Antoine Martin]

temporary patching for build with mingw, without any pywin32 modules

[Antoine Martin]

Started the conversion from pywin32 to pure ctypes in r14683 (+fixup in r14684).
The build still runs as well as before on the old build system and we now get an almost usable client on the new mingw32 build system!

Still TODO:

• cryptography packaging: complains about missing "appdirs".
• gi not bundled, so gstreamer is not available - do we want to continue bundling python3 for sound? (could be more difficult now that python2 uses the same cx_freeze codepath?)
• win32_events, keyboard info: lots of pywin32 to convert to ctypes
• paths: registry access, SHGetFolderPath

etc

[Antoine Martin]

A lot more done in r14686 + r14687.

Note done yet:

• shadow server
• named pipes
• systray icon

Not done because more difficult as those need callbacks to enumerate:

• GetKeyboardLayoutList
• EnumDisplayMonitors
• EnumPrinters

[Antoine Martin]

More done in r14689 + r14690 + r14691 + r14692 + r14693.

Still TODO:

• systray
• named pipes
• shadow server
• win32 events listener

[Antoine Martin]

modified build file for python-gobject2 with the gio-types patch

[Antoine Martin]

libpng 1.6.27 build file

[Antoine Martin]

Looks like there's something missing from the PKGBUILD for (py)gtk.
I needed to do this to get pygtk to build:

mkdir /mingw32/include/gtk-2.0/gdk/win32/
cp ./mingw-w64-gtk2/src/gtk+-2.24.31/gdk/win32/gdkwin32keys.h \
      /mingw32/include/gtk-2.0/gdk/win32/

Then for python-gobject2, apart from the gio patch I also needed to skip the check phase: attachment/ticket/678/PKGBUILD.

We should also patch ​zlib seeing the recent CVEs that have been fixed in 1.2.10, but the large number of mingw patches make this non trivial.

---

Most of the missing bits for gstreamer packaging with python 2.7 (assuming we use python2 for sound):

rsync -rplogtv /mingw32/lib/python2.7/site-packages/gi install-dir/
rsync -rplogtv /mingw32/lib/girepository-1.0 install-dir/
rsync -rplogtv /mingw32/lib/gstreamer-1.0 install-dir/

[Antoine Martin]

Still needed to:

• easy_install comtypes

Converted more code to ctypes: r14700 (cosmetic / for testing), r14701 (shadow server / named pipes), r14702 (don't use wmi)
Minor bugfixes: r14696, r14699 (win32 thread deadlock)

All remaining TODO (IIRC):

• systray: should use a guid?
• submit PKGBUILD updates upstream (zlib: maybe just wait for this one)
• packaging: gstreamer 1.0 (with python 2 or 3?)
• fix named pipes and shadow server bugs (may split into another ticket)
• missing group leader API workaround (maybe use cython?)
• 64-bit build system: document end-to-end setup, add CUDA and NVENC bits

[Antoine Martin]

Some minor fixes were needed: r14713 (python3), r14714 (ignore mingw python3 generated files), r14715 (legacy win32 directory may not exist), r14716 (gcc on mingw compile fix).

Install "script":

#choose your target build arch (or do each one in turn):
#32-bit:
#export XPKG="mingw-w64-i686-"
#64-bit:
export XPKG="mingw-w64-x86_64-"
#most packages get installed here: (python, gtk, etc):
pacman --no-confirm -S ${XPKG}python2 ${XPKG}python2-pygtk ${XPKG}gtkglext
#media libraries (more than we actually need):
pacman --no-confirm -S ${XPKG}ffmpeg ${XPKG}gst-plugins-good ${XPKG}gst-plugins-bad ${XPKG}gst-plugins-ugly
#network layer libraries:
pacman --no-confirm -S ${XPKG}lz4 ${XPKG}lzo2 ${XPKG}xxhash
#python3 GStreamer bindings:
pacman --no-confirm -S ${XPKG}gst-python
#development tools and libs for building extra packages:
pacman --no-confirm -S base-devel ${XPKG}yasm ${XPKG}nasm subversion rsync gtk-doc ${XPKG}cmake ${XPKG}gcc ${XPKG}pkg-config ${XPKG}libffi
#python libraries and install and packaging tools:
pacman --no-confirm -S ${XPKG}python2-numpy ${XPKG}python2-pillow ${XPKG}cython2 ${XPKG}python2-setuptools ${XPKG}python2-cx_Freeze
#python3 versions (not all are really needed if just using python3 for sound):
pacman --no-confirm -S ${XPKG}python3-numpy ${XPKG}python3-pillow ${XPKG}cython ${XPKG}python3-cx_Freeze
#using easy-install for python libraries which are not packaged by mingw:
for x in rencode xxhash netifaces lz4 websocket-client comtypes PyOpenGL PyOpenGL_accelerate websockify cffi pycparser cryptography nvidia-ml-py; do
    easy-install-2.7 -U -Z $x
    easy-install-3.5 -U -Z $x
done

remaining issues (some important like pygtkglext, but not blockers):

• cryptography failed to install unless its dependencies are installed separately first (cffi and pycparser)
• pygtkglext: follow comment:27
• libpng update: from source using attachment/ticket/678/PKGBUILD.2
• pygobject rebuild: from source using attachment/ticket/678/PKGBUILD
• install gsview and ghostscript separately (we could use ghostscript as provided by mingw, but gsview isn't.. now is a good time to deal with #1230)
• websockify: still needs patching (not done at all - won't run out of the box..)
• python3 builds need --with-server --with-shadow to help cx_Freeze, and the sound subapp doesn't work at all..

NVENC related installation issues (only available on 64-bit):

• CUDA needs to be installed using its own EXE installer
• NVENC SDK needs to be unzipped somewhere
• pycuda doesn't find CUDA via easy-install

Build xpra into dist:

cd /e
svn co https://xpra.org/svn/Xpra
cd Xpra/trunk/src
python2 ./add_build_info.py
#x265 causes crashes!
BUILD_ARGS=--without-enc_x265
python2 ./setup.py build_ext --inplace ${BUILD_ARGS}
python2 ./setup.py install_exe --install=./dist
#for sound (still broken)
#python3 ./setup.py build_ext --inplace ${BUILD_ARGS}
#python3 ./setup.py install_exe --install=./dist/Sound --with-server --with-shadow

The resulting executables in dist should all run.

New minor TODO: the setup.py rebuilds all the cython extensions, even when not needed. (cython bug?)

@smo: please try this out, you will need a build system you can use as support for the old py2exe one will be removed soon.

[Antoine Martin]

patch for dsextras to support newer gcc versions

[Antoine Martin]

use cython to access shell functions

[Antoine Martin]

For lack of a better place, some good links for debugging Cython / C++ / DLL issues on win32:

• ​Using C++ in Cython
• ​Wrap C++ lib with Cython
• ​Wrapping a DLL: C++ to Cython to Python
• ​Move manifest file to dll?
• ​BUILDING EXTENSIONS FOR PYTHON 3.5
• ​How to check for DLL dependency?
• ​Debugging LoadLibrary Failures
• ​How to use Dependency Walker
• ​CDB Command-Line Options
• ​Win 7, 64 bit, dll problems

Turns out that the dependency on a ​Windows 8.1 Set that I was seeing on my Windows 7 build system (dependency on api-ms-win-core-libraryloader-l1-2-0.dll) was actually caused by the linker flag -lwindowsapp. Replacing it with -lole32 fixes the problem. Obviously. Not.

[Antoine Martin]

failed attempt at implementing IPropertyStore access using a C++ cython module

[Antoine Martin]

The window grouping code (see #799) has (finally) been re-implemented using a C++ helper in r14795.
Notes:

• I'm not sure it is even possible using a plain C cython module
• using a C++ cython module is a pain, and I couldn't make it work
• using a glue C++ file with a Cython module works
• the windows types are a pain to use in cython, especially when so many functions make use of macros... which cython cannot use.

Documentation useful for writing native code:

• ​Windows Programming/Handles and Data Types
• ​MSDN : Common Data Types
• ​Windows Data Types

[Antoine Martin]

modified websockify for win32

[Antoine Martin]

• r14800 silences console handler warnings during cleanup
• r14801 packaging fix for newer versions of python-cryptography, backported in r14802
• r14803 avoid unnecessary file copying during win32 build

New extra setup steps:

• install python lzo, after applying this patch: attachment/ticket/678/python-lzo-envpath-fix.patch:

  pacman -S liblzo2-devel
  export LZO_DIR=$MINGW_PREFIX
  python2 ./setup.py install
  python3 ./setup.py install
  

• websockify (should be upstreamed): untar these modified files attachment/ticket/678/websockify-0.8.0-win32.tar.xz over the 0.8.0 upstream source before running install
• minify html5 client: easy_install-2.7 minify && easy_install-3.5 minify, install ​Java and add java.exe to your PATH before running the build
• libyuv: filed an upstream ticket requesting info: ​build libyuv.dll on win32
• webcam support, totally not obvious: the opencv package is missing some dependencies you have to install by hand:

  pacman -S ${XPKG}opencv ${XPKG}hdf5 ${XPKG}tesseract-ocr
  

• ssh: we could also ship openssh's ssh.exe and sshpass.exe as an alternative to putty, it can be used with xpra attach --ssh=ssh.exe. The problem is that this would bring a large number of dependencies with it:

  cp /c/msys32/usr/bin/ssh.exe dist/
  cp /c/msys32/usr/bin/sshpass.exe dist/
  for x in ssh.exe sshpass.exe msys-gcc_s-1.dll msys-2.0.dll msys-ssp-0.dll msys-crypto-1.0.0.dll \
      msys-z.dll msys-gssapi-3.dll msys-heim* msys-krb5-26.dll msys-asn1-8.dll \
      msys-com_err-1.dll msys-roken-18.dll msys-roken-18.dll msys-crypt* \
      msys-wind-0.dll msys-hx509-5.dll msys-sqlite3*
      cp /c/msys32/usr/bin/$x dist/
  done
  

And also it doesn't have a GUI for prompting for username or password..
Maybe we can add this later as an option. And maybe this should be kept in a separate sub-directory.

---

Notes:

• there is now a dedicated wiki page for setting up the win32 build env: wiki/Building/MSWindows
• it would be nice to deal with #1231 so we can remove the opencv dependency
• 64-builds are now tracked in #1413
• "pacman -Syu" brings a nice set of updates: cython 0.25.2, libpng-1.6.28, gstreamer 1.10.2, gdk-pixbuf 2.36.3, flac 1.3.2, ffmpeg 3.2.2, lz4 1.7.5, numpy 1.11.2, etc

[Antoine Martin]

we still need to patch python-lzo for 1.11 (fixed in trunk)

[Antoine Martin]

r14808 adds a shell script (yes shell!) for generating the installer, just run this from a mingw shell:

./win32/PY27_MINGW_BUILD.sh

To get the EXE installer.

Still TODO:

• remove some DLLs we should not be including "api-ms-win*"
• trim the dlls (do we really need all the opencv bits?)
• fix gdk pixbuf loader cache warning
• replace MSI Wrapper with something better?
• 64-bit tweaks (#1413)

[Antoine Martin]

• r14811 + r14812 + r14814 + r14815: 64-bit improvements (see #1413)
• r14813: non-mingw build fix

The dlls cannot be trimmed further, best to just move to a new webcam API: #1231.
r14811 removed the "api-ms-win-*dll" which came from Tortoise Plink, and fixes the pixbuf loader.
The latest ​windows beta upload was made using this mingw build, and it looks good! (still need to fix SIGINT hander?)

PS: just how awesome is mingw-w64? the latest update brings python-2.7.13 and zlib 1.2.11!

[Antoine Martin]

Minor updates:

• r14835: replace ctypes code with standard "winreg" module
• r14832: fix shadow server screen size detection "dpi aware" (backport in r14834)
• r14837 + r14838: directsound using ctypes

Note: the named pipes code is still broken, so you need to start shadow servers with "--bind=none" to workaround that.

[Antoine Martin]

libyuv

Here's how I figured out how to build the libyuv.dll:

git clone https://chromium.googlesource.com/libyuv/libyuv
cd libyuv
wget "https://www.xpra.org/trac/raw-attachment/ticket/678/libyuv-nojpeg.patch"
patch -p1 < ./libyuv-nojpeg.patch
cmake -G "MSYS Makefiles"
make
#build may fail on "convert.exe",
#if so then remove it from the Makefile,
#then run:

rm libyuv.a;make VERBOSE=1
#copy the long ar.exe command that generates "libyuv.a",
#and modify to build the DLL instead:
gcc -shared -o libyuv.dll \
    CMakeFiles/yuv.dir/source/compare.cc.obj \
    CMakeFiles/yuv.dir/source/compare_common.cc.obj \
    CMakeFiles/yuv.dir/source/compare_neon.cc.obj \
    CMakeFiles/yuv.dir/source/compare_neon64.cc.obj \
    CMakeFiles/yuv.dir/source/compare_gcc.cc.obj \
    CMakeFiles/yuv.dir/source/compare_win.cc.obj \
    CMakeFiles/yuv.dir/source/convert.cc.obj \
    CMakeFiles/yuv.dir/source/convert_argb.cc.obj \
    CMakeFiles/yuv.dir/source/convert_from.cc.obj \
    CMakeFiles/yuv.dir/source/convert_from_argb.cc.obj \
    CMakeFiles/yuv.dir/source/convert_jpeg.cc.obj \
    CMakeFiles/yuv.dir/source/convert_to_argb.cc.obj \
    CMakeFiles/yuv.dir/source/convert_to_i420.cc.obj \
    CMakeFiles/yuv.dir/source/cpu_id.cc.obj \
    CMakeFiles/yuv.dir/source/mjpeg_decoder.cc.obj \
    CMakeFiles/yuv.dir/source/mjpeg_validate.cc.obj \
    CMakeFiles/yuv.dir/source/planar_functions.cc.obj \
    CMakeFiles/yuv.dir/source/rotate.cc.obj \
    CMakeFiles/yuv.dir/source/rotate_any.cc.obj \
    CMakeFiles/yuv.dir/source/rotate_argb.cc.obj \
    CMakeFiles/yuv.dir/source/rotate_common.cc.obj \
    CMakeFiles/yuv.dir/source/rotate_mips.cc.obj \
    CMakeFiles/yuv.dir/source/rotate_msa.cc.obj \
    CMakeFiles/yuv.dir/source/rotate_neon.cc.obj \
    CMakeFiles/yuv.dir/source/rotate_neon64.cc.obj \
    CMakeFiles/yuv.dir/source/rotate_gcc.cc.obj \
    CMakeFiles/yuv.dir/source/rotate_win.cc.obj \
    CMakeFiles/yuv.dir/source/row_any.cc.obj \
    CMakeFiles/yuv.dir/source/row_common.cc.obj \
    CMakeFiles/yuv.dir/source/row_mips.cc.obj \
    CMakeFiles/yuv.dir/source/row_msa.cc.obj \
    CMakeFiles/yuv.dir/source/row_neon.cc.obj \
    CMakeFiles/yuv.dir/source/row_neon64.cc.obj \
    CMakeFiles/yuv.dir/source/row_gcc.cc.obj \
    CMakeFiles/yuv.dir/source/row_win.cc.obj \
    CMakeFiles/yuv.dir/source/scale.cc.obj \
    CMakeFiles/yuv.dir/source/scale_any.cc.obj \
    CMakeFiles/yuv.dir/source/scale_argb.cc.obj \
    CMakeFiles/yuv.dir/source/scale_common.cc.obj \
    CMakeFiles/yuv.dir/source/scale_mips.cc.obj \
    CMakeFiles/yuv.dir/source/scale_msa.cc.obj \
    CMakeFiles/yuv.dir/source/scale_neon.cc.obj \
    CMakeFiles/yuv.dir/source/scale_neon64.cc.obj \
    CMakeFiles/yuv.dir/source/scale_gcc.cc.obj \
    CMakeFiles/yuv.dir/source/scale_win.cc.obj \
    CMakeFiles/yuv.dir/source/video_common.cc.obj
#then we can install everything by hand:
#(a better way would be to work out how to tell cmake about our prefix)
rsync -rplogtv include/libyuv* ${MINGW_PREFIX}/include/
cp libyuv.a  ${MINGW_PREFIX}/lib/
cp libyuv.dll ${MINGW_PREFIX}/bin/
#add a pkgconfig file so the library can be found by build tools:
#(this .pc version is for mingw 32-bit):
wget "https://www.xpra.org/trac/raw-attachment/ticket/678/libyuv.pc"
mv libyuv.pc ${MINGW_PREFIX}/lib/pkgconfig/

For actually building our csc_libyuv module on 64-bit, you will also need r14853 .
Then you can verify that the libyuv codec has been built properly by running the "Encoding_info.exe" tool.

[Antoine Martin]

don't link jpeg with libyuv

[Antoine Martin]

32-bit pkgconfig file

[Antoine Martin]

Note: we may need to submit this patch to mingw for libvpx: gstreamer ​libvpx segfaults on Windows/x86 / upstream: ​libvpx crashes when encoding on x86_32.

[Smo]

$ python2
Python 2.7.13 (default, Jan 17 2017, 13:56:44)  [GCC 6.3.0 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import gi
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named gi

turns out I was missing this one

pacman -S mingw64/mingw-w64-x86_64-python2-gobject

[Antoine Martin]

Added to wiki: https://xpra.org/trac/wiki/Building/MSWindows?action=diff&version=5

[Antoine Martin]

Note: the system update warns you about shortcuts going stale, I just created new ones by dragging "mingw32.exe" / "mingw64.exe" to the desktop.

[Antoine Martin]

The systray code has been moved to its own ticket: #1434

[Antoine Martin]

r15053 uninstalls xpra (if present) before installing a new version. This needs to be backported. Without this change, installing an older version over a 2.x installation breaks everything. (not sure why innosetup doesn't handle this properly)

[Antoine Martin]

Interesting read: ​win32api handle incompatible with ctypes?.
Following at least some this advice, we now use WinDLL: r15244

[Antoine Martin]

Note: for keeping the python libraries up to date, just re-run this again as needed:

for x in rencode xxhash netifaces lz4 websocket-client comtypes PyOpenGL PyOpenGL_accelerate websockify cffi pycparser cryptography nvidia-ml-py; do
    easy_install-2.7 -U -Z $x
    easy_install-3.5 -U -Z $x
done

python-lz4 0.9.0 doesn't build on win32, not the first breakage in this release: #1464.

[Antoine Martin]

r15307 moves the wiki instructions to shell scripts, including an update script for the python libraries, wiki updated: wiki/Building/MSWindows.

[Antoine Martin]

• r15308 fixes a geometry bug in the systray code
• this ticket: ​https://github.com/python-lz4/python-lz4/issues/23 contains the patch for building under mingw

[Antoine Martin]

Minor fixes: r15610, r15609, r15613

[Antoine Martin]

See also #1523

[Antoine Martin]

See also #1528.

[Antoine Martin]

The 2.0.3 release is being held up by some MSYS2 breakage: ​some update broke numpy on 64-bit.

[Antoine Martin]

Breakage fixed, see link in comment:55.

[Antoine Martin]

Works for me. Not heard too many complaints about 2.x win32 builds either.

[Antoine Martin]

pygobject2 patch

[Antoine Martin]

Made a pull request (github's online editor is painful to use for patch files..): ​https://github.com/Alexpux/MINGW-packages/pull/2686

[Antoine Martin]

More regressions caused by the move to ctypes (these method signatures should be generated correctly once and for all): #1545, r16289.
Python3 fix: r16281.

[Antoine Martin]

More pywin32 legacy code removed in r17317.

[Antoine Martin]

Some recent update broke setuptools (that whole thing is a terrible mess), you get ImportError: No module named 'pkg_resources._vendor.packaging'.

The solution I have used to get going again (and hope that MSYS2 fixes the package sooner or later), on 64-bit:

pacman -S mingw-w64-x86_64-python2-pip
rm -fr /mingw64/lib/python2.7/site-packages/setuptools*
pip install setuptools

[Antoine Martin]

FWIW pywin32 may get ported to MSYS2: ​https://github.com/msys2/MINGW-packages/pull/5615

[migration script]

this ticket has been moved to: ​https://github.com/Xpra-org/xpra/issues/678