python3 tracker ticket: #1568.
TODO:
etc..
This may fix issues like #761 and #404, worth a try.
Does the GTK3 macos port support opengl though? (#1569)
@smo: any updates?
Needed for window transparency: #1570
See also #1708: 2.3 moduleset updates
@smo: please put anything you have already done in this ticket, then I can take a look.
I tried to build the up to date moduleset but had a problem with pygobject. Here is my contact to the mailing list which didn't really go anywhere.
https://mail.gnome.org/archives/gtk-osx-devel-list/2017-September/msg00000.html
Using a clean user account and following the build instructions from https://wiki.gnome.org/Projects/GTK+/OSX/Building:
wget --no-check-certificate \ "https://git.gnome.org/browse/gtk-osx/plain/gtk-osx-build-setup.sh" sh ./gtk-osx-build-setup.sh PATH=/Users/gtk3/.local/bin:$PATH jhbuild bootstrap
Got a few certificate errors with: xz, pkg-config, flex, expat, etc (download without checking certs and continue). Then it fails to build python3 with another cert error:
jhbuild buildone python3 jhbuild buildone: could not download http://git.gnome.org/browse/gtk-osx/plain/modulesets-stable/gtk-osx.modules: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)>
This time fixed by downloading the cacert.pem
from here:
https://curl.haxx.se/docs/caextract.html and then telling jhbuild to use it:
CA_CERT_FILE=$HOME/cacert.pem jhbuild build python3
Make python3 the default (as per docs - though the upstream instructions look wrong for the second symlink):
jhbuild shell cd $JHBUILD_PREFIX/bin ln -s python3 python ln -s python3-config python-config
jhbuild build openssl
(upstream notes apply: interrupt the build and run it by hand - we should upstream our openssl 1.1 build commands)
jhbuid build meta-gtk-osx-bootstrap meta-gtk-osx-gtk3
Got a minor failure building glib:
xzcat -d "/Users/gtk3/gtk/source/pkgs/glib-2.52.2.tar.xz" | tar xf - *** Applying patch https://git.gnome.org/browse/gtk-osx/plain/patches/glib-gint64-long-long.patch *** [14/39] patch -p1 < "/Users/gtk3/.cache/jhbuild/glib-gint64-long-long.patch" patching file configure.ac Hunk #1 succeeded at 3002 (offset 48 lines). Hunk #2 succeeded at 3114 (offset 48 lines). Hunk #3 succeeded at 3266 (offset 49 lines). *** Applying patch https://git.gnome.org/browse/gtk-osx/plain/patches/glib-2.52-Fallback-to-CFStringGetCSTring-if-CFStringGetC.patch *** [14/39] patch -p1 < "/Users/gtk3/.cache/jhbuild/glib-2.52-Fallback-to-CFStringGetCSTring-if-CFStringGetC.patch" patching file gio/gosxappinfo.c patching file gio/gosxcontenttype.c Hunk #1 succeeded at 52 (offset -6 lines). patching file gio/tests/contenttype.c Hunk #1 succeeded at 323 with fuzz 1 (offset -38 lines). Hunk #2 FAILED at 359. 1 out of 2 hunks FAILED -- saving rejects to file gio/tests/contenttype.c.rej *** Error during phase checkout of glib: ########## Error running patch -p1 < "/Users/gtk3/.cache/jhbuild/glib-2.52-Fallback-to-CFStringGetCSTring-if-CFStringGetC.patch" *** [14/39]
Applied the one line from the reject file by hand and continued.
Everything built correctly after that.
@smo: whatever you did (which was not recorded anywhere AFAICT) must have been different?
Continuing from there:
cat > $HOME/.jhbuildrc-custom << EOF moduleset="https://xpra.org/svn/Xpra/trunk/osx/jhbuild/modulesets-stable/gtk-osx.modules" EOF
jhbuild buildone sshpass -f jhbuild build meta-subversion-xpra jhbuild build meta-osx-xpra-pkgtools jhbuild build meta-osx-xpra-codec-deps
Got one failure we should fix on lame:
gzip -dc "/Users/gtk3/gtk/source/pkgs/lame-3.99.5.tar.gz" | tar xf - *** Applying patch http://xpra.org/svn/Xpra/trunk/osx/jhbuild/patches/lame-channels.patch *** [8/23] patch -p0 < "/Users/gtk3/.cache/jhbuild/lame-channels.patch" can't find file to patch at input line 3 Perhaps you used the wrong -p or --strip option? The text leading up to this was: -------------------------- |--- a/libmp3lame/set_get.c |+++ b/libmp3lame/set_get.c -------------------------- File to patch: Skip this patch? [y] Skipping patch. patch: **** malformed patch at line 10:
Next we have to build python and switch back to python3 afterwards so we can build the python deps against python3:
jhbuild buildone python jhbuild shell cd $JHBUILD_PREFIX/bin rm python python-config ln -s python3 python ln -s python3-config python-config
Build the rest:
jhbuild build meta-osx-xpra-python-deps
lz4 had a problem (will fix). Then python3 had a problem finding ssl, it just needed to be rebuilt now that we brought in a newer openssl lib version:
jhbuild buildone python3 -f
At that point, everything is built OK, AFAICT.
Still TODO:
@smo: why are we so far behind https://git.gnome.org/browse/gtk-osx/tree/? Even gtk2 is outdated here. (only our gstreamer and network modulesets are slightly ahead) Is there anything there that we should not merge and why?
I've merged most of the changes:
ImportError
does it)
ln -sf pycairo.h /Users/gtk3/gtk/inst/include/pycairo/py3cairo.h
Also added latest gdb by hand:
wget https://ftp.gnu.org/gnu/gdb/gdb-8.0.1.tar.gz tar -zxvf gdb-8.0.1.tar.gz cd gdb-8.0.1 ./configure --prefix=${JHBUILD_PREFIX} make && make install sudo chgrp procmod ${JHBUILD_PREFIX}/bin/gdb sudo chmod g+s ${JHBUILD_PREFIX}/bin/gdb
But... still needs signing to be used: codesign gdb.
More work on this:
Still TODO:
GtkMenuBar
invalid cast warnings, somehow
Apart from that, it looks usable. Beta builds posted here with the "Python3" filename: https://xpra.org/beta/osx/.
Notes after rebuilding again from scratch to verify:
find . -name "Makefile" -exec sed -i '' 's+-Wl,-stack_size,1000000 ++g' {} \;
+--- a/libxml2-2.9.4/python/types.c 2016-02-09 03:17:33.000000000 -0700 ++++ b/libxml2-2.9.4/python/types.c 2016-12-21 12:34:06.755650986 -0700 +@@ -31,8 +31,6 @@ + const char *mode; + + fd = PyObject_AsFileDescriptor(f); +- if (!_PyVerify_fd(fd)) +- return(NULL); + /* + * Get the flags on the fd to understand how it was opened + */
Saves having to temporarily switch back to python2 to build it.
The python2-only builds started hitting the issue from comment:7, and now the mixed python2-python3 builds do too. Sigh.
One more weird issue started happening: the "py2app" step does something that loads gdk and tries to access the display server and crashes when it fails, means that the builds now have to be executed from a GUI shell. PITA.
So at this point, the python3 jhbuild env is almost straightforward (bar python3 vs python issues) with this file:
cat > .jhbuildrc-custom <<EOF _gtk_osx_use_jhbuild_python = True setup_sdk(target="10.10", sdk_version="10.10", architectures=["x86_64"]) os.environ["CC"] = "/usr/bin/gcc" os.environ["DYLD_LIBRARY_PATH"] = "" build_policy = "updated-deps" modules = [ "openssl", "python3", "yasm", "nasm", "libxml2", "itstool", "gtk-doc", "meta-gtk-osx-bootstrap", "meta-gtk-osx-core", "meta-gtk-osx-gtk3", "libpng", "libcroco", "librsvg","meta-gtk-osx-python", "meta-gtk-osx-themes", "gtk-mac-integration-python", "meta-osx-python3-xpra-deps"] moduleset="https://xpra.org/svn/Xpra/trunk/osx/jhbuild/modulesets-stable/gtk-osx.modules" os.environ["SSL_CERT_FILE"] = "/Users/osx/gtk/inst/etc/ssl/cacert.pem" EOF
Lots more updates (in large part because testing jhbuild is hard, so I commit then fix the numerous mistakes one at a time). I've also sent some questions and pull requests upstream to try to reduce our diff:
Changesets:
With all these changes:
cat > .jhbuildrc-custom <<EOF _gtk_osx_use_jhbuild_python = True setup_sdk(target="10.10", sdk_version="10.10", architectures=["x86_64"]) os.environ["CC"] = "/usr/bin/gcc" os.environ["DYLD_LIBRARY_PATH"] = "" build_policy = "updated-deps" modules = ["meta-osx-xpra-deps"] moduleset="https://xpra.org/svn/Xpra/trunk/osx/jhbuild/xpra-gtk3.modules" os.environ["SSL_CERT_FILE"] = "/Users/osx/gtk/inst/etc/ssl/cacert.pem" EOF
Then just: jhbuild build
.
The only difference for the "GTK2" moduleset is the moduleset URL:
moduleset="http://xpra.org/svn/Xpra/trunk/osx/jhbuild/xpra-gtk2.modules"
Then when building the packages, we have to do:
PYTHON=python3 ./make-all.sh
(or PYTHON=python2
for GTK2 builds)
@smo: does that work for you?
Note: the gi bingings were missing from the python2 builds (meaning no sound), to re-instate them I had to apply this patch: pygobject: remove references to deprecated GI_INFO_TYPE_ERROR_DOMAIN. (basically, just remove all references to GI_INFO_TYPE_ERROR_DOMAIN
then rebuild with introspection enabled)
There might be a better way of doing this.
gtk-engines fails when building for python3/gtk3 is this needed?
checking for GTK... no configure: error: GTK+-2.12 is required to compile gtk-engines *** Error during phase configure of gtk-engines: ########## Error running ./configure --prefix /Users/xpragtk3/gtk/inst *** [47/129]
gtk-engines fails when building for python3/gtk3 is this needed?
It isn't referenced anywhere in our modulesets.
Can you get some info on it? jhbuild info gtk-engines
Name: gtk-engines Module Set: gtk-osx-themes Type: autogen Install version: not installed Install date: not installed URL: http://ftp.gnome.org/pub/GNOME/sources/gtk-engines/2.20/gtk-engines-2.20.2.tar.bz2 Version: 2.20.2 Tree-ID: 2.20.2-d41d8cd98f00b204e9800998ecf8427e Sourcedir: /Users/xpragtk3/gtk/source/gtk-engines-2.20.2 Requires: automake, libtool, make Required by: meta-gtk-osx-themes After: meta-gtk-osx-core
meta-gtk-osx-themes
is found here: https://github.com/GNOME/gtk-osx/blob/master/modulesets-stable/gtk-osx-themes.modules, but isn't referenced anywhere else in the modulesets.
There is a meta-gtk-osx-gtk3-core-themes
, so I've switched to that in r18020.
@smo: does that fix things? Do we need to add icon-naming-utils
?
Started from scratch again today ran into some issues with libxml2. It would always just build the python2 variant and then would also fail with
ld: -stack_size option can only be used when linking a main executable
We can see the configure line here
./configure --prefix /Users/xpragtk3/gtk/inst --libdir="$JHBUILD_LIBDIR" --with-python --with-python-install-dir=/Users/xpragtk3/gtk/inst/lib/python2.7/site-packages
I'm not sure where the --with-python-install-dir comes from it doesn't appear in the moduleset?
If you run configure again and change it to the correct python install dir it seems to install minus having to stop the build to use the same line as comment:11
python -c "import libxml2; print(libxml2)" <module 'libxml2' from '/Users/xpragtk3/gtk/inst/lib/python3.6/site-packages/libxml2.py'>
seems to load properly now
itstool 2.0.2 doesn't work with python3 itstool 2.0.4 works with python3 but segfaults when used building gtk-doc itstool from git seems to work just fine had to install this manually
seems to be some problems with building lame still as well
** Checking out lame *** [105/135] curl --continue-at - -L http://winswitch.org/src/lame-3.100.tar.gz -o /Users/xpragtk3/gtk/source/pkgs/lame-3.100.tar.gz % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 3983 100 3983 0 0 11362 0 --:--:-- --:--:-- --:--:-- 11380 *** Error during phase checkout of lame: file hash is incorrect (expected ddfe36cab873794038ae2c1210557ad34857a4b6bdc515785d1da9e175b1da1e, got 30641cf26a0d4dc26ca6b97e719c52b4c72d49014a09cd28309a9d989a0408a2) *** [105/135]
Hash is wrong but the url is actually invalid for the download. Should this be switched or was there a good reason for having it on winswitch.org?
*** Checking out lame *** [105/135] gzip -dc "/Users/xpragtk3/gtk/source/pkgs/lame-3.100.tar.gz" | tar xf - *** Applying patch http://xpra.org/svn/Xpra/trunk/osx/jhbuild/patches/lame-channels.patch *** [105/135] patch -p1 < "/Users/xpragtk3/.cache/jhbuild/lame-channels.patch" patching file libmp3lame/set_get.c Hunk #1 FAILED at 91. 1 out of 1 hunk FAILED -- saving rejects to file libmp3lame/set_get.c.rej *** Error during phase checkout of lame: ########## Error running patch -p1 < "/Users/xpragtk3/.cache/jhbuild/lame-channels.patch" *** [105/135]
After getting the correct tarball this patch still fails.
If I ignore the patch failing and try to build I get build errors like these
Undefined symbols for architecture x86_64: "_lame_init_old", referenced from: -exported_symbol[s_list] command line option ld: symbol(s) not found for architecture x86_64 clang: error: linker command failed with exit code 1 (use -v to see invocation) make[3]: *** [libmp3lame.la] Error 1 make[2]: *** [all-recursive] Error 1 make[1]: *** [all-recursive] Error 1 make: *** [all] Error 2 *** Error during phase build of lame: ########## Error running make -j 3 *** [105/135]
gst-plugins-base has issue building the examples
libtool: link: unable to infer tagged configuration libtool: error: specify a tag with '--tag'
Don't build the examples with --disable-examples
configure flag and everything builds fine.
scons doesn't build under python3
*** Building scons *** [131/135] python setup.py build scons: *** SCons does not run under Python version 3.6.4. Python 3 and above are not yet supported. *** Error during phase build of scons: ########## Error running python setup.py build *** [131/135]
need this to build serf and subversion
Bunch of warnings I noticed as well when you start the build
W: meta-osx-xpra-python3-deps has a dependency on unknown "gst-python" module W: meta-osx-xpra-python3-deps has a dependency on unknown "python3-pyasn1" module W: meta-osx-xpra-python3-deps has a dependency on unknown "python3-pyasn1" module W: python3-pyasn1-modules has a dependency on unknown "python3-pyasn1" module W: python3-pyasn1-modules has a dependency on unknown "python3-pyasn1" module W: python3-ldap has a dependency on unknown "python3-pyasn1" module W: python3-ldap has a dependency on unknown "python3-pyasn1" module
Started from scratch again today ran into some issues with libxml2. It would always just build the python2 variant and then would also fail with
Weird, I didn't hit this one.
itstool from git seems to work just fine had to install this manually
Again, I didn't hit this one. (I'm building on 10.10) We'll just wait for the next release and update itstool in the moduleset then.
seems to be some problems with building lame still as well
Yep. It's an upstream bug:
gst-plugins-base has issue building the examples
Fixed in r19356.
I hit a more fundamental issue rebuilding gstreamer:
GEN Gst-1.0.gir (..) ImportError: dlopen(/Users/gtk3/gtk/inst/lib/gobject-introspection/giscanner/_giscanner.so, 2): Symbol not found: _PyInt_FromLong Referenced from: /Users/gtk3/gtk/inst/lib/gobject-introspection/giscanner/_giscanner.so Expected in: flat namespace in /Users/gtk3/gtk/inst/lib/gobject-introspection/giscanner/_giscanner.so
No idea when the broke, or why.
Had to rebuild gobject-introspection
which caused dozens of other modules to get rebuilt.
Then something else was not quite right - looks like some headers aren't installed by gst-plugings-base
, gst-libav
fails with:
/Users/gtk3/gtk/inst/include/gstreamer-1.0/gst/audio/audio.h:36:10: fatal error: 'gst/audio/gstaudioaggregator.h' file not found #include <gst/audio/gstaudioaggregator.h> ^ 1 error generated.
Fixed by:
cp ${JHBUILD_SOURCE}/gst-plugins-base-1.14.1/gst-libs/gst/audio/gstaudioaggregator.h ${JHBUILD_PREFIX}/include/gstreamer-1.0/gst/audio/
Then after that:
In file included from /Users/gtk3/gtk/inst/include/gstreamer-1.0/gst/audio/audio.h:36: /Users/gtk3/gtk/inst/include/gstreamer-1.0/gst/audio/gstaudioaggregator.h:27:10: fatal error: 'gst/base/gstaggregator.h' file not found #include <gst/base/gstaggregator.h> ^ 1 error generated.
Fixed by:
cp ${JHBUILD_SOURCE}/gstreamer-1.14.1/libs/gst/base/gstaggregator.h ${JHBUILD_PREFIX}/include/gstreamer-1.0/gst/base/
Bunch of warnings I noticed as well when you start the build pyasn1 ...
r19355 fixes that.
They seem to have a fix for the lame issue over at homebrew site
https://github.com/Homebrew/homebrew-core/blob/master/Formula/lame.rb
# Fix undefined symbol error _lame_init_old # https://sourceforge.net/p/lame/mailman/message/36081038/ inreplace "include/libmp3lame.sym", "lame_init_old\n", ""
Just remove the offending line from include/libmp3lame.sym and all is well. I'll add a patch.
r19418 adds the patch to the repo r19419 adds it to the moduleset
lame now builds without error
this ticket has been moved to: https://github.com/Xpra-org/xpra/issues/1575