#840 closed task (fixed)
64-bit builds for OSX using moduleset
Reported by: | Antoine Martin | Owned by: | Smo |
---|---|---|---|
Priority: | blocker | Milestone: | 2.0 |
Component: | platforms | Version: | trunk |
Keywords: | osx | Cc: |
Description (last modified by )
Split from #533. See ticket:533#comment:77
Minor fixes + tweaks already done: r9089, r9090.
Things that need fixing:
- the moduleset needs updating, adding cups, etc as per the comments in #533.
- fix the
pyobjc
build failure - detect when trying to run on a 32-bit system and provide a helpful error message
- add new download links for it
- nice-to-have: libvpx 1.4 (see failure in ticket/832#comment:1, still fails on 64-bit..)
That's it!
Change History (39)
comment:1 Changed 7 years ago by
Description: | modified (diff) |
---|
comment:2 Changed 7 years ago by
Owner: | changed from Antoine Martin to Smo |
---|
See also:
@smo: you may want to post the errors to the pyobjc mailing list, that's the blocker. The rest is relatively easy or optional.
comment:3 Changed 7 years ago by
Lots of builds updates, see ticket:533#comment:79 and later comments.
In particular: r9123 which removes lots of i386
comment:5 Changed 7 years ago by
Updated in r10674 from upstream. We may still need to make further updates to this for our needs.
comment:7 Changed 7 years ago by
Owner: | changed from Smo to Antoine Martin |
---|
Can we reschedule this for a further release. Unless you think 64bit builds are not worth the bother and we can just close this.
comment:8 Changed 7 years ago by
Milestone: | 0.16 → 1.0 |
---|---|
Owner: | changed from Antoine Martin to Smo |
Re-scheduling. Worth doing, maybe after the next LTS so we have both 32-bit and 64-bit.
comment:9 Changed 7 years ago by
Tried this myself today on osx 10.9.5 didn't get as far as I hoped some weird issue here.
(cd /Users/user/gtk/inst/_jhbuild/root-tango-icon-theme/Users/user/gtk/inst/share/icons/Tango/16x16 && /Users/user/gtk/inst/libexec/icon-name-mapping -c actions) Bizarre copy of UNKNOWN in scalar assignment at /Users/user/gtk/inst/lib/perl5/5.16.2/darwin-thread-multi-2level/XML/Parser/Expat.pm line 125. make[3]: *** [install-data-local] Error 25 make[2]: *** [install-am] Error 2 make[1]: *** [install-recursive] Error 1 make: *** [install-recursive] Error 1 *** Error during phase install of tango-icon-theme: ########## Error running make install DESTDIR=/Users/user/gtk/inst/_jhbuild/root-tango-icon-theme *** [41/97]
If I run this utility I also get the same issue.
/Users/user/gtk/inst/libexec/icon-name-mapping -c actions Bizarre copy of UNKNOWN in scalar assignment at /Users/user/gtk/inst/lib/perl5/5.16.2/darwin-thread-multi-2level/XML/Parser/Expat.pm line 125.
Doesn't appear to be a 64bit issue really just some sort of broken perl module?
comment:10 Changed 6 years ago by
Building on 10.11.1 with xcode 7.1 and these were the ones that failed
*** the following modules were not built *** [97/97] libvpx sdl gst-plugins-bad-xpra pygtkglext meta-osx-xpra-deps
sdl was missing headers for X
libvpx needs updating to 1.5.0 which builds fine need to update moduleset
gst-plugins-bad-xpra failed to build with an error about ssl header files we might be able to disable this
In file included from gstapexraop.c:33: ./gstapexraop.h:33:10: fatal error: 'openssl/rand.h' file not found #include <openssl/rand.h> ^ In file included from gstapexplugin.c:29: In file included from ./gstapexsink.h:32: ./gstapexraop.h:33:10: fatal error: 'openssl/rand.h' file not found #include <openssl/rand.h>
pygtkglext failed with this error
gdkglext.override:36:10: fatal error: 'GL/gl.h' file not found #include <GL/gl.h>
comment:11 Changed 6 years ago by
- we don't need SDL
- for libvpx, can we add a moduleset per version, or add one via the customrc file?
- pygtkglext needs the opengl headers and libs to link against, which may have moved: Where are the OpenGL header files located on MacOSX?
comment:12 Changed 6 years ago by
gtk/gdkgl/gdkglext.c
change to #include <OpenGL/gl.h>
afterwards this builds no problem.
I'm not sure about having multiple modulesets at the moment I include it by url in customrc file but one of the files that gets included also has a url we include from svn trunk
comment:14 Changed 6 years ago by
Milestone: | 1.0 → 0.18 |
---|
Should be a priority for the next milestone.
comment:15 Changed 6 years ago by
Still having issues trying to do this now I think its an issue with jhbuild during the configure phase of x264 it produces this stacktrace
*** Configuring x264 *** [72/116] Traceback (most recent call last): File "/Users/user/.local/bin/jhbuild", line 32, in <module> jhbuild.main.main(sys.argv[1:]) File "/Users/user/Source/jhbuild/jhbuild/main.py", line 155, in main rc = jhbuild.commands.run(command, config, args, help=lambda: print_help(parser)) File "/Users/user/Source/jhbuild/jhbuild/commands/__init__.py", line 183, in run return cmd.execute(config, args, help) File "/Users/user/Source/jhbuild/jhbuild/commands/__init__.py", line 52, in execute return self.run(config, options, args, help) File "/Users/user/Source/jhbuild/jhbuild/commands/base.py", line 268, in run return build.build() File "/Users/user/Source/jhbuild/jhbuild/frontends/buildscript.py", line 163, in build error, altphases = module.run_phase(self, phase) File "/Users/user/Source/jhbuild/jhbuild/modtypes/__init__.py", line 420, in run_phase method(buildscript) File "/Users/user/Source/jhbuild/jhbuild/modtypes/autotools.py", line 217, in do_configure cmd = self._get_configure_cmd(buildscript) File "/Users/user/Source/jhbuild/jhbuild/modtypes/autotools.py", line 130, in _get_configure_cmd cmd = self.static_analyzer_pre_cmd(buildscript) + template % vars KeyError: u'libdir'
comment:16 Changed 6 years ago by
Any progress? We can skip x264 and do it by hand if need be.
Blocker for #1231.
comment:17 Changed 6 years ago by
The problem is with libdir
replacement in those strings maybe this isn't needed anymore since if I remove it I can compile this package through jhbuild.
I will commit fix for this shortly to trunk. Looks like a similar issue with ffmpeg as well.
comment:18 Changed 6 years ago by
my current .jhbuildrc-custom for el capitan machine
setup_sdk(target="10.9", sdk_version="10.11", architectures=["x86_64"]) os.environ["CC"] = "/usr/bin/gcc" os.environ["DYLD_LIBRARY_PATH"] = "" build_policy = "updated-deps" modules = [ "python", "libxml2", "itstool", "gtk-doc", "meta-gtk-osx-bootstrap", "meta-gtk-osx-core", "libcroco", "librsvg","meta-gtk-osx-python", "meta-gtk-osx-themes", "gtk-quartz-engine","gtk-mac-integration-python", "gstreamer", "gst-plugins-base", "gst-plugins-good", "meta-gstreamer-1.0", "meta-osx-xpra-deps"] #skip iconv and add some switches to turn off stuff that was breaking skip.append("libiconv") #skip.append("expat") append_autogenargs('gstreamer', '--enable-introspection=no') append_autogenargs('gst-plugins-base', '--enable-introspection=no') append_autogenargs('gst-plugins-good', '--enable-introspection=no --disable-deinterlace') append_autogenargs('gst-plugins-bad-1.0', '--enable-opengl=no --disable-apple_media') #change moduleset moduleset="http://xpra.org/svn/Xpra/trunk/osx/jhbuild/modulesets-stable/gtk-osx.modules"
comment:19 Changed 6 years ago by
This does not look ready to me, or I need some more precise instructions.
Following, more or less: https://winswitch.org/dev/macosx.html but with the new customrc on a 10.10 system. Found some useful commands along the way:
$ sw_vers -productVersion 10.10.5 $ xcrun --show-sdk-version 10.11 $ xcrun --show-sdk-path /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk
And now for the (long) list of problems:
- during boostrap:
ERROR: no certificate subject alternative name matches requested host name `cmake.org'.
- more problematic during the build step:
*** Configuring itstool *** [5/116] (..) checking for python module libxml2... ./configure: line 2422: 16059 Done echo "import $py_module" 16060 Abort trap: 6 | python - >&/dev/null not found configure: error: Python module libxml2 is needed to run this package
Probably the same cause as this other error:
Making all in doc /Users/osx/gtk/inst/etc/xml/catalog:1: parser error : Document is empty I/O error : Attempt to load network entity http://www.oasis-open.org/docbook/xml/4.1.2/docbookx.dtd (..)
And another related one (as I found out much later..):
GISCAN GLib-2.0.gir Traceback (most recent call last): File "./g-ir-scanner", line 53, in <module> from giscanner.scannermain import scanner_main File "./giscanner/scannermain.py", line 29, in <module> import tempfile File "/Users/osx/gtk/inst/lib/python2.7/tempfile.py", line 32, in <module> import io as _io File "/Users/osx/gtk/inst/lib/python2.7/io.py", line 51, in <module> import _io ImportError: No module named _io make[2]: *** [GLib-2.0.gir] Error 1 make[2]: *** Waiting for unfinished jobs.... make[1]: *** [all-recursive] Error 1 make: *** [all] Error 2 *** Error during phase build of gobject-introspection: ########## Error running make -j 2 *** [27/116]
Same for gst-python and most python modules, pango (gtk-doc), gtk-mac-integration, librsvg, gtk-engines, gtk-quartz-engine ..
It tries to build pycairo-python2.6
- that looks wrong.
Looks like we want to force python to build earlier. I found that out later, after much time wasted. Most python modules then build OK.
Maybe just a _gtk_osx_use_jhbuild_python = True
line?
Or maybe I was meant to build meta-osx-xpra-deps
and not just a plain build?
- the gi bindings end up missing, see ticket:1204#comment:4
- gst-plugins-bad errors out on openssl, we need to build openssl earlier
- a lot of the sound codecs seem to get built after the gstreamer plugins, that looks wrong?
- gtkglext errors out with the
KeyError: u'libdir'
problem. - gtk is still at 2.24.28?
- gdk-pixbuf error:
C compiler cannot create executables
- librvg is missing gdk-pixbuf
- gst-plugins-bad-1.0: errors on missing gir files
- python-pyobjc-framework-cocoa errors on
cannot find protocol declaration for 'NSProgressReporting'
,cannot find protocol declaration for 'NSURLSessionStreamDelegate'
- spotted this in gst-libav, which looks wrong:
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk/usr/include/c++/4.2.1
- lots of other modules need to be updated for https (and forced to use curl for download?):
- hicolor-icon-theme now lives here: https://icon-theme.freedesktop.org/releases/hicolor-icon-theme-0.13.tar.gz
- cairo: https://www.cairographics.org/releases/cairo-1.14.0.tar.xz
- py2cairo: https://www.cairographics.org/releases/py2cairo-1.10.0.tar.bz2
- liboil: https://liboil.freedesktop.org/download/liboil-0.3.17.tar.gz
- all the gstreamer downloads, ie: orc got moved to https://gstreamer.freedesktop.org/src/orc/orc-0.4.25.tar.xz
- Cython: https://pypi.python.org/packages/b1/51/bd5ef7dff3ae02a2c6047aa18d3d06df2fb8a40b00e938e7ea2f75544cac/Cython-0.24.tar.gz
- pkg-config
- pixman
- it rebuilds openssl every time, which makes things painfully slow
- subversion has even more problems
- the opengl build error can be solved with
ln -sf /System/Library/Frameworks/OpenGL.framework/Headers /Users/osx/gtk/inst/include/GL
comment:20 Changed 6 years ago by
At some point we will have to switch the existing build environment to a locked version, and move forward with the trunk / 64-bit / newer build env.
Until then, r12959 fixes building ffmpeg but this may also get in the way of supporting webcam capture in the future (we can't have both..).
comment:22 Changed 6 years ago by
FYI: bumped to openssl 1.1.0 in r13481, which requires a newer perl version than the one available on my 10.5.x VM, so then I had to figure out how to build the latest stable perl there:
tar -zxvf perl-5.24.0.tar.gz cd perl-5.24.0 ./Configure -des -Dprefix={JHBUILD_PREFIX} \ -Accflags="-arch i686 -I{JHBUILD_PREFIX}/include/ -L{JHBUILD_PREFIX}/lib/" \ -Aldflags="-arch i686 -I{JHBUILD_PREFIX}/include/ -L{JHBUILD_PREFIX}/lib/" make && make install
Bonus: this version seems to install cleanly without needing a chmod.
Downside: python-cryptography needs updating to 1.5 to support openssl 1.1 : https://cryptography.io/en/latest/changelog/. (done in r13482)
@smo: if that works for you, please add perl to the moduleset.
comment:23 Changed 6 years ago by
Okay I ran into the perl issue as well.
I will add this to the moduleset but should we require it or leave it as an optional thing on older systems that need it?
comment:24 Changed 6 years ago by
Hopefully we will be moving to a newer 64-bit build system soon, so this should be optional.
comment:25 Changed 6 years ago by
r13692:13695 adds perl
Was some issues with the configure line since it is non standard. Not sure if I need to add the cc and ld flags so I left them out for now.
If the new openssl seems to install properly i'll see if I can remove some of the hacks from the moduleset for installing it in the first place.
comment:26 Changed 6 years ago by
Tried with the latest changes to the moduleset on 64 bit. Seems to get further now. I got this error today
build/temp.macosx-10.9-x86_64-2.7/_openssl.c:415:10: fatal error: 'openssl/opensslv.h' file not found #include <openssl/opensslv.h> ^ 1 error generated. error: command '/usr/bin/gcc' failed with exit status 1 *** Error during phase build of python-cryptography: ########## Error running python setup.py build *** [106/119]
Confirmed that openssl is installed.
comment:27 Changed 6 years ago by
Milestone: | 1.0 → 2.0 |
---|---|
Priority: | major → blocker |
comment:28 Changed 6 years ago by
Tried again as part of #1340:
- got an error with librsvg not finding gdk-pixbuf gir
- ffmpeg complained about missing libvpx
- occasionally had to switch things around with code-select: jhbuild requires the developer path but compiling things require the command line tools.. PITA - how do get get both to work at the same time?
comment:30 Changed 6 years ago by
gstreamer 1.10: https://gstreamer.freedesktop.org/releases/1.10/.
comment:31 Changed 5 years ago by
After moving to a newer build host, we can start using a number of new APIs and do:
- #776
- webcam support
etc
comment:32 Changed 5 years ago by
I have pointed my 10.5.x build VM to the stable 1.0.x branch after merging the latest set of safe updates (r14622).
Now we can start "breaking" things again to make 64-bit builds work properly, I am building on 10.10.x with xcode 7.2.1:
- r14623: libvpx 1.6.0 + ffmpeg webcam (+ r14625 to fix the same error as comment:15, fix hinted in comment:17 but without a link to the actual commit)
- r14624: openssl 1.1.x now that we have a newer python 2.7.3 (r14619)
- drop gstreamer 0.10: r14628 + r14629, r14630 + r14632
- 64bit package naming: r14633
After that:
jhbuild buildone -f openssl
to make sure it is rebuilt before python (this should probably be in the moduleset instead)jhbuild build -f
- and wait a while (see minor fixes below)- make a signed build in one swoop (based on the instructions in ticket:1340#comment:19):
security unlock-keychain -p THEACCOUNTPASSWORD export CODESIGN_KEYNAME="Antoine Martin" ./make-all.sh
- the resulting DMG / PKG are not much smaller than before: http://xpra.org/beta/osx
Things left TODO:
- openssl is still a pain to build + install, still requires
chmod +x ${JHBUILD_PREFIX}/lib/libssl.* ${JHBUILD_PREFIX}/lib/libcrypto.*
- the gi bindings are missing and we really need those for gstreamer.. but [ https://github.com/anholt/libepoxy libepoxy] won't build (could be related to Adds support for OS X without X11), and it is required by GTK3, which is needed by the gobject-introspection module... I still managed to build it with
jhbuild buildone -f pygobject3
- gstreamer 1.10.x (version bumped in r14627, fix for gst-plugins-bad in r14631): downloads fail because they redirect to https and we get a certificate issue, again (as found in comment:19)
- speex codec is missing, needs adding to moduleset?
- pyobjc-core moaned:
error: SDK root '/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk' does not exist
Not sure why it has the same path 3 times as the command it runs to get that path only returns it once, but running like this allows me to move on:
export SDKROOT=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk python ./setup.py build python ./setup.py install
- IIRC, some versions of glib / gi were held back for compatibility reasons (with what?) - can we move forward again?
Note: there is a weird setup issue on my 10.10.x system which forces me to switch to command line tools to ensure the build finds gcc / clang but switch back to "developer" mode to launch jhbuild:
sudo xcode-select -switch /Applications/Xcode.app/Contents/Developer
sudo xcode-select -switch /Library/Developer/CommandLineTools
comment:33 Changed 5 years ago by
Updates:
- r14648 removes libdir from gtkglext
- r14647 fixes mpfr download link, r14649 uses a reliable link (one that will not expire when the next release comes out), r14650 (undoes an unwanted change)
- r14646 opus update to 1.1.3
- r14645 fixes openssl install (no longer installed every time or needing chmod)
- r14651 + r14652 + r14653: use "1.0" suffix for gstreamer modules, enable gobject-introspection
comment:34 Changed 5 years ago by
- r14656 updates to libpng 1.6.27 following fix a potential "NULL dereference" bug
- r14660 adds speex
Some of these updates have been applied to the v1.0.x branch: r14657 (libpng), r14658 (mpfr), r14659 (opus)
TODO:
- pygobject3 dependency? (build gtk3, surely that can be fixed?)
- gi / glib updates?
Will ignore: pyobjc and other build env weirdness for now.
comment:35 Changed 5 years ago by
Everything builds for me on this machine
sw_vers && xcodebuild -version ProductName: Mac OS X ProductVersion: 10.12.1 BuildVersion: 16B2555 Xcode 8.2.1 Build version 8C1002
comment:36 Changed 5 years ago by
Also works on this system
sw_vers && xcodebuild -version ProductName: Mac OS X ProductVersion: 10.11.1 BuildVersion: 15B42 Xcode 7.1 Build version 7B91b
I put nasm in the modules part of the config because libjpeg-turbo will need it.
using this config
~/.jhbuildrc-custom
_gtk_osx_use_jhbuild_python = True setup_sdk(target="10.9", sdk_version="10.11", architectures=["x86_64"]) os.environ["CC"] = "/usr/bin/gcc" os.environ["DYLD_LIBRARY_PATH"] = "" build_policy = "updated-deps" modules = [ "openssl", "python", "yasm", "nasm", "libxml2", "itstool", "gtk-doc", "meta-gtk-osx-bootstrap", "meta-gtk-osx-core", "libcroco", "librsvg","meta-gtk-osx-python", "meta-gtk-osx-themes", "gtk-quartz-engine","gtk-mac-integration-python", "meta-osx-xpra-deps"] #change moduleset moduleset="http://xpra.org/svn/Xpra/trunk/osx/jhbuild/modulesets-stable/gtk-osx.modules"
comment:37 Changed 5 years ago by
Resolution: | → fixed |
---|---|
Status: | new → closed |
Works for me too (doing another full rebuild from scratch to check), minus the weirdness of having to run xcodeselect on this build system. (PITA but not a showstopper)
We can follow up in #1404.
The next release will be 64-bit only for OSX! (and targetting 10.9+ only)
comment:39 Changed 16 months ago by
this ticket has been moved to: https://github.com/Xpra-org/xpra/issues/840
FWIW: I've uploaded a beta 0.15.0 build which works quite well, see http://xpra.org/beta/osx/x86_64.
@afarr: it would be worth playing with it a bit to see if there are any other issues - stability issues (or even improvements), or if there are any differences in performance, etc..