xpra icon
Bug tracker and wiki

Opened 4 years ago

Closed 2 years ago

Last modified 7 months ago

#533 closed task (fixed)

migrate osx build to using jhbuild and modulesets

Reported by: Antoine Martin Owned by: Smo
Priority: blocker Milestone: 0.15
Component: packaging Version:
Keywords: osx focus Cc:

Description


Attachments (10)

xpra.modules (11.4 KB) - added by Smo 4 years ago.
Xpra Moduleset
nasm-makefile-destdir.patch (926 bytes) - added by Smo 4 years ago.
pygtkglext-git.patch (17.7 KB) - added by Antoine Martin 3 years ago.
patch for building git version of pygtkglext
gtkglext-1.2.0.osx-hack.tar.gz (1.3 MB) - added by Antoine Martin 3 years ago.
1.2 snapshot for osx found here: http://lists.gnu.org/archive/html/bug-gnubg/2011-04/msg00006.html
gtkglext-8c13cc4-20110529.tar.bz2 (3.9 MB) - added by Antoine Martin 3 years ago.
gtkglext git snapshot
pygtkglext-896582f-20100112.tar.bz2 (469.3 KB) - added by Antoine Martin 3 years ago.
pygtkglext git snaptshot
gtkglext-1.2.0.tar.bz2 (460.8 KB) - added by Smo 3 years ago.
gtkglext snapshot from git works for building osx with patches
osx-moduleset-updates.patch (12.6 KB) - added by Antoine Martin 3 years ago.
not fully tested moduleset updates
osx-sound-autoreleasepool.patch (2.4 KB) - added by Antoine Martin 3 years ago.
my attempts at getting rid of the warnings
quartz-style-fix.patch (531 bytes) - added by Antoine Martin 3 years ago.
blatant bug caught by newer versions of gcc / clang

Change History (104)

comment:1 Changed 4 years ago by Smo

Status: newassigned

Started work on this here is the first start. Hoping these can be added to svn somewhere.

Still needs work for dependencies however they all seem to build nicely.
Only one patch was needed for "nasm" to support make install DESTDIR= or jhbuild complains.

I'm attaching the first files to this ticket.

Changed 4 years ago by Smo

Attachment: xpra.modules added

Xpra Moduleset

Changed 4 years ago by Smo

Attachment: nasm-makefile-destdir.patch added

comment:2 Changed 4 years ago by Smo

Added these in r5829

After doing

jhbuild bootstrap
jhbuild build
jhbuild buildone gstreamer
jhbuild buildone gst-plugins-base
jhbuild buildone gst-plugins-good

You should be able to now cd to trunk/osx/jhbuild

Run jhbuild -m ./xpra.modules build meta-osx-xpra-deps

This should build and install everything else necessary to build xpra..

Currently it is missing (py)gtkglext which is the next step to be worked on.

Optional if you want subversion updated and installed you can try this as well

jhbuild -m ./xpra.modules build meta-xpra-subversion

Last edited 4 years ago by Antoine Martin (previous) (diff)

comment:3 Changed 4 years ago by Antoine Martin

Very good stuff, what a timesaver overall! I've updated the winswitch osx build page to refer to this moduleset.

For reference here's a link to the jhbuild directory


Only found a few minor things that we want to tweak / note:

  • I've bumped the version of Pillow (2.3.1) and lz4 (0.6.1) in r5882
  • we should use https for the download links and/or add checksums for the downloads
  • GTK and pygobject need patching (probably patch the module before building):
    • include this patch to fix some warnings, at least until upstream finally merges it (if ever?)
    • include this patch for fullscreen support (see #496)
  • gstreamer update: we should either get gtk-osx upstream to bump the gstreamer version to the latest 0.10, carry a patch against the moduleset, or duplicate the gstreamer modules - no biggie
  • libpng, libtiff and libjpeg: see #544
  • PyOpenGL should be bumped to 3.1 beta (see #531), using the same download or build arguments as setuptools if needed (in the setup.cfg?), I've just recorded it as I did the upgrade:
    Downloading https://pypi.python.org/packages/source/P/PyOpenGL/PyOpenGL-3.1.0b1.zip#md5=98cf868fac68e57d1712bc8b52bc8a4b
    Processing PyOpenGL-3.1.0b1.zip
    Writing /tmp/easy_install-18IUtE/PyOpenGL-3.1.0b1/setup.cfg
    Running PyOpenGL-3.1.0b1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-18IUtE/PyOpenGL-3.1.0b1/egg-dist-tmp-L7SbV6
    (...)
    Downloading https://pypi.python.org/packages/source/P/PyOpenGL-accelerate/PyOpenGL-accelerate-3.1.0b1.zip#md5=8db7b69ba0553ca5b927fe2ddbf2424e
    Processing PyOpenGL-accelerate-3.1.0b1.zip
    Writing /tmp/easy_install-33_Ewg/PyOpenGL-accelerate-3.1.0b1/setup.cfg
    Running PyOpenGL-accelerate-3.1.0b1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-33_Ewg/PyOpenGL-accelerate-3.1.0b1/egg-dist-tmp-7MQaJK
    
  • important: I had problems resolving pypi.douban.com from here (today?) and even downloading from my UK servers it was pretty slow too at times (5KB/s at some point), which made building the python bits really tedious. Not sure if there's a better mirror we could/should be using? (easy_install must be using a different download domain, because it didn't have any problems)
  • for sure, we should not be using pypi.douban.com for things like Cython (and probably others too) which have known good upstream download locations
  • easy_install will automagically grab the latest release for us, whereas the moduleset will require updating everytime one of the dependencies is updated. I'm OK with that, it will make the upgrades more obvious (we'll have a commit reference). The old macosx build page has links to most of the dependencies to make it easier to check for new releases, we should probably do the same for the python dependencies, maybe here: wiki/Building or in a new wiki page
  • the ffmpeg build is the winswitch one, it should be the xpra minimal one instead, ideally we could have both in the module somehow (unfortunately, I don't think that jhbuild does "virtual packages" ala portage)
  • nice to have: add gdb (not sure how to handle the chgrp and chmod at the end..)
  • it would be nice not to have to use the --insecure download flag at all to bootstrap the process, but I can't think of an easy way..
  • I seem to have a broken (py)gtkglext (#226) again now... can't wait to see that streamlined!
Last edited 4 years ago by Antoine Martin (previous) (diff)

comment:4 Changed 4 years ago by Smo

  • added libpng, libjpeg-turbo, libtiff in r6068
  • added meta module named meta-osx-image-libs for easier building of the 3 libs

I've had mixed results with not using --insecure or -k with curl in the end I ended up having to add the flag to the tarball.py within jhbuild to get by some issues.

comment:5 Changed 4 years ago by Smo

  • PyOpenGL and PyOpenGL version bump in r6090 built both no problem still needs testing
Last edited 4 years ago by Smo (previous) (diff)

comment:6 Changed 4 years ago by Antoine Martin

New updates:

For ffmpeg, I am now using a better, more limited configure line (also on win32):

./configure --prefix=${JHBUILD_PREFIX} \
    --cpu=i686 --enable-runtime-cpudetect --enable-pic --enable-memalign-hack \
    --enable-static --enable-shared --enable-gpl \
    --disable-everything \
    --enable-swscale --enable-libx264 --enable-decoder=h264 \
    --enable-libvpx --enable-decoder=vp8 --enable-decoder=vp9 --enable-decoder=hevc \
    --disable-protocol=tcp --disable-protocol=rtp \
    --disable-filter=aformat --disable-filter=crop --disable-filter=setpts \
    --disable-filter=anull --disable-filter=format --disable-filter=trim \
    --disable-filter=atrim --disable-filter=null \
    --disable-programs --disable-avfilter

It took a lot of fiddling and trial and error:

  • --disable-all sounds like what we want... but it doesn't work: nothing gets installed
  • we now get problems with assembler optimizations... maybe we should move to OSX Lion as a build platform and target 64-bit only in the future?
Last edited 4 years ago by Antoine Martin (previous) (diff)

comment:7 Changed 4 years ago by Smo

Added patch for gtkglext in r6096

Can use it by doing something like this

git clone git://git.gnome.org/gtkglext
cd gtkglext
curl -O http://xpra.org/svn/Xpra/trunk/osx/jhbuild/xpra-gtkglext.patch
patch -p1 <xpra-gtkglext.patch 
autoreconf -fiv
./configure --prefix=${JHBUILD_PREFIX}
make && make install

This isn't ideal but jhbuild doesn't allow me to apply a patch when pulling source from git only when using tarballs

Next is pygtkglext.

comment:8 Changed 4 years ago by Smo

Added

python-macholib
python-modulegraph
python-altgraph 

in r6098

As I was unable to build without these perhaps easy_install pulls these in for us normally?

After compiling and installing these I was able to build and package xpra.

Installing pygtkglext is still quite a hack as the installing portion usually fails and requires you to install them manually into site-packages/

Last edited 4 years ago by Smo (previous) (diff)

comment:9 Changed 4 years ago by Antoine Martin

New ones:

These are going to keep accumulating until this ticket is closed..

As for the question above: macholib, modulegraph and altgraph are normally pulled in automatically when installing py2app via easy_install.

Last edited 3 years ago by Antoine Martin (previous) (diff)

comment:10 Changed 3 years ago by Smo

  • ORC version bump in r6292
  • numpy version bump in r6293

Changed 3 years ago by Antoine Martin

Attachment: pygtkglext-git.patch added

patch for building git version of pygtkglext

comment:11 Changed 3 years ago by Antoine Martin

The patch above builds pygtkglext for me using:

PYTHONPATH=$JHBUILD_PREFIX/share/pygobject/2.0/:$PYTHONPATH python2.7 \
    ./setup.py build

We cannot use setup.py for installing (see #226 for details), so we install it with rsync (ugly):

rsync -rpvlgto build/lib.macosx*/gtk/* $JHBUILD_PREFIX/lib/python2.7/site-packages/gtk-2.0/gtk/

Note: be aware of ticket:563#comment:14 when choosing which version to use (git tree vs 1.1 snapshots)

Changed 3 years ago by Antoine Martin

comment:12 Changed 3 years ago by Antoine Martin

Newer versions should be updated in the moduleset:

Note to smo: work is going to keep piling on until you close this ticket...

Last edited 3 years ago by Antoine Martin (previous) (diff)

comment:13 Changed 3 years ago by Smo

updated versions in r6479

Compiling this whole moduleset to make sure its still working. I will use the patch that I made against gtkglext this time but should I be using this new hack that you attached? Have you tried this?

comment:14 Changed 3 years ago by Antoine Martin

The 1.2.0 patched version I posted is a newer version with some osx patches already applied, in particular the gtk>=2.20 compatibility patches. I haven't found any regressions, so I think we should use this rather than patching an older version ourselves.

comment:15 Changed 3 years ago by Antoine Martin

New:

comment:16 Changed 3 years ago by Antoine Martin

And more updates will continue to accumulate until this ticket is closed..

Today:

comment:17 Changed 3 years ago by Smo

PyOpenGL version update in r6691
Python 2.7.7 I also changed the url in .jhbuildrc-custom and recompiled everything with no issues.

I have some modifications that work well for this as well that should be updated in the winswitch osx dev page.

Can we host gtkglext and the patch for pygtkglext somewhere on xpra.org so that I can put those into the moduleset as well?

Was your pygtkglext patch for the tarball?

Last edited 3 years ago by Antoine Martin (previous) (diff)

comment:18 Changed 3 years ago by Antoine Martin

Can we host gtkglext and the patch for pygtkglext somewhere on xpra.org so that I can put those into the moduleset as well?


Sure, point me to them and I'll upload them.

Was your pygtkglext patch for the tarball?


Which one?

comment:19 Changed 3 years ago by Smo

They were both attachments to this ticket that you posted before however as I see now the pygtkglext patch is for git which won't work with jhbuild.

We can however put the hacked copy of gtkglext in the moduleset and that removes one more thing.

comment:21 Changed 3 years ago by Smo

Added gtkglext-1.2.0.osx-hack.tar.gz in r6720

had to configure with --enable-gtk-doc or make install failed with missing html doc file

Last edited 3 years ago by Smo (previous) (diff)

comment:22 Changed 3 years ago by Antoine Martin

I've had to pull the 0.12.4 OSX build because pygtkglext is now completely broken (not that I could easily have noticed since I don't have GL in my macosx vms...), rebuilding it against the latest gtk+ update does not seem sufficient to make it run, see #593.
You can easily get a crash just by trying to import gtkgl:

python -c "from gtk.gtkgl import *"

The error seems to be the one mentioned here Fatal Python error: can't initialize module gtk.gtkgl but I can't see anything wrong, this same source code used to build and run fine...

Last edited 3 years ago by Antoine Martin (previous) (diff)

comment:23 Changed 3 years ago by Antoine Martin

I've just reviewed the GTK+ changes since 2.24.16, and we want quite a few of those (many are also relevant to win32 rebuilds discussed in ticket:263#comment:10):

comment:24 Changed 3 years ago by Smo

r6897 updates many libraries

I had issues trying to rebuild after this as gst-plugins-ugly links against x264

jhbuild -m ./xpra.modules uninstall gst-plugins-ugly-xpra 
jhbuild -m ./xpra.modules -f buildone gst-plugins-ugly-xpra 

Should be enough to get it building properly again.

comment:25 Changed 3 years ago by Antoine Martin

Priority: minorcritical

It's taken a while but I think we will soon be able to close this ticket!

Please update as discussed:

  • Python 2.7.8
  • PyOpenGL 3.1 and PyOpenGL_accelerate 3.1: had to do the latter by hand as easy_update failed to build it for some reason (not investigated)
  • Pillow 2.5.1 (and check webp is enabled)
  • ffmpeg 2.3
  • x264 stable latest
  • py2app 0.8.1
  • gdb 7.8 which as Betters Python Scripting
  • gtk+ 2.24.24
  • gdk-pixbuf 2.30.8
  • pango 1.36.5
  • glib 2.40
  • gmp 6.0.0 - important: please specify a build host type to prevent building cpu optimizations which may not be available at runtime
  • setuptools 5.4.1
  • libpng 1.6.12
  • pyobjc 3.0.1 (worth a try - first update in almost 2 years, should be a drop in replacement, more lightweight too)

Finally, you can find three patches which allow us to build (py)gtkglext from git which I have added to svn in r6940 + r6941:

  • gtkglext: xpra-gtkglext.patch and gtkglext-osx-quartztagfix.diff
    git clone git://git.gnome.org/gtkglext
    cd gtkglext
    curl -O "http://xpra.org/trac/browser/xpra/trunk/osx/jhbuild/xpra-gtkglext.patch?rev=6940"
    patch -p1 < xpra-gtkglext.patch
    autoreconf -fiv
    curl -O "http://xpra.org/trac/browser/xpra/trunk/osx/jhbuild/gtkglext-osx-quartztagfix.diff?rev=6940"
    patch -p1 < gtkglext-osx-quartztagfix.patch
    ./configure --prefix=${JHBUILD_PREFIX} --with-gdktarget=quartz
    make && make install
    
  • pygtkglext: pygtkglext-osx-v4.patch
    git clone git://git.gnome.org/pygtkglext
    cd pygtkglext
    curl -O "http://xpra.org/trac/browser/xpra/trunk/osx/jhbuild/pygtkglext-osx-v4.patch?rev=6940"
    patch -p1 < pygtkglext-osx-v4.patch
    ./autogen.sh --prefix ${JHBUILD_PREFIX}
    make && make install
    

If needed for the moduleset, we can host the patches in a more canonical location, or even make a patched source archive with the patches already applied.

Note: the pygtkglext patch is not so different from the previous version (which may still work?), and I've seen weird things happen as I rebuilt the libraries. I believe jhbuild leaves some crud behind and we must forcibly remove the build directories before trying to rebuild a library. Something to keep in mind.. since I don't see a way of forcing jhbuild to completely remove a build directory.

It may be worthwhile adding a switch somewhere so we can get debug builds for all those libraries. Not sure exactly how to add -g to the CFLAGS however.


Once that's all done and tested, please re-assign to me so I can close this.

Last edited 7 months ago by Antoine Martin (previous) (diff)

comment:26 Changed 3 years ago by Antoine Martin

Dammit, looks like we're not the first ones to hit this "invalid drawable" bug (recorded in ticket:563#no4 - which is closed, so moving here):

But for gtkgl on osx, the best I found is this (which has a dead link in it - it should point here instead: GtkGLExt OS X Quartz hack patch):

I think I see it just by running gl_check which should help in testing a fix.

Running the examples in gtkglext works without major problems, but the examples in pygtkglext show the invalid drawable problem.
For the record, we can also build pygtkglext-1.1.0 with the patch above, ignoring the changes to autogen.sh and then using:

CFLAGS="-I/usr/X11/include" ./configure --prefix=${JHBUILD_PREFIX}

But the bug remains..

We can also build gtkglext with debug enabled:

    --enable-debug=[no/minimum/yes]
Last edited 3 years ago by Antoine Martin (previous) (diff)

comment:27 Changed 3 years ago by Antoine Martin

Priority: criticalblocker

Edited comment:25 with instructions I have tested repeatedly from scratch, this seem to fix the "invalid drawable" bug from ticket:563#no4, now tracked in ticket:593#comment:4 (at least it works on my 10.5 and 10.6 test machines so far)

Let's try to close this ticket whilst things work OK! (so raising to blocker)

If jhbuild doesn't allow us to apply the quartztagfix patch after the autoreconf step, we can just generate our own archive from git and stick it in svn.

Last edited 3 years ago by Antoine Martin (previous) (diff)

Changed 3 years ago by Antoine Martin

gtkglext git snapshot

Changed 3 years ago by Antoine Martin

pygtkglext git snaptshot

comment:28 Changed 3 years ago by Smo

I've updated most of the stuff from comment:25

I've had to make a copy of the modulesets-stable to our source control so that I can modify them to update a few modules. We can contribute this upstream when we are satisfied everything is working.

r7184 brings in all the necessary changes to our moduleset
r7186 updates many libraries to current versions

I still have to build gtkglext and pygtkglext by hand but the instructions work nicely along with the patches. We may have to make source tarballs for these to apply the patches.

I've tried to find a nicer way to fix autoconf for gtkglext so we don't have to do the second patch but didn't figure it out yet.

I've tested building all of these on an mac mini running osx 10.6.8 64bit hence all the build switches I commited for 32bit builds

I am now in the process of building this all again on a mac mini running 10.5.8 32bit but I assume it will work even easier than the other machine.

So again we are very close to closing this ticket but still some work needs to be done.

This is what my .jhbuildrc-custom looks like at the moment on the 32bit machine

module_autogenargs['cairo'] = '--enable-ft=no'
setup_sdk(target="10.5", sdk_version="10.5", architectures=["i386"])
os.environ["CC"] = "/usr/bin/gcc-4.2"
os.environ["DYLD_LIBRARY_PATH"] = ""
build_policy = "updated-deps"
modules = [ "python",
"meta-gtk-osx-bootstrap", "meta-gtk-osx-core",
"libcroco", "librsvg",
"meta-gtk-osx-python", "meta-gtk-osx-themes",
"gtk-quartz-engine",
"gtk-mac-integration-python" ]

#skip iconv and add some switches to turn off stuff that was breaking
skip.append("libiconv")
append_autogenargs('gstreamer', '--enable-introspection=no')
append_autogenargs('gst-plugins-base', '--enable-introspection=no')
append_autogenargs('gst-plugins-good', '--enable-introspection=no --disable-deinterlace')

#change moduleset
moduleset="http://xpra.org/svn/Xpra/trunk/osx/jhbuild/modulesets-stable/gtk-osx.modules"

comment:29 Changed 3 years ago by Smo

More problems it doesn't use bootstrap.modules from that location it only uses it from local ~/Source/jhbuild/modulesets/bootstrap.modules

Which is troublesome as we need to update gtk-doc so that we can build new glib if memory serves right I will also need another package to build new gtk-doc

I checked bootstrap.modules against the one locally and the only change was the change I made so its safe to use this one to edit for now.

Unfortunatly I need to just copy over the one that exists with ours to do this not ideal

cd ~/Source/jhbuild/modulesets && curl -O http://xpra.org/svn/Xpra/trunk/osx/jhbuild/modulesets-stable/bootstrap.modules

Then run jhbuild bootstrap

Last edited 3 years ago by Smo (previous) (diff)

comment:30 Changed 3 years ago by Smo

Ran into another problem

gtk-doc requires itstool but itstool requires python

configure: error: itstool not found
*** Error during phase configure of gtk-doc: ########## Error running ./configure --prefix /Users/blank/gtk/inst --libdir '/Users/blank/gtk/inst/lib' --disable-scrollkeeper --with-xml-catalog=$JHBUILD_PREFIX/etc/xml/catalog   *** [16/19]

jhbuild buildone python
jhbuild buildone libxml2
jhbuild buildone itstool
jhbuild bootstrap

lets me get past this point but I guess python, libxml2 and itstool need to move to bootstrap? Or build gtk-doc without itstool?

After this I get

*** success *** [19/19]

That is enough to move on to the next steps

comment:31 Changed 3 years ago by Antoine Martin

Please see also:

  • missing AppKit from ticket:627#comment:4
  • r7317 (ffmpeg 2.3.3 version update)
  • r7324 (yasm 1.3.0 version update)
  • r7327 (webp 0.4.1 update - same update needed for win32 builds)
  • please ensure that all modules have md5sums, since not all urls are https
Last edited 3 years ago by Antoine Martin (previous) (diff)

comment:32 Changed 3 years ago by Smo

I'll add the md5sums to the missing packages. Looks like we still need a patch for GTK for dual monitor support I found that out today.

As for (py)gtkglext I really think we should probably just make snapshots of the repos and host our own .tar.bz2 for those 2 so we can use them with jhbuild.

What would be a good place to stick these?

comment:33 Changed 3 years ago by Antoine Martin

comment:34 Changed 3 years ago by Smo

I found a way to have jhbuild pull our patch and run it after autoreconf however I can't seem to build from the gtkglext git snapshot in comment:33

I always end up with a build failure like this

i686-apple-darwin9-gcc-4.2.1: .libs/gdkglversion.o: No such file or directory
i686-apple-darwin9-gcc-4.2.1: .libs/gdkglinit.o: No such file or directory
i686-apple-darwin9-gcc-4.2.1: .libs/gdkglquery.o: No such file or directory
i686-apple-darwin9-gcc-4.2.1: .libs/gdkglconfig.o: No such file or directory
i686-apple-darwin9-gcc-4.2.1: .libs/gdkglcontext.o: No such file or directory
i686-apple-darwin9-gcc-4.2.1: .libs/gdkgldrawable.o: No such file or directory
i686-apple-darwin9-gcc-4.2.1: .libs/gdkglpixmap.o: No such file or directory
i686-apple-darwin9-gcc-4.2.1: .libs/gdkglwindow.o: No such file or directory
i686-apple-darwin9-gcc-4.2.1: .libs/gdkglglext.o: No such file or directory
make[4]: *** [libgdkglext-quartz-1.0.la] Error 1
make[3]: *** [all-recursive] Error 1
make[2]: *** [all] Error 2
make[1]: *** [all-recursive] Error 1

I followed the instructions from comment:25 just to be sure and it works no problem so i'm thinking maybe that snapshot is bad?

comment:35 Changed 3 years ago by Antoine Martin

I doubt the snapshot is bad, but it's possible.

Those errors are about object files, which are meant to be generated during the build, so a bad snapshot shouldn't be causing this unless it messed with the build scripts.

comment:37 Changed 3 years ago by Antoine Martin

https://pypi.python.org/pypi/Pillow/2.5.3 low priority: only CVE entries for things we don't use.

Changed 3 years ago by Smo

Attachment: gtkglext-1.2.0.tar.bz2 added

gtkglext snapshot from git works for building osx with patches

comment:38 Changed 3 years ago by Smo

r7472 adds gtkglext and pygtkglext

The source tarball for gtkglext is one I attached to this ticket so this should probably change.

I made this one by doing this

git clone git://git.gnome.org/gtkglext && cd gtkglext
git archive --prefix=gtkglext-1.2.0/ -o ../gtkglext-1.2.0.tar HEAD
cd .. && bzip2 gtkglext-1.2.0.tar

Seems to build now with that snapshot i'm not sure what the difference is. Feel free to make a new one and just provide me with the url and I will update it.

I will now delete my ~/gtk directory and start this all from scratch hopefully now it will just build everything we need.

comment:39 Changed 3 years ago by Antoine Martin

I've uploaded your snapshot in the source dir: http://xpra.org/src/gtkglext-1.2.0.tar.bz2

comment:40 Changed 3 years ago by Antoine Martin

Note: Cython should be updated to 0.21.x (I have reported fairly big bug, they may well do a point release before too long).

I have created a separate ticket for OSX GTK3: #680.

comment:41 Changed 3 years ago by Smo

Many new changes seems to work quite well now. Still needs some polish but pretty happy with it now.

You'll need the .jhbuildrc-custom from r7651 which includes our modulesets from by url.

r7642 adds gtk-doc from bootstrap.modules to our moduleset as we need itstool and libxml2 which requires python to build it now.

r7648 adds a patch for liboil to fix autreconf from breaking found it here https://github.com/GNOME/frogr/tree/master/osx/jhbuild/patches

r7652 Cython 0.21

You need to overwrite ~/Source/jhbuild/modulesets/bootstrap.modules with ours I haven't found any way to use ours instead.

If everything is in place I'm able to do

jhbuild bootstrap
jhbuild build
jhbuild build meta-subversion-xpra

bootstrap has 17 steps and build has 88

I had to modify ~/Source/jhbuild/jhbuild/versioncontrol/tarball.py to add -k to curl as a lot a few https tarballs were failing since the certificates on this old mac are outdated.

comment:42 Changed 3 years ago by Smo

Owner: changed from Smo to Antoine Martin
Status: assignednew

comment:43 Changed 3 years ago by Antoine Martin

My best guess at how to run this build from start to finish (took a few tries to get every step) - to be used as documentation template? There should be at least a README file telling you what to do:

  • setup jhbuild env (I really wished we didn't have to start by running a shell script we download with --insecure!):
    curl --insecure -O https://git.gnome.org/browse/gtk-osx/plain/gtk-osx-build-setup.sh
    sh gtk-osx-build-setup.sh
    export PATH=$PATH:~/.local/bin/; jhbuild shell
    
  • checkout xpra (this assumes svn is installed somehow, you may just untar it instead, or rsync it over, whatever):
    svn co http://xpra.org/svn/Xpra
    
  • copy our modulesets over (not obvious - some not needed):
    cp Xpra/trunk/osx/jhbuild/modulesets-stable/*.modules ~/Source/jhbuild/modulesets/
    
  • copy the customized .jhbuildrc-custom (not obvious - I would prefer appending to the existing one rather than duplicating some of it):
    cp Xpra/trunk/osx/jhbuild/jhbuildrc-custom-xpra ~/.jhbuildrc-custom
    
  • run the build:
    jhbuild bootstrap
    jhbuild build
    jhbuild build meta-subversion-xpra
    

I got some errors, but I think they were due to the fact that I had originally missed some of the undocumented steps.


Build started at: Wed 17 Sep 09:42:47 ICT 2014

Build finished at: Wed Sep 17 16:46:12 ICT 2014

A fair amount of this time was spent downloading stuff again on my slow DSL line (as I had wiped the whole home directory to start clean), but also re-doing things, and fixing cert errors...


Comments:

  • the version number in the moduleset for gst-plugins-base was wrong, fixed in r7664
  • gtk-mac-bundler is missing altogether, means we can't run the xpra build (note: there is no "real" install step for it, you must keep the source directory around...)
  • I got some https certificate errors (at least for many of the python packages), can we workaround that somehow? Did I miss something? (none of those are part of the bootstrap, so we can build a newer version of wget or curl if needed - without having to build a new openssl somehow? maybe we can just download the CA from our own server then use the --cacert option?)
  • I don't think we need all these modulesets? (javascript, glade, etc..) Some of those look identical to upstream, and we don't even build anything from them, so they should be removed from svn.
  • Note: some files are downloaded from https, others not. I know I said we should have checksums for all of them to be on the safe side, but until we download the script using https... not much point in doing that!
  • Please add python-lzo and python-yaml
  • as of r7671, you also need unbundled rencode: see ticket:683#comment:1
  • The full OSX build instructions (start to finish) should go here eventually: wiki/Building (and maybe move them off to a sub page if they're too long)
  • Some version updates should be done now before closing this ticket (mostly trivial minor updates):
    • xz 5.0.6
    • setuptools 5.7
    • netifaces 0.10.4
    • Twisted 14.0.0
    • numpy 1.9.0 (just out!)
    • py2app 0.9, macholib 1.7, modulegraph 0.12
    • gettext 0.19.2, pixman 0.32.6, pango 1.36.7, atk 2.12
    • libxslt 1.1.28
    • freetype 2.5.3, fontconfig 2.11.1
    • hicolor-icon-theme 0.13
    • pkg-config 0.28
  • These, I am not too sure about (I seem to remember some errors in at least one module - also, because of the duplication, it's not really clear to me what gets built when):
    • gstreamer 0.10.36
    • gst-plugins-base 0.10.36
    • gst-plugins-good 0.10.31

Others can wait for #680: I've added the info there.

Last edited 2 years ago by Antoine Martin (previous) (diff)

comment:44 Changed 3 years ago by Antoine Martin

Owner: changed from Antoine Martin to Smo

It looks bad because of the amount of pending issues, but really it isn't. One more round and we should be able to close this... and move on to #680 and #640.

I've made a beta build (with some version updates on top, as per above) which works fine: http://xpra.org/beta/osx/x86/

Changed 3 years ago by Antoine Martin

Attachment: osx-moduleset-updates.patch added

not fully tested moduleset updates

comment:45 Changed 3 years ago by Smo

Applied your patch uploaded the modulesets to a test webserver and tested them. Just a couple issues.

  • libffi newer versions don't work

https://github.com/atgreen/libffi/issues/128

  • pkg-config had to add --with-internal-glib to configure or you get
configure: error: Either a previously installed pkg-config or "glib-2.0 >= 2.16" could not be found. Please set GLIB_CFLAGS and GLIB_LIBS to the correct values or pass --with-internal-glib to configure to use the bundled copy.

things that were updated in r7693

  • xz 5.0.6
  • apr 1.5.1
  • apr-util 1.5.3 (I think these 2 are duplicated in subversion meta module. Might need to clean this up)
  • gettext 0.19.2 (path no longer needed)
  • pkg-config 0.28 (built with internal glib)
  • libpng 1.6.13 (1.6.12 source doesn't exist anymore?)
  • libxml2 2.9.1
  • libxslt 1.1.28
  • freetype 2.5.3
  • fontconfig 2.11.1
  • hicolor-icon-theme 0.13
  • gstreamer 0.10.36
  • gst-plugins-base 0.10.36
  • gst-plugins-good 0.10.31
  • Python 3.4.1 (we don't use this yet)
  • pygobject 3.12.2
  • gnome-icon-theme 3.12.0
  • gnome-icon-theme-symbolic 3.12.0
  • gnome-themes-standard 3.12.0
  • gobject-introspection 1.41.4
  • pango 1.36.7
  • atk 2.12.0
  • gtk 3.12.2
  • pixman 0.32.6

I tested this like you did by updating a few versions then I recompiled the whole lot.

Looks good the client will require some testing to make sure nothing breaks as this is quite a big update.

Next I will add the winswitch stuff that is left i'm just not sure if I want to make a winswitch.modules and include it or just add it to the current moduleset.

I will write out some simplified instructions and attact them to this ticket.
Your instructions from comment:43 are roughly correct except you don't have to copy all the .modules files they are included via urls

comment:46 Changed 3 years ago by Antoine Martin

Notes:

  • if libpng don't have stable and reliable URL download locations, maybe we should mirror it. We want these instructions to work everytime, today, next month and next year
  • don't forget the rencode unbundling
Last edited 3 years ago by Antoine Martin (previous) (diff)

comment:47 Changed 3 years ago by Smo

I'm having issues trying to add the winswitch stuff can't link with this nxcomp library

ld: in /Users/user/gtk/inst/lib/libXcomp.so, can't link with bundle (MH_BUNDLE) only dylibs (MH_DYLIB)

I've been following the steps http://winswitch.org/dev/macosx.html for the winswitch stuff.

I will try a few more things.

comment:48 Changed 3 years ago by Smo

r8054

  • Add patch for gtk-mac-bundler the current one linked in the manual osx documentation doesn't apply cleanly.
  • xpra version update in winswitch.modules
  • partial working module for winswitch

I needed to use the autotools module to do some hacks to download a 2nd tarball and extract it plus make a link.

This might not be the smartest way to do this perhaps I can just have it download a simply makefile that does all this stuff instead of hacking it into jhbuild that way satisfying both problems.

I will add in gtk-mac-bundler soon as well as it doesn't seem to matter where it is actually extracted to it simply makes a helper script in ~/.local/bin pointing to it. With a working patch this should install cleanly.

comment:49 Changed 3 years ago by Antoine Martin

More updates needed (I've already done most of them outside the moduleset):

  • serf?
  • apr-util?
  • gdb
  • ffmpeg 2.4.4

more?

comment:50 Changed 3 years ago by Antoine Martin

comment:51 Changed 3 years ago by Smo

gdb is still problematic to add to the moduleset since it requires one extra step to make it work after installing

comment:52 Changed 3 years ago by Smo

New serf uses scons build system which is problematic to adding to moduleset
as well.

I believe I came across this before so unless we absolutely need the latest version I don't think we should update.

comment:53 Changed 3 years ago by Antoine Martin

the changeset was r8250.

FWIW: I'm still on ffmpeg 2.4 (2.4.4 as of this writing), not switching to 2.5 yet.

comment:54 Changed 3 years ago by Antoine Martin

comment:55 Changed 3 years ago by Smo

r8653 brings in many changes from upstream git repo

https://git.gnome.org/browse/gtk-osx/tree/modulesets-stable

Few were left behind if we were using newer versions like for gmp and python 2.7.9

Still needs testing which I'm rebuilding right now will update with instructions if successful

comment:56 Changed 3 years ago by Smo

r8668
Newer versions of gtk-mac-integration will not build on osx < 10.6

r8671
Host patch for gtk-mac-integration as the old one seems to be missing

r8673
removes 2 modules we had put in that are now bootstrap modules

r8676
adds python-nose as we didn't have this and is needed now by lz4

r8678
we don't use scons for building serf

r8683
add gtk-mac-bundler to dependencies for meta-osx-xpra-deps this install works now

r8684
added proper old catched patch for gtk-mac-integration

comment:57 Changed 3 years ago by Smo

Will update soon with new jhbuildrc-custom and a few tweaks.

Haven't found a way to override bootstrap.modules yet so I typically copy over it with ours.

Also I typically have to also make a change in jhbuild/versioncontrol/tarball.py to make curl use the '-k' option as certificate checking is a bit old and often fails when downloading files.

Another bug I discovered is that you want to build everything outside of jhbuild shell so that you use the system python for calling jhbuild. If you do not the ssl module in python 2.7.9 now checks certificates by default and this also causes problems as they use urllib2 to download and cache patches.

comment:58 Changed 3 years ago by Smo

Here are some rough instructions should be good for updating the winswitch osx dev page if this all works.

curl -k -O https://git.gnome.org/browse/gtk-osx/plain/gtk-osx-build-setup.sh

sh gtk-osx-build-setup.sh

export PATH=$PATH:~/.local/bin/

curl -k -o ~/Source/jhbuild/modulesets/bootstrap.modules http://xpra.org/svn/Xpra/trunk/osx/jhbuild/modulesets-stable/bootstrap.modules

sed -i -e 's/^setup_sdk/#setup_sdk/g' .jhbuildrc-custom

cat << EOF >> .jhbuildrc-custom
setup_sdk(target="10.5", sdk_version="10.5", architectures=["i386"])
os.environ["CC"] = "/usr/bin/gcc-4.2"
os.environ["DYLD_LIBRARY_PATH"] = ""
build_policy = "updated-deps"
modules = [ "python", "libxml2", "itstool", "gtk-doc",
"meta-gtk-osx-bootstrap", "meta-gtk-osx-core",
"libcroco", "librsvg","meta-gtk-osx-python", "meta-gtk-osx-themes",
"gtk-quartz-engine","gtk-mac-integration-python",
"gstreamer", "gst-plugins-base", "gst-plugins-good",
"meta-osx-xpra-deps"]

#skip iconv and add some switches to turn off stuff that was breaking
skip.append("libiconv")
append_autogenargs('gstreamer', '--enable-introspection=no')
append_autogenargs('gst-plugins-base', '--enable-introspection=no')
append_autogenargs('gst-plugins-good', '--enable-introspection=no --disable-deinterlace')

#change moduleset
moduleset="http://xpra.org/svn/Xpra/trunk/osx/jhbuild/modulesets-stable/gtk-osx.modules"
EOF

jhbuild bootstrap
jhbuild build

jhbuild build meta-subversion-xpra

jhbuild shell
Last edited 3 years ago by Antoine Martin (previous) (diff)

comment:59 Changed 3 years ago by Smo

Owner: changed from Smo to Antoine Martin

Another day another fun time with osx.

Found that I was missing libz.1.2.8.dylib not because of de-duplication but because new jhbuild modulesets build it instead of using the system one.

After wasting my type trying to debug and fix the de-duplication part of the make-app.sh script I found that I just needed to add this to Xpra.bundle so that it was actually included.

Now I've run into further problems with libjpeg-turbo and PIL which I will have to solve now.

2015-03-19 10:58:21,680 draw error
Traceback (most recent call last):
  File "/Users/test/gtk/inst/lib/python2.7/site-packages/xpra/client/ui_client_base.py", line 1961, in _do_draw
  File "/Users/test/gtk/inst/lib/python2.7/site-packages/xpra/client/client_window_base.py", line 423, in draw_region
  File "/Users/test/gtk/inst/lib/python2.7/site-packages/xpra/client/window_backing_base.py", line 476, in draw_region
  File "/Users/test/gtk/inst/lib/python2.7/site-packages/xpra/client/gtk2/window_backing.py", line 35, in paint_image
  File "/Users/test/gtk/inst/lib/python2.7/site-packages/xpra/client/window_backing_base.py", line 227, in paint_image
  File "PIL/Image.pyc", line 644, in tobytes
    one of :py:attr:`PIL.Image.NEAREST` (use nearest neighbour),
  File "PIL/ImageFile.pyc", line 191, in load
  File "PIL/Image.pyc", line 413, in _getdecoder
    the image.
IOError: decoder jpeg not available

Could be that PIL wasn't built with jpeg support as I can see the library is packaged but I am looking for any advice you can give me to debug this.

comment:60 Changed 3 years ago by Smo

Here is output from forcing PIL to rebuild. Looks like jpeg is compiled in so perhaps just missing something.

--------------------------------------------------------------------
PIL SETUP SUMMARY
--------------------------------------------------------------------
version      Pillow 2.5.3
platform     darwin 2.7.9 (default, Feb 19 2015, 11:53:52)
             [GCC 4.2.1 (Apple Inc. build 5577)]
--------------------------------------------------------------------
--- TKINTER support available
--- JPEG support available
*** OPENJPEG (JPEG2000) support not available
--- ZLIB (PNG/ZIP) support available
--- LIBTIFF support available
--- FREETYPE2 support available
*** LITTLECMS2 support not available
--- WEBP support available
*** WEBPMUX support not available
--------------------------------------------------------------------
Last edited 3 years ago by Smo (previous) (diff)

comment:61 Changed 3 years ago by Antoine Martin

To build pycups on OSX 10.5.x, I had to add the following missing defines to cupsmodule.c before building:

#define CUPS_FORMAT_AUTO     "application/octet-stream"
#define CUPS_FORMAT_COMMAND  "application/vnd.cups-command"
#define CUPS_FORMAT_PDF      "application/pdf"
#define CUPS_FORMAT_POSTSCRIPT "application/postscript"
#define CUPS_FORMAT_RAW      "application/vnd.cups-raw"
#define CUPS_FORMAT_TEXT     "text/plain"
#define CUPS_GET_DOCUMENT   0x4027

It fails at runtime on 10.5.
(I think that the version of cups in 10.5.x is just too old. I guess we could try to build a newer version of cups and bundle that too.. but at that point, we might as well make our own OS - so we just won't support 10.5 for printing)

$ python -c "import cups"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: dlopen(/Users/MacAdmin/gtk/inst/lib/python2.7/site-packages/cups.so, 2): Symbol not found: _cupsCreateJob
  Referenced from: /Users/MacAdmin/gtk/inst/lib/python2.7/site-packages/cups.so
  Expected in: dynamic lookup

It does run with 10.6 onwards though, see ticket:598#comment:34.

PS: in 0.15, you also need rencode as a separate module.

Last edited 3 years ago by Antoine Martin (previous) (diff)

comment:62 Changed 3 years ago by Smo

r8828 added configure switch for libjpeg-turbo to turn on v8 compatibility which now works with python-pillow

PyObjcTools? seems to be missing __init__.py in ~/gtk/inst/lib/python2.7/site-packages/PyObjCTools which causes py2app to not bundle it.

You can see the file here though which should be there
https://bitbucket.org/ronaldoussoren/pyobjc/src/f4334b2f9f2d0b4a940eb9d0a6f1be98d5db9d02/pyobjc-framework-Cocoa/Lib/PyObjCTools/__init__.py?at=default

Not sure why this is missing will have to look into this some more. Once I added this file everything is packaged again.

My previous builds were always missing this stuff and I believe it may have been causing issues.

comment:63 Changed 3 years ago by Smo

r8835 adds patch for pygobject to get rid of annoying messages as noted here

http://winswitch.org/dev/macosx-manual.html

That is the last of the cleanup and everything seems to be running smoothly now

comment:64 Changed 3 years ago by Smo

Can't add patches for disutils stuff in jhbuild so I may have to add pycups some other way if I want to patch it to include these constants.

Builds fine when I add them same as you.

comment:65 Changed 3 years ago by Antoine Martin

Owner: changed from Antoine Martin to Smo

I have gone through the steps in comment:58.

I've hit the following https certificate issues (this is a 10.5.x VM), worked around by adding --insecure to the curl download command from a shell:

  • during bootstrap: intltool.
  • during build: setuptools, netifaces, pam, pycrypto, pyasn1, Pillow, py2app, Cython, numpy, Twisted, nose, macholib, modulegraph, altgraph, PyOpenGL (+accelerate), pyobjc (all 3 packages).

(which means that it took me a lot longer than 3 hours to build because of the manual steps)

Also, the versions of many packages is way too old: netifaces, Pillow, Cython, numpy, etc.. (and probably many others too), see comment:43 (which must be out of date by now, but more up to date than the moduleset). Please go through them all as I did before and ensure we use the most up to date version, unless there is a good reason (Twisted had problems with winswitch - but we'll need the latest version to get the ssh fix anyway, so the fix will have to be in winswitch for that one)

I've seen the missing pyobjc warning, PIL is having problems too (module has no attribute 'Image'), and obviously pycups is also missing.
If this helps, I've uploaded this beta osx x86 build as Xpra-0.15.0-r8825-MODULESET.dmg - beware: because of the issues above, it can't be used for anything useful.

Last edited 3 years ago by Antoine Martin (previous) (diff)

comment:66 Changed 3 years ago by Smo

Owner: changed from Smo to Antoine Martin

I can go through and add update all packages to their latest version that is not a problem.

The libjpeg problem I think I see the issue is that libjpeg.*.dylib needs to be added to Xpra.bundle and possible libz.*.dylib

Svn is marking this file as a binary file for some reason so I missed it when I was doing a diff.

Can you try adding those 2 and making another build that will probably fix up the PIL issue.

As I said in comment:62 pyobjctools seems to be missing init.py in site-packages after it is installed.

comment:67 Changed 3 years ago by Smo

Owner: changed from Antoine Martin to Smo

r8873:r8889 updates versions in this moduleset

I will have to rebuild and test it all again to make sure everything is fine.

comment:68 Changed 3 years ago by Antoine Martin

For the record (and for my own reference), I have updated my OSX build machine with the latest:

  • Cython 0.22
  • nasm 2.11.08
  • libwebp 0.4.3 (note to self, using these instructions for win32: https://developers.google.com/speed/webp/docs/compiling - compiling with both static then dynamic arguments to get the libwebp.lib and libwebp.dll)
  • libvorbis 1.3.5
  • ffmpeg 2.4.8 (I will stick with the 2.4 branch which is still maintained - will probably move to the 2.5 or 2.6 branch with the 0.15.x release)
  • py2app 0.9 + deps (macholib, modulegraph, altgraph)
  • pyobjc + deps 3.0.4

Not updating pyopengl - will wait for the 3.1.1 proper.

The win32 build VM needed most of the same updates (added difficulty: both the python 2.7 and python 3.4 environments), done them too.

After that I uploaded a new beta build made from those environments, this may help in comparing with what you're getting (or with tickets such as #598).


FWIW: the OSX "moduleset" based builds are not usable at this point.
So I've uploaded both a "MODULESET" build and a "normal" one in the osx beta area.
PS: Some NS Autorelease Pool errors (in both builds now), somewhat similar to __NSAutoreleaseNoPool error … - just leaking. I haven't seen those before, or at least not recently, so this may be related to the update of pyobjc. But I will try the fix suggested in that link.

Last edited 3 years ago by Antoine Martin (previous) (diff)

comment:69 Changed 3 years ago by Smo

r8903 rolled back pyobjc stuff to older versions however still seeing these kinds of errors

_NSAutoreleaseNoPool(): Object 0x1803a660 of class NSCFDictionary autoreleased with no pool in place - just leaking

I figured it would be an updated version causing this problem. the link in comment:68 said something about matplotlib but we don't use this. Could the problem be with numpy?

comment:70 Changed 3 years ago by Smo

Owner: changed from Smo to Antoine Martin

r8904 rolls back numpy to 1.8.1 and still I get the same error messages as comment:69

Not sure where to look from here maybe we do need to fix this properly?

comment:71 Changed 3 years ago by Smo

Tried my client with --speaker=off

No more NSAutoreleaseNoPool messages. Must be something new in sound that causes this?

Changed 3 years ago by Antoine Martin

my attempts at getting rid of the warnings

comment:72 Changed 3 years ago by Antoine Martin

The patch above doesn't help...
As per Working with threads, it sounds like this is a threading issue but we already process all the packets in the main thread... not sure what to do (see ticket:669#comment:24).

comment:73 Changed 3 years ago by Antoine Martin

New update: libvpx 1.4.0, see #832. (download location has changed)

comment:74 Changed 3 years ago by Antoine Martin

I don't see libvorbis anywhere in the moduleset, but this one has been bumped to 1.3.5 last month.

Last edited 3 years ago by Antoine Martin (previous) (diff)

comment:75 Changed 3 years ago by Antoine Martin

Pillow 2.8.1 has been released...

comment:76 Changed 3 years ago by Antoine Martin

Owner: changed from Antoine Martin to Antoine Martin
Status: newassigned

@smo: FWIW, I am giving this a go on a clean OSX 10.9 (aka Mavericks) + Xcode 6.2 and the command line tools.
Using these changes to .jhbuildrc-custom:

setup_sdk(target="10.9", sdk_version="10.9", architectures=["x86_64"])
os.environ["CC"] = "/usr/bin/gcc"

It's going to take a while... but so far so good! (the bootstrap stage finished without problems - yay!)

Changed 3 years ago by Antoine Martin

Attachment: quartz-style-fix.patch added

blatant bug caught by newer versions of gcc / clang

comment:77 Changed 3 years ago by Antoine Martin

Hurdles (not mentioning the dozens of packages which spit out thousands of warnings because this is using clang instead of gcc):

  • *** Building gtk-quartz-engine *** [44/97]:
    quartz-style.c:1573:45: error: variable 'height' is uninitialized when used here [-Werror,-Wuninitialized]
          item_rect = CGRectMake (x1, y, x2-x1, height);
                                                ^~~~~~
    

This is fixed by applying attachment/ticket/533/quartz-style-fix.patch.
Which we should apply in any case, using an uninitialized variable like this is a bug. It is probably fixed upstream already, so maybe we just need a newer version of the library.

  • *** Configuring yasm *** [49/97]:
    configure: error: C preprocessor "/lib/cpp" fails sanity check
    

That's because the configure line looks like this:

./configure --prefix ${JHBUILD_PREFIX} --libdir ${JHBUILD_PREFIX}/lib --build=i386-darwin CC="gcc -arch i386" CXX="g++ -arch i386"

Removing the CC and CXX definitions and using --build=x86_64-darwin allows it to build (completely removing --build= might also work).
@smo: do we really need those overrides for the i386 case? Why? If so, how can we make this conditional?

  • *** Building libtheora *** [53/97]

Similar problem, solved by just adding --build=x86_64-darwin to the configure line.
Then you hit this problem: -fforce-addr causes clang error with Xcode 5.1, best solution is found here: macports: libtheora build fails with Xcode 5.1 due to -fforce-addr flag, solved with this patch: https://trac.macports.org/browser/trunk/dports/multimedia/libtheora/files/patch-configure.diff?rev=118032.

  • *** Configuring libvpx *** [63/97]

Same problem.
Since I am building on 10.9, I used --target=x86_64-darwin13-gcc.
That's libvpx 1.3, we should try 1.4 to see if that builds ok.

  • *** Configuring x264 *** [64/97]

Same problem.

  • *** Configuring ffmpeg *** [66/97]

Same problem. --cpu=i686 should be --cpu=x86_64

  • *** Building sdl *** [67/97]:
    ./include/SDL_syswm.h:58:10: fatal error: 'X11/Xlib.h' file not found
    

Lots of answers: Can't install PIL after Mac OS X 10.9, but what I did was simply:

ln -sf /System/Library/Frameworks/Tk.framework/Versions/8.5/Headers/X11 ${JHBUILD_PREFIX}/include/

Which is nice because it doesn't require root.
But the build then fails with:

./src/video/quartz/SDL_QuartzVideo.h:94:5: error: unknown type name 'CGDirectPaletteRef'

Which is fixed using this: http://hg.libsdl.org/SDL/rev/e9466ead70e5.
This looks like a cleaner fix: http://www.emaculation.com/doku.php/compiling_sheepshaver_basilisk#tidbits. (not tested).
Then it will moan about missing X11/Xmd.h, and we don't care about this X11 video stuff so just add this to configure: --disable-video-x11.

  • *** Configuring gmplib *** [71/97]

Same arch problem.
And then a new obscure one after changing arch to x86_64:

tmp-invert_limb_table.s:46:1: error: unknown directive
.hidden __gmpn_invert_limb_table
^
make[2]: *** [invert_limb_table.lo] Error 1

I tried the latest development version to see if that fixes the problem, sadly not.
The solution turns out to be quite simple: just take out the whole arch thing and let the configure script guess what to do.

  • *** Building mpfr *** [72/97]:

Note: I didn't wait to see if it was going to build, because it's wrong to build for i386...
Same thing: just removed arch completely to build it.

  • *** Building python-pyobjc-core *** [91/97]:

Obscure errors:

error: missing ',' between enumerators
error: missing ',' between enumerators
In file included from Modules/objc/block_support.m:11:
In file included from Modules/objc/pyobjc.h:14:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.10.sdk/System/Library/Frameworks/Foundation.framework/Headers/Foundation.h:52:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.10.sdk/System/Library/Frameworks/Foundation.framework/Headers/NSOperation.h:6:
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.10.sdk/usr/include/sys/qos.h:128:4: error: redefinition of
      enumerator '__AVAILABILITY_INTERNAL__MAC_10_10'
                        __QOS_CLASS_AVAILABLE_STARTING(__MAC_10_10, __IPHONE_8_0) = 0x19,
                        ^
...

I give up on this one, and the ones that depend on it (pyobjc cocoa, pyobjc quartz).
We need it so I will have to find a fix.

  • *** Building pygtkglext *** [95/97]:
    gdkglext.override:36:10: fatal error: 'GL/gl.h' file not found
    #include <GL/gl.h>
    

Solved in a similar way to the X11 header problem:

ln -sf /System/Library/Frameworks/OpenGL.framework/Versions/A/Headers ${JHBUILD_PREFIX}/include/GL

@smo: a lot of the problems are because of the configure arguments, can't we use append_autogenargs for this? Then have one custom file for i386 and one for x86_64?

Small nitpick: when installing gtk-mac-bundler, it doesn't really install anything but links back to where the source is. Which means that when cleaning gtk/source... we break it. It would be worth copying to gtk/inst/lib and running install from there.

Important: we do need some of those updates. Pillow 2.8 for example.

Last edited 3 years ago by Antoine Martin (previous) (diff)

comment:78 Changed 3 years ago by Antoine Martin

Owner: changed from Antoine Martin to Smo
Status: assignednew

I installed pycups using easy_install, because I didn't see in the moduleset..
Then I hit disk space issues... made some space.
Then I hit a PIL.Image packaging issue, needed to add libz to the bundle list, fixed in r9089.
I am moving the 64-bit only issues to #840 (not many at all - at least so far), link to a beta 64-bit build there.
Then I bumped a bunch of libraries: r9091 + r9092.

@smo: I am happy with this and I think we can finally close the ticket.
I'm not sure if I will be using it for the 0.15.x release, but going forward I will definitely be using it for 64-bit builds (#840).
Either close and follow up there, or re-assign to me if you have more fixes / updates.

comment:79 Changed 3 years ago by Smo

r9116 Many version updates from upstream
r9119 libffi version downgrade
r9120 + r9121 gtk-mac-integration version downgrade
r9122 ffmpeg downgrade to latest in 2.5 branch since 2.6 requires newer libvpx and this won't build on osx 10.5

comment:80 Changed 3 years ago by Smo

r9123 remove i386 build options since they are only needed when compiling 32bit on 64bit systems.

I've tested these being removed on a 32bit system and all is fine without them.

This should help us moving to ticket #840. There are some version of libs that we can build on newer systems that we cannot for 32bit builds on 10.5 so we may have to split this up a bit going forward.

I tried to keep it as easy to update from upstream as possible so we can help test their stable builds as well.

Pycups requires a patch on osx 10.5 which is not possible with distutils instructions through jhbuild I may be able to hack this up though.

comment:81 Changed 3 years ago by Smo

r9124 commited the patch you put here for gtk-quartz-engine can't hurt
r9125 add that patch for the moduleset

comment:82 Changed 3 years ago by Smo

I'm going to rebuild the lot from scratch again and if I don't have any serious issues with the 32bit builds I will close this and move on to the 64bit ticket.

comment:83 Changed 3 years ago by Antoine Martin

@smo:

  • why downgrade apr and apr-util in r9116? (no biggie - as long as svn still builds)
  • are you sure about r9122? I've got ffmpeg 2.6.1 installed on my 10.5 VM without problems - what's the error you're seeing?

comment:84 Changed 3 years ago by Smo

the downgrade to apr and apr-util in r9116 wasn't intentional I guess I missed that one when I was looking at it. This is probably something that should be fixed as apr and apr-util are now in bootstrap so they aren't necessary in xpra.modules. I will look into this further.

I will have to try ffmpeg 2.6.1 again and see what the error was it was related to vp9.

I'm also in the process of cleaning this up a bit. Made some mess when I was dealing with patches.

comment:85 Changed 3 years ago by Smo

My bad about ffmpeg 2.6.1 those were warnings and jhbuild wasn't printing out the actual error which was

LD      libavformat/libavformat.56.dylib
HTML    doc/ffmpeg-utils.html
HTML    doc/ffmpeg-scaler.html
HTML    doc/ffmpeg-resampler.html
Unknown open() mode ':encoding(utf-8-strict)' at /usr/bin/texi2html line 11111.
make: *** [doc/ffmpeg-utils.html] Error 255
make: *** Waiting for unfinished jobs....
Unknown open() mode ':encoding(utf-8-strict)' at /usr/bin/texi2html line 11111.
make: *** [doc/ffmpeg-scaler.html] Error 255
Unknown open() mode ':encoding(utf-8-strict)' at /usr/bin/texi2html line 11111.
make: *** [doc/ffmpeg-resampler.html] Error 255

adding --disable-doc to the configure line seems to fix this issue so I will commit those fixes

comment:86 Changed 3 years ago by Smo

Still a couple things to do here.

  • fix gtk-mac-bundler Makefile to install it differently
  • add pycups somehow so we can have printer support without installing this manually

comment:87 Changed 3 years ago by Antoine Martin

Milestone: 0.15

(just setting milestone)

comment:88 Changed 2 years ago by Antoine Martin

Tested with a completely clean user account on my 10.5.8 VM and got this error:

configure: error: Oops, mp_limb_t is 32 bits, but the assembler code
in this configuration expects 64 bits.
You appear to have set $CFLAGS, perhaps you also need to tell GMP the
intended ABI, see "ABI and ISA" in the manual.
*** Error during phase configure of gmplib: ########## \
    Error running ./configure --prefix /Users/osx/gtk/inst --libdir '/Users/osx/gtk/inst/lib'    *** [71/97]

Fixed by re-adding --build=i386-darwin. That's odd, is anything else building for 64 bit??
@smo: we may have to re-add this one and workaround it for 64-bit.

Another really annoying problem is that the apr downloads seem to disappear with new releases (see http://www.eu.apache.org/dist/apr/), which is a really bad idea for those building from source like us, maybe we should mirror those? Otherwise we run the risk of not being able to build things in the future.
I've used apr 1.5.2 for building and updated the moduleset, also updated subversion to 1.8.13

comment:89 Changed 2 years ago by Antoine Martin

I have just uploaded an osx beta build (tagged with r9198) made from this moduleset environment.
@afarr: testing it wouldn't hurt, making sure there are no regressions from previous "non moduleset" builds.

PS: unless problems are reported against these builds, from now on all the 0.15 builds (and later) will be made from this "moduleset" environment.

Last edited 2 years ago by Antoine Martin (previous) (diff)

comment:91 Changed 2 years ago by Smo

Resolution: fixed
Status: newclosed

r9279 updates gtk to 2.24.27 builds fine

I'm going to close this now while I can and move on to 64bit builds.

comment:92 Changed 2 years ago by Antoine Martin

@smo: FYI r10091 bumped x264 to a newer stable snapshot. This caused hard to decipher problems at package time because some dylibs still referred to the old one. (libx264.142.dylib vs libx264.146.dylib)

Fixed by rebuilding gstreamer with:

jhbuild buildone -f 

I wished jhbuild had the equivalent of revdep-rebuild under Gentoo. There might be a way of cooking up a check script using otool to verify all the dylibs found in gtk/inst/lib can be loaded.
Until then, we'll fix things when they break and hope the breakage manifests itself at package time rather than at runtime..

Similar dependency problem in #971.

Follow up for 64-bit in #840.

Last edited 16 months ago by Antoine Martin (previous) (diff)

comment:93 Changed 22 months ago by Antoine Martin

@smo: FYI lots of updates in r11631.

comment:94 Changed 8 months ago by Antoine Martin

Note: the pygtkglext patch should be updated to support "get_depth" which we commented out: #1443

Note: See TracTickets for help on using tickets.