2.3 moduleset updates

[Antoine Martin]

Follow up from #1587.

One important update worth re-trying is gstreamer 1.12 (or 1.14 if available), see ticket:1501#comment:6 for issues.

Link to osx changelog: log/xpra/trunk/osx.

[Antoine Martin]

• r17694 + r17695: pyobjc 4.1
• r17615: ffmpeg 3.4.1
• r17633: libjpeg-turbo 1.5.3
• r17730: libx264 20171222 - this requires a rebuild of gst-plugings-ugly so dep added in r17737
• r17728: python-lz4 0.13.0
• r17747 + r17748 + r17749 + r17808: python-lz4 0.18.1 (and its new dependencies..)
• r17762: websocket-client 0.45.0

[Antoine Martin]

Full rebuild:

• png error:

  In file included from png.c:14:
  ./pngpriv.h:898:4: error: ZLIB_VERNUM != PNG_ZLIB_VERNUM "-I (include path) error: see the notes in pngpriv.h"
  #  error ZLIB_VERNUM != PNG_ZLIB_VERNUM \
     ^
  

• x264 builds fine but fails to package because the library references the temporary path it gets installed into:

  $ grep -r jhbuild/root-x264 /Users/osx/gtk/inst/lib
  Binary file /Users/osx/gtk/inst/lib/gstreamer-1.0/libgstx264.so matches
  Binary file /Users/osx/gtk/inst/lib/libx264.148.dylib matches
  Binary file /Users/osx/gtk/inst/lib/libx264.dylib matches
  Binary file /Users/osx/gtk/inst/lib/python2.7/site-packages/xpra/codecs/enc_x264/encoder.so matches
  

Rebuilding and copying it by hand fixes things:

jhbuild shell
cd ${JHBUILD_SOURCE}/x264-snapshot-20171222-2245-stable
make clean && make
cp libx264.*.dylib ${JHBUILD_PREFIX}/

(then wipe clean and rebuild gst-plugins-ugly-1.0)

@smo: ideas?

[Antoine Martin]

Some important updates, most already recorded in ticket:1575#comment:9 and ticket:1575#comment:10:

• r17828: Pillow 5.0
• r17840 + r17841 + r17844: split meta modules
• r17842: apr 1.6.3
• r17843: openssl-1.1.0g
• r17845: Python 3.6.4
• r17846 + r17847 + r17848 + r17849: reduce our diff against upstream
• r17850: setuptools 38.2.5
• r17851: macholib-1.9, modulegraph-0.16, altgraph-0.14
• r17856: add asn1crypto, needed by python-cryptography
• r17864: python-lz4 mess, needs python-test-runner now,
• r17866: remove Quartz theme

[Antoine Martin]

Lots more moduleset refactoring done to support python3 (see ticket:1575#comment:12), and also:

• r17943: gstreamer 1.12 does seem to work ok

[Smo]

As this is for gtk2 portion of the moduleset I will post my findings here.

There were only a few issues with the gtk2 moduleset.

** Applying patch https://xpra.org/svn/Xpra/trunk/osx/jhbuild/patches/glib-bug-772454-disable-kqueue.patch *** [19/130]
patch -p1 < "/Users/xpra/.cache/jhbuild/glib-bug-772454-disable-kqueue.patch"
patch: **** Only garbage was found in the patch input.
*** Error during phase checkout of glib: ########## Error running patch -p1 < "/Users/xpra/.cache/jhbuild/glib-bug-772454-disable-kqueue.patch" *** [19/130]

The patch here has the wrong contents.

building 'lzo' extension
/usr/bin/gcc -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -arch x86_64 -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk/usr/include -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk -mmacosx-version-min=10.13 -I/Users/xpra/gtk/inst/include -arch x86_64 -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk/usr/include -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk -I/Users/xpra/gtk/inst/include/lzo -I/Users/xpra/gtk/inst/include/python2.7 -c lzomodule.c -o build/temp.macosx-10.13-x86_64-2.7/lzomodule.o
lzomodule.c:35:10: fatal error: 'lzo1x.h' file not found
#include <lzo1x.h>
         ^~~~~~~~~
1 error generated.
error: command '/usr/bin/gcc' failed with exit status 1
*** Error during phase build of python-lzo: ########## Error running python setup.py build *** [72/130]

lzo isn't being built before python-lzo can be fixed quickly by jhbuild buildone lzo

*** Applying patch http://xpra.org/svn/Xpra/trunk/osx/jhbuild/patches/lame-channels.patch *** [100/130]
patch -p1 < "/Users/xpra/.cache/jhbuild/lame-channels.patch"
patching file libmp3lame/set_get.c
patch: **** unexpected end of file in patch
*** Error during phase checkout of lame: ########## Error running patch -p1 < "/Users/xpra/.cache/jhbuild/lame-channels.patch" *** [100/130]

Seems to be an issue with this patch too.

[Antoine Martin]

browser/xpra/trunk/osx/jhbuild/patches/glib-bug-772454-disable-kqueue.patch
The patch here has the wrong contents.

Yep. And I can't find an old copy anywhere. I believe I found the bug this refers to: ​Bug 772454 - Gnucash crashes when opening new file
Going forward we may need to mirror the patches if we want to have a way to rebuild things reliably.

We tried moving to the new glib in r17849, then downgraded the python2 / gtk2 moduleset back to the version requiring this patch in r17910 (+ r17913 for gi).
The patches were mirrored in our repo since r17912, but a few didn't make it (including this one - maybe the urls went 404 and those ended up being the contents?).

Anyway, maybe it is worth trying glib 2.50 or 2.52 again?

lzo isn't being built before python-lzo can be fixed quickly by jhbuild buildone lzo

Saw this too, we already have <after>lzo</after> in the python-lzo module, maybe I misunderstand what "after" means then?
Anyway, I did the same thing (jhbuild buildone lzo) and moved on.

lame-channels.patch
Seems to be an issue with this patch too.

Was fixed a few days ago in r17923 (and also r17885 for the patch prefix), maybe you need to wipe your copy in the jhbuild cache (.cache/jhbuild/*patch)

[Smo]

r17971 adds back proper ​https://www.xpra.org/trac/browser/xpra/trunk/osx/jhbuild/patches/glib-bug-772454-disable-kqueue.patch

Found a cached copy on an older machine.

[Antoine Martin]

Updates:

• r17994 + r17995 + r18091: lz4 / python-lz4 (are we correctly using the pseudo system lz4 rather than the one bundled with python-lz4?)
• r18015 + r18020: remove quartz theme, use gtk3 themes with gtk3 moduleset
• r18152: gtk3 packaging fixes
• r18174 + r18175: libvpx 1.7 - build errors

@smo: can you try libvpx 1.7?

[Antoine Martin]

• r18294 + r18295: libvpx is now fixed
• r18152: gi packaging fixes
• r18243: python-lz4 0.19.2 - important, memleak!
• r18246: build AVFoundation for #1231

[Antoine Martin]

Regarding python3 support, here's the reply from jralls on gtk-osx-devel: ​Re: python3 modules.
I'm trying to get my head around this one, smo: what do you think we should do about this? at least the after thing?

[Antoine Martin]

Updates:

• AVFoundation webcam bits: r18246, r18298 workaround for packaging: ticket:1231#comment:11
• fixups: r18296, r18297,
• switch back to newer gobject-introspection + glib, get closer to gtk-osx upstream: r18306 + r18307 (works if built in the correct order..)
• cosmetic, reduce delta with upstream gtk-osx now that our changes have been merged: r18308
• python-lz4: r18318 + r18455
• opengl script fix: r18437
• kerberos + gssapi: r18698 (#1691)
• cython 0.28.1: r18717 + r18757 + r18762
• ffmpeg 3.4.2: r18755

Will follow up in #1787

[Antoine Martin]

Updates:

• r18781: python-gssapi for #1691
• r18828: python-ldap for #1791
• r18844: python-ldap3 for #1791
• r18802: python-u2f for #1789, r18816: add python-u2f utility script
• minor fixes: r18806, r18829, r18952, r18953, r19011
• r18852: websocket-client 0.47.0
• r18972: python-cryptography 2.2.2
• r18973: x264 20180401
• r19010: pillow 5.1.0

[Smo]

Couple issues when building from scratch again.

to fix pygobject building with new > 2.52 exit to shell and do this then continue build.

cp ../glib-2.52.2/gio/gdesktopappinfo.h ~/gtk/inst/include/glib-2.0/gio/

LZO doesn't seem to build before python-lzo maybe we need <after> instead of depends or both?

*** Building python-lzo *** [70/136]
python setup.py build
/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py:251: UserWarning: 'licence' distribution option is deprecated; use 'license'
  warnings.warn(msg)
running build
running build_ext
building 'lzo' extension
/usr/bin/gcc -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -arch x86_64 -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk/usr/include -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk -mmacosx-version-min=10.13 -I/Users/xpra/gtk/inst/include -arch x86_64 -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk/usr/include -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk -arch i386 -arch x86_64 -pipe -I/Users/xpra/gtk/inst/include/lzo -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c lzomodule.c -o build/temp.macosx-10.13-intel-2.7/lzomodule.o
lzomodule.c:35:10: fatal error: 'lzo1x.h' file not found
#include <lzo1x.h>

To fix just exit the build and jhbuild buildone lzo then continue and it should be fine

Seems to be an issue with one of the lame patches?

*** Checking out lame *** [106/136]
curl --continue-at - -L http://winswitch.org/src/lame-3.99.5.tar.gz -o /Users/xpra/gtk/source/pkgs/lame-3.99.5.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 1411k  100 1411k    0     0   241k      0  0:00:05  0:00:05 --:--:--  293k
gzip -dc "/Users/xpra/gtk/source/pkgs/lame-3.99.5.tar.gz" | tar xf -
*** Applying patch http://xpra.org/svn/Xpra/trunk/osx/jhbuild/patches/lame-channels.patch *** [106/136]
patch -p1 < "/Users/xpra/.cache/jhbuild/lame-channels.patch"
patching file libmp3lame/set_get.c
patch: **** unexpected end of file in patch

Not sure what this one is about.