xpra icon
Bug tracker and wiki

Opened 9 months ago

Closed 2 months ago

#1787 closed task (fixed)

2.4 moduleset updates

Reported by: Antoine Martin Owned by: Smo
Priority: major Milestone: 2.4
Component: platforms Version: trunk
Keywords: Cc:

Description (last modified by Antoine Martin)

Follow up from #1678.

Log URL: log/xpra/trunk/osx

Change History (14)

comment:1 Changed 9 months ago by Antoine Martin

Description: modified (diff)
Status: newassigned

comment:2 Changed 7 months ago by Antoine Martin

Milestone: 3.02.4
Summary: 3.0 moduleset updates2.4 moduleset updates
Version: 2.2.xtrunk

comment:3 Changed 7 months ago by Antoine Martin

Updated:

Last edited 7 months ago by Antoine Martin (previous) (diff)

comment:4 Changed 7 months ago by Antoine Martin

comment:5 Changed 7 months ago by Antoine Martin

Description: modified (diff)

Updated:

Last edited 6 months ago by Antoine Martin (previous) (diff)

comment:6 Changed 6 months ago by Antoine Martin

Owner: changed from Antoine Martin to Smo
Status: assignednew

r19672 updates to ffmpeg 4.0.1 and I now hit a problem during packaging:

calling 'gtk-mac-bundler Xpra.bundle' in /Users/osx/Xpra-svn/trunk/osx
Cannot find source to copy: /Users/osx/gtk/inst/_jhbuild/root-ffmpeg/Users/osx/gtk/inst/lib/libavcodec.58.dylib
ERROR: gtk-mac-bundler failed

That's because the library dependency is wrong on all the ffmpeg libraries, ie:

$ otool -L image/.Launcher.app/Contents/Resources/lib/libavformat.58.12.100.dylib
image/.Launcher.app/Contents/Resources/lib/libavformat.58.12.100.dylib:
        /Users/gtk3/gtk/inst/_jhbuild/root-ffmpeg/Users/gtk3/gtk/inst/lib/libavformat.58.dylib (compatibility version 58.0.0, current version 58.12.100)
        /Users/gtk3/gtk/inst/_jhbuild/root-ffmpeg/Users/gtk3/gtk/inst/lib/libavcodec.58.dylib (compatibility version 58.0.0, current version 58.18.100)
        /Users/gtk3/gtk/inst/_jhbuild/root-ffmpeg/Users/gtk3/gtk/inst/lib/libavutil.56.dylib (compatibility version 56.0.0, current version 56.14.100)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1226.10.1)
        /Users/gtk3/gtk/inst/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11)
        /usr/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0)
        /System/Library/Frameworks/VideoToolbox.framework/Versions/A/VideoToolbox (compatibility version 1.0.0, current version 1.0.0)
        /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1256.14.0)
        /System/Library/Frameworks/CoreMedia.framework/Versions/A/CoreMedia (compatibility version 1.0.0, current version 1.0.0)
        /System/Library/Frameworks/CoreVideo.framework/Versions/A/CoreVideo (compatibility version 1.2.0, current version 1.5.0)
        /System/Library/Frameworks/CoreServices.framework/Versions/A/CoreServices (compatibility version 1.0.0, current version 728.6.0)

And this is not caused by the packaging scripts themselves but this is just how the library is installed on the system by jhbuild:

$ otool -L /Users/gtk3/gtk/inst/lib/libswscale.5.1.100.dylib 
/Users/gtk3/gtk/inst/lib/libswscale.5.1.100.dylib:
	/Users/gtk3/gtk/inst/_jhbuild/root-ffmpeg/Users/gtk3/gtk/inst/lib/libswscale.5.dylib (compatibility version 5.0.0, current version 5.1.100)
	/Users/gtk3/gtk/inst/_jhbuild/root-ffmpeg/Users/gtk3/gtk/inst/lib/libavutil.56.dylib (compatibility version 56.0.0, current version 56.14.100)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1226.10.1)
	/System/Library/Frameworks/VideoToolbox.framework/Versions/A/VideoToolbox (compatibility version 1.0.0, current version 1.0.0)
	/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1256.14.0)
	/System/Library/Frameworks/CoreMedia.framework/Versions/A/CoreMedia (compatibility version 1.0.0, current version 1.0.0)
	/System/Library/Frameworks/CoreVideo.framework/Versions/A/CoreVideo (compatibility version 1.2.0, current version 1.5.0)
	/System/Library/Frameworks/CoreServices.framework/Versions/A/CoreServices (compatibility version 1.0.0, current version 728.6.0)

Where to go from here?

  • report upstream?
  • diff 4.0 and 4.0.1 to find the buggy commit?
  • patch it up using install_name_tool (tedious), ie:
    install_name_tool
        -change /Users/gtk3/gtk/inst/_jhbuild/root-ffmpeg/Users/gtk3/gtk/inst/lib/libavutil.56.dylib
        /Users/gtk3/gtk/inst/lib/libavutil.56.dylib
        /Users/gtk3/gtk/inst/lib/libswscale.5.1.100.dylib
    

@smo: ideas?

comment:7 Changed 6 months ago by Antoine Martin

I've downgraded ffmpeg to 4.0 by hand and packaging is working again.

comment:8 Changed 6 months ago by Antoine Martin

  • r19775 updates to python 3.6.6, but the build steps get stuck in a loop! Ideas?
  • r19807: pyobjc 4.2.2 and pillow 5.2.0
  • r19811: python-lz4 2.0.1
  • r19871: python-lz4 2.0.2
Last edited 5 months ago by Antoine Martin (previous) (diff)

comment:9 Changed 5 months ago by Antoine Martin

The GTK3 build issues I have seen seem to be caused by gtk 3.22.28: symbol not found (missing GDK_AVAILABLE_IN_ALL) and updating the GTK3 version from gtk-osx upstream should fix that.

paramiko has a compatibility problem with gssapi:paramiko 2.4.1 fails when current version of gssapi is installed so we disable gssapi: r19964.

Last edited 5 months ago by Antoine Martin (previous) (diff)

comment:10 Changed 4 months ago by Antoine Martin

Had problems with libjpeg-turbo 2.0.0:

  • r19974: libjpeg-turbo 2.0.0
  • r20085: use github for downloading libjpeg-turbo (some people still use sf.net??)
  • r20086: use cmake

It no longer uses autotools to build, so I had to find the new magic incantations, we cannot use the macos version of nasm which is too old and buggy, we have to tell libjpeg-turbo to honour the one that's first in the path (and why doesn't it?):

cmake -DCMAKE_ASM_NASM_COMPILER=$PREFIX/bin/nasm -DCMAKE_INSTALL_PREFIX:PATH=$PREFIX .
make
make install

comment:11 Changed 4 months ago by Antoine Martin

Remaining problems:

  • gobject-introspection with python2:
      GICOMP   GIRepository-2.0.gir
    /Volumes/Scratch/GTK2/inst/include/glib-2.0/gio/gio-autocleanups.h:24: syntax error, unexpected typedef-name, expecting identifier or '(' in 'typedef GAction *GAction_autoptr;' at 'GAction_autoptr'
    /Volumes/Scratch/GTK2/inst/include/glib-2.0/gio/gio-autocleanups.h:25: syntax error, unexpected typedef-name, expecting identifier or '(' in '# 24 "/Volumes/Scratch/GTK2/inst/include/glib-2.0/gio/gio-autocleanups.h"' at 'GActionMap_autoptr'
    ...
    Undefined symbols for architecture x86_64:
      "_g_desktop_app_info_get_type", referenced from:
          _GI_GET_TYPE_FUNCS_ in Gio-2.0.o
      "_g_desktop_app_info_lookup_get_type", referenced from:
          _GI_GET_TYPE_FUNCS_ in Gio-2.0.o
    ld: symbol(s) not found for architecture x86_64
    clang: error: linker command failed with exit code 1 (use -v to see invocation)
    linking of temporary binary failed: Command '[u'/bin/sh', u'./libtool', u'--mode=link', u'--tag=CC', u'--silent', '/usr/bin/gcc', u'-o', u'/Volumes/Scratch/GTK2/source/gobject-introspection-1.52.1/tmp-introspectIdBHrC/Gio-2.0', u'-export-dynamic', '-I/Users/osx/gtk/inst/include', '-arch', 'x86_64', '-I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk/usr/include', '-isysroot', '/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk', '-fno-strict-aliasing', '-Wsign-compare', '-Wcast-align', '-Wpointer-arith', '-Wnested-externs', '-Wmissing-prototypes', '-Wmissing-declarations', '-Wchar-subscripts', '-Wall', '-arch', 'x86_64', '-I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk/usr/include', '-isysroot', '/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk', '-mmacosx-version-min=10.10', u'tmp-introspectIdBHrC/Volumes/Scratch/GTK2/source/gobject-introspection-1.52.1/tmp-introspectIdBHrC/Gio-2.0.o', u'-L/Users/osx/gtk/inst/lib', u'-lgmodule-2.0', u'-lgio-2.0', u'-lgobject-2.0', u'-lglib-2.0', u'-lintl', u'-Wl,-framework', u'-Wl,CoreFoundation', '-lgio-2.0', '-L/Users/osx/gtk/inst/lib', '-L/Users/osx/gtk/inst/lib', '-arch', 'x86_64', '-L/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk/usr/lib', '-isysroot', '/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk', '-mmacosx-version-min=10.10', '-Wl,-headerpad_max_install_names']' returned non-zero exit status 1
    make[2]: *** [Gio-2.0.gir] Error 1
    make[1]: *** [all-recursive] Error 1
    make: *** [all] Error 2
    
  • python 3.7 goes into a loop:
    make -j 5 
    CONFIG_FILES=Makefile.pre CONFIG_HEADERS= /bin/sh config.status
    -----------------------------------------------
    Modules/Setup.dist is newer than Modules/Setup;
    check to make sure you have all the updates you
    need in your Modules/Setup file.
    Usually, copying Modules/Setup.dist to Modules/Setup will work.
    -----------------------------------------------
    config.status: creating Makefile.pre
    /Applications/Xcode.app/Contents/Developer/usr/bin/make -f Makefile.pre Makefile
    CONFIG_FILES=Makefile.pre CONFIG_HEADERS= /bin/sh config.status
    (...)
    config.status: creating Makefile.pre
    /Applications/Xcode.app/Contents/Developer/usr/bin/make -f Makefile.pre Makefile
    CONFIG_FILES=Makefile.pre CONFIG_HEADERS= /bin/sh config.status
    config.status: fork: Resource temporarily unavailable
    config.status: fork: Resource temporarily unavailable
    make[639]: *** [Makefile.pre] Error 128
    make[638]: *** [Makefile.pre] Error 2
    make[637]: *** [Makefile.pre] Error 2
    make[636]: *** [Makefile.pre] Error 2
    (..)
    make[2]: *** [Makefile.pre] Error 2
    make[1]: *** [Makefile.pre] Error 2
    make: *** [Makefile.pre] Error 2
    *** Error during phase build of python3: ########## Error running make -j 5  *** [10/139]
    
  • gtk3:
            GTK+ 3.22.30
            ===========
    
            GDK backends:         quartz
            Print backends:       file lpr cups
            Dynamic modules:      yes
            Included immodules:   none
            colord support:       no
            Introspection:        yes
            Debugging:            minimum
            Documentation:        no
    
    (..)
      GISCAN   Gtk-3.0.gir
    gtkentry.c:2094: Warning: Gtk: multiple comment blocks documenting 'GtkEntry:inner-border:' identifier (already seen at gtkentry.c:885).
    dyld: Symbol not found: __gtk_marshal_BOOLEAN__BOXED_BOXED
      Referenced from: /Volumes/Scratch/GTK3/source/gtk+-3.22.30/gtk/.libs/libgtk-3.0.dylib
      Expected in: flat namespace
     in /Volumes/Scratch/GTK3/source/gtk+-3.22.30/gtk/.libs/libgtk-3.0.dylib
    Command '['/Volumes/Scratch/GTK3/source/gtk+-3.22.30/gtk/tmp-introspectw2opb1cl/Gtk-3.0', '--introspect-dump=/Volumes/Scratch/GTK3/source/gtk+-3.22.30/gtk/tmp-introspectw2opb1cl/functions.txt,/Volumes/Scratch/GTK3/source/gtk+-3.22.30/gtk/tmp-introspectw2opb1cl/dump.xml']' died with <Signals.SIGTRAP: 5>.
    make[3]: *** [Gtk-3.0.gir] Error 1
    make[2]: *** [all] Error 2
    make[1]: *** [all-recursive] Error 1
    make: *** [all] Error 2
    
  • pygobject3:
    Making all in tests
    test -L libgimarshallingtests.dylib || ln -s .libs/libgimarshallingtests.dylib libgimarshallingtests.dylib
    test -L libregress.dylib || ln -s .libs/libregress.dylib libregress.dylib
      GEN      testhelper.la
    ld: -stack_size option can only be used when linking a main executable
    clang: error: linker command failed with exit code 1 (use -v to see invocation)
    make[2]: *** [testhelper.la] Error 1
    make[1]: *** [all-recursive] Error 1
    make: *** [all] Error 2
    

This can be ignored, either by removing the offending stack_size option or by skipping the tests altogether.

comment:12 Changed 4 months ago by Antoine Martin

Still more problems caused by the new libjpeg-turbo built with cmake:

$ otool  -L /Users/osx/gtk/inst/lib/libjpeg.dylib 
/Users/osx/gtk/inst/lib/libjpeg.dylib:
	@rpath/libjpeg.62.dylib (compatibility version 62.0.0, current version 62.3.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1226.10.1)

And then a number of libraries end up referencing that:

$ otool -L /Users/osx/gtk/inst/lib/gdk-pixbuf-2.0/2.10.0/loaders/libpixbufloader-jpeg.so 
/Users/osx/gtk/inst/lib/gdk-pixbuf-2.0/2.10.0/loaders/libpixbufloader-jpeg.so:
	@rpath/libjpeg.62.dylib (compatibility version 62.0.0, current version 62.3.0)
	/Users/osx/gtk/inst/lib/libgdk_pixbuf-2.0.0.dylib (compatibility version 3601.0.0, current version 3601.6.0)
	/Users/osx/gtk/inst/lib/libgmodule-2.0.0.dylib (compatibility version 5201.0.0, current version 5201.2.0)
	/Users/osx/gtk/inst/lib/libgio-2.0.0.dylib (compatibility version 5201.0.0, current version 5201.2.0)
	/Users/osx/gtk/inst/lib/libgobject-2.0.0.dylib (compatibility version 5201.0.0, current version 5201.2.0)
	/Users/osx/gtk/inst/lib/libglib-2.0.0.dylib (compatibility version 5201.0.0, current version 5201.2.0)
	/Users/osx/gtk/inst/lib/libintl.9.dylib (compatibility version 11.0.0, current version 11.4.0)
	/Users/osx/gtk/inst/lib/libpng16.16.dylib (compatibility version 51.0.0, current version 51.0.0)
	/Users/osx/gtk/inst/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1226.10.1)
	/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1256.14.0)

So then things fail much later in mysterious ways. ie: the bundler fails with:

Cannot find a matching prefix for @rpath/libjpeg.62.dylib

Trying to debug the cmake build is hard, so much of it is just pretty colouring, the rest is levels and levels of indirection... I'm no fan of Makefiles and autotools, but at least that's a known quantity.
Linking libjpeg:

[ 92%] Linking C shared library ../libjpeg.dylib
+ cd /Users/osx/gtk/source/libjpeg-turbo-2.0.0/sharedlib
+ /Users/osx/gtk/inst/bin/cmake -E cmake_link_script CMakeFiles/jpeg.dir/link.txt --verbose=
+ cd /Users/osx/gtk/source/libjpeg-turbo-2.0.0/sharedlib
+ /Users/osx/gtk/inst/bin/cmake -E cmake_symlink_library ../libjpeg.62.3.0.dylib ../libjpeg.62.dylib ../libjpeg.dylib
+ /Users/osx/gtk/inst/bin/cmake -E touch_nocreate libjpeg.dylib
+ /Users/osx/gtk/inst/bin/cmake -E cmake_echo_color --switch= \
    --progress-dir=/Users/osx/gtk/source/libjpeg-turbo-2.0.0/CMakeFiles \
    --progress-num=12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27 'Built target jpeg'

And in CMakeFiles/jpeg.dir/link.txt:

/Applications/Xcode.app/Contents/Developer/usr/bin/gcc -arch x86_64 \
    -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk/usr/include \
    -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk \
    -mmacosx-version-min=10.10 -O3 -DNDEBUG \
    -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk \
    -mmacosx-version-min=10.10 -dynamiclib -Wl,-headerpad_max_install_names \
    -L/Users/osx/gtk/inst/lib -L/Users/osx/gtk/inst/lib -arch x86_64 \
    -L/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk/usr/lib \
    -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk \
    -mmacosx-version-min=10.10 -Wl,-headerpad_max_install_names -compatibility_version 62.0.0 -current_version 62.3.0 \
    -o ../libjpeg.62.3.0.dylib -install_name @rpath/libjpeg.62.dylib \
    CMakeFiles/jpeg.dir/__/jcapimin.c.o CMakeFiles/jpeg.dir/__/jcapistd.c.o CMakeFiles/jpeg.dir/__/jccoefct.c.o \
    (..)
    ../simd/CMakeFiles/simd.dir/x86_64/jsimd.c.o 
Last edited 4 months ago by Antoine Martin (previous) (diff)

comment:13 Changed 4 months ago by Antoine Martin

merge from upstream:

  • r20087 + r20089 + r20100: switch to https urls, gnome moved to gitlab, setuptools on pypi
  • r20088 downgrade gtk+ to fix introspection bug, add pixman patch
  • r20090: gtk-mac-integration 2.1.2, +r20107 patch fix for crash

libjpeg-turbo:

  • r20093 + r20101: fix cmake tax syntax, fix cmake lib installation directory
  • r20091 + r20092 + r20095: url change + reverted (was cmake tag syntax - hidden by jhbuild cryptic errors)
  • r20102: revert to version 1.5.3 (couldn't figure out how to make it link properly)

updates:

  • r20096: python 3.7
  • r20097: new setuptools version needed for python 3.7
  • r20094: remove gst-python, fix typo in paramiko
  • r20098: cut&paste typo
  • r20099: cffi 1.11.5
  • r20103: revert to ffmpeg 4.0 (4.0.1 onwards don't package properly - no clue from the git diff between those two versions)
  • r20106: updated linker patch for python 2.7.15
  • r20108: re-instate gflags patch, again
  • r20140 + r20148: pyopengl 3.1.3b1
  • r20141: websocket-client 0.51 (constant breakage there - every version seems to break something..)
  • r20149 + r20150 + r20151: lz4 1.8.2 (with github URL change PITA)
  • r20167: GTK3 segfault fix (#1938)
  • r20171: python-lz4 packaging (see #1929)
  • r20203: numpy 1.15.1
  • r20311: websocket-client 0.52.0
  • r20388: openssl 1.1.1
  • r20415: use upstream patch
  • r20416: move patches
  • r20441: workaround jhbuild dependencies being ignored and list all of them in the main meta module
  • r20531: gtk-mac-integration 2.1.3
  • r20552: numpy 1.15.2
  • r20640: pillow 5.3.0
Last edited 2 months ago by Antoine Martin (previous) (diff)

comment:14 Changed 2 months ago by Antoine Martin

Resolution: fixed
Status: newclosed

For 2.5, see #1985

Note: See TracTickets for help on using tickets.