xpra icon
Bug tracker and wiki

Opened 2 years ago

Closed 2 years ago

Last modified 4 months ago

#1306 closed defect (fixed)

osx moduleset changes tracker ticket

Reported by: Antoine Martin Owned by: Smo
Priority: critical Milestone: 1.0
Component: platforms Version: trunk
Keywords: osx Cc:

Description

Tracking the latest changes and recording why / how things work the way they do.

Related to #533 (original moduleset ticket), #840 (64-bit), #641 (osx packaging)
Recent osx tickets: #1271 (websocket client changes)
Moduleset updates and build errors: #1007 and #544 (misc lib updates), #971 (build failures), r9116, ..
Sound changes: #970 (switch to gstreamer 1.x) #1069 (gstreamer 1.6), #1168 (gstreamer 1.8)

Recent changes: r13715, r13714, r13713, r13712, r13711, r13710, r13709, r13708, r13707, r13706, r13705, r13704, r13703, r13702, r13701, r13700, r13699, r13698, r13697, r13696, r13695, r13694, r13693, r13692

Got some errors updating today:

  • python tries to build the outdated 2.7.10 version and fails:
    xzcat -d "/Users/osx/gtk/source/pkgs/Python-2.7.10.tar.xz" | tar xf -
    *** Error during phase checkout of python: could not download patch (error: 404) *** [3/119]
    

(skipped)

  • libjpeg turbo is using an older version - why? (I have 1.5 build on 10.5.x so I've skipped this change too)
  • gtk+ fails (patch not needed?):
    *** Applying patch http://git.gnome.org/browse/gtk-osx/plain/patches/0001-Bug-707945-GTK2-Quartz-typeahead-find-in-GtkTreeView.patch *** [31/119]
    patch -p1 < "/Users/osx/.cache/jhbuild/0001-Bug-707945-GTK2-Quartz-typeahead-find-in-GtkTreeView.patch"
    patching file modules/input/imquartz.c
    Reversed (or previously applied) patch detected!  Assume -R? [n] 
    
  • are there any other modules we are downgrading because of this upstream merge?
  • libffi fails to build:
    libtool: compile:  /usr/bin/gcc-4.2 -DHAVE_CONFIG_H -I. -I. -I./include -Iinclude -I./src -I/Users/osx/gtk/inst/include -arch i386 -I/Developer/SDKs/MacOSX10.5.sdk/usr/include -I/Users/osx/gtk/inst/include -arch i386 -I/Developer/SDKs/MacOSX10.5.sdk/usr/include -isysroot /Developer/SDKs/MacOSX10.5.sdk -isysroot /Developer/SDKs/MacOSX10.5.sdk -I. -I./include -Iinclude -I./src -arch i386 -I/Developer/SDKs/MacOSX10.5.sdk/usr/include -arch i386 -I/Developer/SDKs/MacOSX10.5.sdk/usr/include -isysroot /Developer/SDKs/MacOSX10.5.sdk -mmacosx-version-min=10.5 -isysroot /Developer/SDKs/MacOSX10.5.sdk -mmacosx-version-min=10.5 -MT src/x86/win32.lo -MD -MP -MF src/x86/.deps/win32.Tpo -c src/x86/win32.S  -fno-common -DPIC -o src/x86/.libs/win32.o
    src/x86/win32.S:1325:section difference relocatable subtraction expression, ".LFE5" minus ".LFB5" using a symbol at the end of section will not produce an assembly time constant
    src/x86/win32.S:1325:use a symbol with a constant value created with an assignment instead of the expression, L_const_sym = .LFE5 - .LFB5
    src/x86/win32.S:1317:section difference relocatable subtraction expression, ".LEFDE5" minus ".LASFDE5" using a symbol at the end of section will not produce an assembly time constant
    src/x86/win32.S:1317:use a symbol with a constant value created with an assignment instead of the expression, L_const_sym = .LEFDE5 - .LASFDE5
    src/x86/win32.S:unknown:missing indirect symbols for section (__IMPORT,__jump_table)
    make[2]: *** [src/x86/win32.lo] Error 1
    make[1]: *** [all-recursive] Error 1
    make: *** [all] Error 2
    *** Error during phase build of libffi: ########## Error running make -j 2 *** [16/119]
    
  • pixman fails to build:
      CCLD     infinite-loop
    Undefined symbols:
      "_blend_soft_light", referenced from:
          _blend_funcs.10173 in libutils.a(utils.o)
      "_blend_darken", referenced from:
          _blend_funcs.10173 in libutils.a(utils.o)
      "_blend_hard_light", referenced from:
          _blend_funcs.10173 in libutils.a(utils.o)
      "_blend_difference", referenced from:
          _blend_funcs.10173 in libutils.a(utils.o)
      "_blend_exclusion", referenced from:
          _blend_funcs.10173 in libutils.a(utils.o)
      "_blend_overlay", referenced from:
          _blend_funcs.10173 in libutils.a(utils.o)
      "_blend_color_dodge", referenced from:
          _blend_funcs.10173 in libutils.a(utils.o)
      "_blend_screen", referenced from:
          _blend_funcs.10173 in libutils.a(utils.o)
      "_blend_multiply", referenced from:
          _blend_funcs.10173 in libutils.a(utils.o)
      "_blend_color_burn", referenced from:
          _blend_funcs.10173 in libutils.a(utils.o)
      "_blend_lighten", referenced from:
          _blend_funcs.10173 in libutils.a(utils.o)
    ld: symbol(s) not found
    collect2: ld returned 1 exit status
    make[2]: *** [infinite-loop] Error 1
    make[2]: *** Waiting for unfinished jobs....
      CCLD     trap-crasher
    make[1]: *** [all-recursive] Error 1
    make: *** [all] Error 2
    *** Error during phase build of pixman: ########## Error running make -j 2 *** [18/119]
    
  • gtk-mac-integration-python fails to build:
    libtool: compile:  /usr/bin/gcc-4.2 -DHAVE_CONFIG_H -I. -I.. -I.. -I/Users/osx/gtk/inst/include -arch i386 -I/Developer/SDKs/MacOSX10.5.sdk/usr/include -I/Users/osx/gtk/inst/include -arch i386 -I/Developer/SDKs/MacOSX10.5.sdk/usr/include -isysroot /Developer/SDKs/MacOSX10.5.sdk -isysroot /Developer/SDKs/MacOSX10.5.sdk -D_REENTRANT -I/Users/osx/gtk/inst/include/gtk-3.0 -I/Users/osx/gtk/inst/include/gio-unix-2.0/ -I/Users/osx/gtk/inst/include/cairo -I/Users/osx/gtk/inst/include/pango-1.0 -I/Users/osx/gtk/inst/include/atk-1.0 -I/Users/osx/gtk/inst/include/cairo -I/Users/osx/gtk/inst/include/pixman-1 -I/Users/osx/gtk/inst/include/freetype2 -I/Users/osx/gtk/inst/include/libpng16 -I/Users/osx/gtk/inst/include/freetype2 -I/Users/osx/gtk/inst/include/libpng16 -I/Users/osx/gtk/inst/include/gdk-pixbuf-2.0 -I/Users/osx/gtk/inst/include/libpng16 -I/Users/osx/gtk/inst/include/glib-2.0 -I/Users/osx/gtk/inst/lib/glib-2.0/include -xobjective-c -arch i386 -I/Developer/SDKs/MacOSX10.5.sdk/usr/include -arch i386 -I/Developer/SDKs/MacOSX10.5.sdk/usr/include -isysroot /Developer/SDKs/MacOSX10.5.sdk -mmacosx-version-min=10.5 -isysroot /Developer/SDKs/MacOSX10.5.sdk -mmacosx-version-min=10.5 -MT libgtkmacintegration_gtk3_la-GtkApplicationNotify.lo -MD -MP -MF .deps/libgtkmacintegration_gtk3_la-GtkApplicationNotify.Tpo -c GtkApplicationNotify.c  -fno-common -DPIC -o .libs/libgtkmacintegration_gtk3_la-GtkApplicationNotify.o
    In file included from GtkApplicationDelegate.c:23:
    GtkApplicationDelegate.h:25: error: cannot find protocol declaration for 'NSApplicationDelegate'
    make[2]: *** [libgtkmacintegration_gtk3_la-GtkApplicationDelegate.lo] Error 1
    make[2]: *** Waiting for unfinished jobs....
    mv -f .deps/libgtkmacintegration_gtk3_la-GtkApplicationNotify.Tpo .deps/libgtkmacintegration_gtk3_la-GtkApplicationNotify.Plo
    make[1]: *** [all-recursive] Error 1
    make: *** [all] Error 2
    *** Error during phase build of gtk-mac-integration-python: ########## Error running make -j 2 *** [41/119]
    
  • many more packages were not built because of earlier failures or package downgrades I decided to skip

Until we move to a clean 64-bit build env, we need this one to remain stable.
(and we'll keep updating this one with minor stuff like ffmpeg and friends after then 1.0 release, and use trunk for the more bleeding edge stuff)

Change History (25)

comment:1 Changed 2 years ago by Smo

There are many changes and I didn't have a chance to test them all. Some will have to be reverted i'm sure and some patches tweaked.

Right now I'm trying to solve the issue why it is ignoring my overriding of moduleset with

moduleset = "http://xpra.org/svn/Xpra/trunk/osx/jhbuild/modulesets-stable/gtk-osx.modules"

No matter what I do it seems to be ignoring it. I can't seem to figure this one out I will continue to try.

in .jhbuildrc-custom

comment:2 Changed 2 years ago by Smo

libffi should be reverted to old version for 10.5 even left myself warning in there and missed it

<!-- anything higher than this fails on osx 10.5 -->

r13725 reverts this to older version once again

comment:3 Changed 2 years ago by Smo

pixman had same build error reverting it for now

done in r13728

comment:4 Changed 2 years ago by Smo

I believe I've discovered another issue. Openssl 1.1 doesn't seem to work properly with Python 2.7.12. Sometimes we can see errors like this

AttributeError: 'module' object has no attribute 'sha1'

because the hashlib module is broken

I am going to try and compile python before I install openssl and use the system openssl but this is probably not a good fix.

comment:5 Changed 2 years ago by Smo

r13729 removes the patch that is not needed for gtk+

comment:6 Changed 2 years ago by Smo

I've built python before doing openssl which seems to fix the issue for now. Not sure if python is built with openssl support or not at this point.

Maybe it never did build with openssl support in the first place?

comment:7 Changed 2 years ago by Smo

new glib doesn't build on old system with this error

checking OSX version >= 10.9.0... no
configure: error: OSX version is too old!

Will have to revert this change as well

comment:8 Changed 2 years ago by Smo

glib reverted to 2.44.1 in r13730

comment:9 Changed 2 years ago by Smo

problem with gobject-introspection

configure: error: Package requirements (glib-2.0 >= 2.48.0) were not met:

Requested 'glib-2.0 >= 2.48.0' but version of GLib is 2.44.1

comment:10 Changed 2 years ago by Smo

r13732 reverts gobject-introspection and all associated packages

comment:11 Changed 2 years ago by Antoine Martin

The python 2.7.x vs openssl 1.1 issue is tracked here: https://bugs.python.org/issue26470, this includes a fairly large patch which we may be able to use until python 2.7.13 comes out.

comment:12 Changed 2 years ago by Smo

Everything builds once again for me.

*** success *** [119/119]

Still might be some issues with patches but these are probably from sources that redirect to https.

The process was something like this

jhbuild bootstrap
jhbuild build python 

jhbuild build perl
jhbuild -m ~/Source/jhbuild/modulesets/bootstrap.modules build perl-xml-simple --force

jhbuild build curl

export SSL_CERT_FILE="$HOME/gtk/inst/etc/ssl/cacert.pem" 
curl -o ${SSL_CERT_FILE} -k https://curl.haxx.se/ca/cacert.pem

add to .jhbuildrc-custom so curl works for downloads
os.environ["SSL_CERT_FILE"] = "/Users/osx/gtk/inst/etc/ssl/cacert.pem"

jhbuild build

On older systems here where we can't update jhbuild anymore we probably need to overwrite ~/Sources/jhbuild/modulesets/bootstrap.modules with the one we have.

An update would do this normally its the only file you can't override in .jhbuildrc-custom

Maybe it would be simpler to hold on off new openssl for a bit is there some reason we need the latest?

Last edited 2 years ago by Smo (previous) (diff)

comment:13 Changed 2 years ago by Antoine Martin

Links to changesets: r13725 (libffi downgrade), r13726 (syntax typo), r13727 (removed python patch), r13728 (pixman revert), r13729 (gtk patch removed), r13730 (glib revert), r13731 (patch no longer applies?), r13732 (revert gobject-introspection)

Looks like comment:12 needs editing, shouldn't it be "jhbuild build python"?

We can go with openssl 1.0.2 until python 2.7.13 comes out.

Last edited 2 years ago by Antoine Martin (previous) (diff)

comment:14 Changed 2 years ago by Smo

r13786 changes openssl back to 1.0.2h until new python comes out.

Probably change this in the next release.

Going to do a full rebuild to make sure all is well then close this ticket.

comment:15 Changed 2 years ago by Smo

Full rebuild worked no problem now I need to get pkg and dmg out to testing to see how it works.

comment:16 Changed 2 years ago by Smo

r13828 adds patch for bomutils to fix the prefix it installs to

comment:17 Changed 2 years ago by Smo

Here are some changes beteween the stable-moduleset we use and the one from gtk-osx project

Some of these I don't remember so I will have to look into them. Others are because our build host is osx 10.5.x

xpra version gtk-osx version reason
libjpeg-turbo-1.4.2.tar.gzjpegsrc.v9a.tar.gzbetter
libxml2-2.9.2.tar.gzlibxml2-2.9.4.tar.gz??
orc-0.4.25.tar.xzorc-0.4.22.tar.xznewer
gstreamer-1.8.3.tar.xzgstreamer-1.4.3.tar.xznewer
gst-plugins-base-1.8.3.tar.xzgst-plugins-base-1.4.3.tar.xznewer
gst-plugins-good-1.8.3.tar.xzgst-plugins-good-1.4.3.tar.xznewer
gst-plugins-ugly-1.8.3.tar.xzgst-plugins-ugly-1.4.3.tar.xznewer
gst-plugins-bad-1.8.3.tar.xzgst-plugins-bad-1.4.3.tar.xznewer
glib-2.44.1.tar.xzglib-2.48.tar.xzcompatibility
libffi-3.0.13.tar.gzlibffi-3.2.1.tar.gzcompatibility
gobject-introspection-1.44.0.tar.xzgobject-introspection-1.48.0.tar.xzcompatibility
pango-1.36.8.tar.xzpango-1.38.1.tar.xz??
gdk-pixbuf-2.30.8.tar.xzgdk-pixbuf-2.34.0.tar.xz??
gtk+-3.16.7.tar.xzgtk+-3.20.3.tar.xznewer
pixman-0.32.6.tar.gzpixman-0.34.0.tar.gz??
gtk-mac-integration-2.0.5.tar.xzgtk-mac-integration-2.0.8.tar.xzcompatibility
openssl-1.0.2h.tar.gzopenssl-1.0.2f.tar.gznewer
pygobject-3.16.2.tar.xzpygobject-3.16.1.tar.xznewer
gmp-6.0.0.tar.bz2gmp-5.0.5.tar.bz2newer
Last edited 2 years ago by Smo (previous) (diff)

comment:18 Changed 2 years ago by Antoine Martin

Priority: majorcritical

comment:19 Changed 2 years ago by Antoine Martin

Saw the openssl update: r13851 + r13852, added cryptography 1.5.1 in r13853.

For reference, I wanted up update and found some failures:

*** the following modules were not built *** [119/119]
pycairo-python2.6 pygtk gtk-mac-integration-python meta-gtk-osx-python

Why does it even try to build pycairo-python2.6?? It also rebuilt a lot of things I didn't think needed doing.
Could this have been caused by a transient network failure to download our modules overrides? I've switched to https for the base moduleset URL since, and the problem seems to have gone away.

Last edited 2 years ago by Antoine Martin (previous) (diff)

comment:20 Changed 2 years ago by Antoine Martin

BTW, why are we still on 2.4.30? I see this bug has been fixed and looks relevant: improved fullscreen behaviouur: The attached patch improves the behaviour of fullscreen windows on El Capitan and perhaps other somewhat recent versions of OS X.

comment:21 Changed 2 years ago by Antoine Martin

Last edited 2 years ago by Antoine Martin (previous) (diff)

comment:22 Changed 2 years ago by Smo

openssl updated in r14112
disabled libyuv building for libvpx in r14113
build libvpx 1.6.0 again for ffmpeg in r14114

comment:23 Changed 2 years ago by Antoine Martin

  • r13900: x264 20160906
  • r13898: enable libx264 encoder for shadow servers support
  • r13853 + r13888: cryptography 1.5.1 then 1.5.2
  • r13887: enable more encoders so we can use enc_ffmpeg on osx (#1107)
  • r14087: pillow 3.4.1, ffmpeg 3.1.4

comment:24 Changed 2 years ago by Antoine Martin

Resolution: fixed
Status: newclosed

Milestone complete.

comment:25 Changed 4 months ago by Antoine Martin

For 2.4 see #1787

Note: See TracTickets for help on using tickets.