Xpra: Ticket #1404: 2.0 moduleset updates

Now that we've dropped support for older build systems, we can update the modulesets with upstream changes again:

Tue, 10 Jan 2017 10:44:28 GMT - Antoine Martin: owner changed

I encountered some build problems on "yelp-xsl", "pango" and "gdk-pixbuf":

dyld: Library not loaded: /Users/osx/gtk/inst/lib/libintl.8.dylib
  Referenced from: /Users/osx/gtk/inst/bin/xgettext
  Reason: image not found

Which clearly we're not the first ones to hit: dyld: Library not loaded: /usr/local/libodep/lib/libintl.8.dylib (and many more hits)

Fixed by rebuilding libcroco. Hit a similar thing later on "gtk+", solved by rebuilding "atk".

Then it gets wedged with gobject-introspection:

*** Configuring gobject-introspection *** [27/111]
checking for flex... flex
checking lex output file root... configure: error: cannot find output from flex; giving up

Fixed by re-installing flex. ("make install" from the source dir - odd that "jhbuild buildone flex" didn't work here, oh well)

Maybe I should have just ran everything from the top ("jhbuild bootstrap").

@smo: any changes to the list of safe/unsafe candidates for applying to the 1.x branch?

Tue, 10 Jan 2017 12:16:19 GMT - Antoine Martin:

Building from scratch ("jhbuild bootstrap" then "jhbuild build") works fine, just takes longer.

But trying to build the xpra package then fails with:

calling 'gtk-mac-bundler Xpra.bundle' in /Users/osx/Xpra-svn/trunk/osx
Traceback (most recent call last):
  File "/Users/osx/.local/bin/gtk-mac-bundler", line 6, in <module>
  File "/Users/osx/gtk/source/gtk-mac-bundler-0.7.3/bundler/main.py", line 18, in main
  File "/Users/osx/gtk/source/gtk-mac-bundler-0.7.3/bundler/bundler.py", line 611, in run
  File "/Users/osx/gtk/source/gtk-mac-bundler-0.7.3/bundler/bundler.py", line 207, in copy_binaries
    dest = self.copy_path(path)
  File "/Users/osx/gtk/source/gtk-mac-bundler-0.7.3/bundler/bundler.py", line 224, in copy_path
    source = self.project.evaluate_path(Path.source)
  File "/Users/osx/gtk/source/gtk-mac-bundler-0.7.3/bundler/project.py", line 288, in evaluate_path
    path = utils.evaluate_pkgconfig_variables(path)
  File "/Users/osx/gtk/source/gtk-mac-bundler-0.7.3/bundler/utils.py", line 27, in evaluate_pkgconfig_variables
    raise Exception("pkg-config variable '%s %s' is undefined" % (key, module))
Exception: pkg-config variable 'pango_module_version pango' is undefined
ERROR: gtk-mac-bundler failed

Following the advice from Make pango modules related code optional to fix build with pango 1.38, r14746 removes pango from the bundle file. We still need the patch to be applied to gtk-mac-bundler (it applies cleanly - skipping the examples), this needs to be added to the moduleset.

Tue, 10 Jan 2017 23:31:21 GMT - Smo:

r14752 adds patch for gtk-mac-bundler without examples r14753 upgrades to newer version of gtk-mac-bundler and applies new patch also old patch is not necessary anymore

Wed, 11 Jan 2017 13:06:21 GMT - Antoine Martin:

One of those updates, likely r14753, broke our packaging scripts, r14760 fixes that.

Sat, 14 Jan 2017 06:05:21 GMT - Smo:

r14782 add patch for pygtkglext r14785 renames patch r14786 uses the patch in the moduleset r14787 enabled dependency for pygobject3 which should build gtk3 properly now.

Sat, 14 Jan 2017 06:06:29 GMT - Smo: owner changed

Sat, 14 Jan 2017 06:24:36 GMT - Antoine Martin: owner changed

Works for me too, closed ticket:840#comment:37.

Can you comment on which libraries are safe to apply to 1.0? (see ticket description). My guess: gettext and autoconf only? gdk-pixbuf?

Tue, 24 Jan 2017 19:58:14 GMT - Smo:

Sorry for the delay.

This is what I think

shoud be safe gettext, autoconf, libxml2, gettext

maybe pango, pixman, gdk-pixbuf

not safe gtk-mac-integration, libffi

Thu, 26 Jan 2017 17:45:45 GMT - Antoine Martin: status changed; resolution set


Fixed by: jhbuild buildone -f gtk+-3.0

Mon, 30 Jan 2017 14:51:59 GMT - Antoine Martin:

Was wrong about the pixman update, reverted in r14904.

Tue, 31 Jan 2017 03:43:34 GMT - Antoine Martin:

r14910 also updates the bundler and properly fixes the issues with the new pango version

Wed, 01 Feb 2017 11:30:10 GMT - Antoine Martin:

For the record: my VM got hosed by a system update.. (wouldn't boot no matter what bootloader or boot CD I tried to use..) so I reverted back to an earlier snapshot.. and jhbuild just fell over itself too many times trying to update + ugprade (libintl.8.dylib and more) so I decided to start again from scratch - a bad idea (tm).

That didn't work until I completely wiped the gtk directory. And even then it failed to build jpeg because nasm was the default system one (too old). Got a few SSL certificate problems along the way too: cairographics.org, gstreamer.freedesktop.org, etc.. PITA python-cryptography needs openssl, but somehow that wasn't built by then.. and then I found out that the current openssl version in the moduleset had been removed from the download location we use... so I bumped openssl in r14944. (we need permanent urls for everything instead)

I'll then forced a full rebuild (jhbuild build -f) at the end to ensure that everything is linked against the library version we ship. (ie: our newer openssl for python, etc)

Tue, 14 Feb 2017 15:05:30 GMT - Antoine Martin:

More in r15055:

Wed, 22 Feb 2017 13:20:36 GMT - Antoine Martin:

Lots more updates in r15138 (copying changelog message):

Some of those should be backported sooner rather than later: sshpass, opus - and maybe also numpy.

Wed, 22 Feb 2017 14:08:23 GMT - Antoine Martin: status changed; resolution deleted

Two of those update are problematic:

I'd rather fix this rather than downgrade again, but can't spend too much time on this either.

Wed, 22 Feb 2017 14:08:32 GMT - Antoine Martin: owner, status changed

Wed, 22 Feb 2017 21:06:51 GMT - Smo:

r15143 changes setup tools to 33.1.1. There seems to be an issue with 34 which will take some more work to fix.

Fri, 24 Feb 2017 06:23:23 GMT - Antoine Martin: owner changed

Then it also seems to get confused about which platform it is running on and blows up some other win32 modules we don't care about. (removing them from the source tree helps, temporarily at least) Downgrading to 0.11 still shows problems: it fails during its qt5 import recipes (missing 'sip' dependency)... but we don't even use qt, what are you doing there py2app? So r15165 downgrades py2app from version 0.12 to 0.10, also adds a warning message in the moduleset file.

@smo: just a FYI, things should be packaged OK again. If you had rebuilt python-cryptography with a newer cffi, you may need to run

jhbuild buildone -f python-cryptography

Sat, 25 Feb 2017 20:18:04 GMT - Smo:

Everything builds fine. Even forced rebuild of cffi and python-cryptography but i'm seeing this unit test fail now.

ERROR: test_perf (__main__.TestCrypto)
Traceback (most recent call last):
  File "/Users/build/xpra-trunk/src/unittests/unit/net/crypto_test.py", line 31, in setUp
  File "/Users/build/gtk/inst/lib/python2.7/site-packages/xpra/net/pycryptography_backend.py", line 52, in init
    backend = default_backend()
  File "/Users/build/gtk/inst/lib/python2.7/site-packages/cryptography/hazmat/backends/__init__.py", line 71, in default_backend
    _default_backend = MultiBackend(_available_backends())
  File "/Users/build/gtk/inst/lib/python2.7/site-packages/cryptography/hazmat/backends/multibackend.py", line 36, in __init__
    "Multibackend cannot be initialized with no backends. If you "
ValueError: Multibackend cannot be initialized with no backends. If you are seeing this error when trying to use default_backend() please try uninstalling and reinstalling cryptography.
Ran 2 tests in 5.384s

Maybe I need to do something more to get this to work properly again?

Sun, 26 Feb 2017 06:15:07 GMT - Antoine Martin:

The error in comment:19 is definitely the error that made me downgrade cffi in r15168. You may need to ensure you don't have an old version laying around (remove all and re-install it). Then you may need to zap the python-cryptography source directory too. If that still doesn't help, try zapping all the other downgrades (setuptools, etc).

Tue, 28 Feb 2017 22:52:01 GMT - Smo:

After deleting all directories having to do with setuptools, cffi and python-cryptography I was able to rebuild them then the unittests pass.

Must have been some issue with the downgrades.

I should note I also deleted stuff from


cffi*, cryptography* and setuptools*

Tue, 28 Feb 2017 22:53:36 GMT - Smo:

Even on newer versions of osx we still need the package tools to make the pkg.

Making Xpra-x86_64-2.0-r15188.pkg
229163 blocks
4 blocks
mkbom: invalid option -- u
Usage: mkbom [-h] [-s] directory bom
       mkbom [-s] -i filelist bom
PKG Signing skipped (no keyname)
33      ./image/Xpra-x86_64-2.0-r15188.pkg
Done PKG

I built them with this

jhbuild build meta-osx-xpra-pkgtools

Tue, 28 Feb 2017 23:37:18 GMT - Smo:

Ran into this issue and it appears it is a problem with openssl 1.1. This might take some work to fix. Not sure at the moment. Going to check if we really need this on newer versions of osx.

checking for openssl/evp.h... yes
checking for OpenSSL_add_all_ciphers in -lcrypto... no
configure: error: Cannot build without libcrypto (OpenSSL)
*** Error during phase configure of xar: ########## Error running ./configure --prefix /Users/build/gtk/inst    *** [1/4]
*** module meta-osx-xpra-pkgtools not built due to non buildable xar *** [4/4]

Tue, 28 Feb 2017 23:44:47 GMT - Smo:

Making Xpra-x86_64-2.0-r15188.pkg
229148 blocks
4 blocks
PKG Signing skipped (no keyname)
34      ./image/Xpra-x86_64-2.0-r15188.pkg
Done PKG

r15189 comments out xar and adds a note

Wed, 01 Mar 2017 11:17:45 GMT - Antoine Martin:

The pygobject flags patch had gone MIA: I have removed it in the r15149 cleanup because it was not referenced anywhere since r13713, but that was a mistake: we need it to avoid the warnings (see ticket:1444#comment:2). Re-added in r15191. I wasn't just seeing things then.

Thu, 09 Mar 2017 05:27:25 GMT - Antoine Martin:

We should do something about the git checkouts: either use a specific git id, or take a snapshot (version it ourselves). ie: libepoxy and others. And since we also want to be able to apply patches.. maybe snapshots is the way to go?

Fri, 10 Mar 2017 06:37:51 GMT - Smo:

r15254 adds grep and pcre which is needed for grep

Added these because I needed them to rule out issues with cmake and configuring libyuv. The system grep doesn't support the '-P' option so it errors when running cmake for libyuv.

This didn't fix the issue with libyuv but I figure I should leave them in there in case we need them later.

As for the stuff from git goes if we need to apply any patches with the jhbuild system that doesn't work for us. The only way to do this is with tarball snapshots.

Mon, 27 Mar 2017 23:59:37 GMT - Smo:

I was updating my builds today and for some reason the dependency for setuptools_scm is being ignored for some reason I see that it was added for lz4 which is why I needed it.

It looks right in xpra.modules but it isn't doing what we want.

Tue, 28 Mar 2017 00:25:36 GMT - Smo:

Added libyuv now so we can build it within the moduleset and added it to the meta module so it should automatically get built.

details in #1280

Sat, 15 Apr 2017 08:02:39 GMT - Antoine Martin:

Follow up for 2.1: #1501

Please close unless something still needs doing?

Sun, 30 Apr 2017 20:00:51 GMT - Smo: status changed; resolution set

Sat, 23 Jan 2021 05:23:21 GMT - migration script:

this ticket has been moved to: https://github.com/Xpra-org/xpra/issues/1404