xpra icon
Bug tracker and wiki

Opened 9 months ago

Last modified 3 weeks ago

#2040 new task

buildbot switch to docker

Reported by: Antoine Martin Owned by: Smo
Priority: critical Milestone: 3.0
Component: packaging Version: 2.4.x
Keywords: Cc:

Description

Based on #2028, we can have a base image with the main build dependencies for each distro supported.
ie for centos7, here's a list of packages, not just xpra but also the build dependencies for some of the other packages we have to provide:

xorg-x11-server-devel sudo epel-release krb5-devel python-devel rpm-build pkgconfig which
#gstreamer plugin:
gstreamer1-devel gstreamer1-plugins-base-devel
#(py)gtkgl:
gtk2-devel libGLU-devel libXmu-devel libXt-devel pangox-compat-devel
ncurses-devel libXinerama-devel
pango-devel pygtk2-devel
#pycuda:
numpy python-distribute boost-devel
openssl-devel python-setuptools
#python-uinput:
libudev-devel
python-six
xmlto 
#python-pillow:
tkinter tk-devel libjpeg-devel lcms2-devel PyQt4

Not sure if all the dependencies should be in the base image, or only added when we build the package that requires them. (so many packages..)
Then we have to install the packages we build for the next step:

  • pangox-compat
  • gtkglext
  • pygtkglext

or:

  • lz4
  • python-lz4

So some form of scripting is likely to be needed to manage those dependencies.

Other distros may not need to build some packages (ie: python-pillow is in the Fedora repository).

This can then be used to build the master image for building xpra.

Attachments (1)

InstallUtils.pm (2.7 KB) - added by Antoine Martin 7 months ago.
missing perl module

Download all attachments as: .zip

Change History (17)

comment:1 Changed 9 months ago by Antoine Martin

Status: newassigned

See also el7 vs el7_6 in ticket:2028#comment:13

comment:2 Changed 8 months ago by Antoine Martin

Example of something difficult to handle: http://winswitch.org/trac/ticket/307#comment:6: we need to build lz4 on centos, except on centos 7.4 and later..

comment:3 Changed 8 months ago by Antoine Martin

Another problem: mock 1.4.2-1.el7 fails to build el6 packages and systemd-nspawn cannot change user with EL6 chroot.
I was getting getent initgroups failed with error code 1., worked around that by using nspawn as root and running su -l buildbot from there..

comment:4 Changed 8 months ago by Antoine Martin

And another: we need a recent version of nasm to build x264, r21315 adds a spec file so we can package the latest release. But on centos6, it does not build:

configure.ac:3: error: Autoconf version 2.69 or higher is required
...

comment:5 Changed 7 months ago by Antoine Martin

more:

  • pycuda 2014.1 is the last version that can be built against the version of numpy found in the centos6 repos, and the specfile for 2014.1 is found in our 0.15.x release!
  • cuda 7.5 is the last release suported by pycuda 2014 / centos6

And installing older versions of CUDA on fedora is not easy because we're missing attachment/ticket/2040/InstallUtils.pm... so run it like so:

INSTALLER_DIR=/opt/cuda-5.5-installer
INSTALL_DIR=/opt/cuda-5.5
sh cuda_5.5.22_linux_64.run -verbose -override -extract $INSTALLER_DIR
cd $INSTALLER_DIR
wget https://xpra.org/trac/raw-attachment/ticket/2040/InstallUtils.pm
PERL5LIB=. ./cuda-linux64-rel-5.5.22-16488124.run --prefix=$INSTALL_DIR -noprompt
Last edited 7 months ago by Antoine Martin (previous) (diff)

Changed 7 months ago by Antoine Martin

Attachment: InstallUtils.pm added

missing perl module

comment:6 Changed 7 months ago by Antoine Martin

Another issue similar to comment:2 is when the distro ships a broken package (ie: either out of date version or buggy packaging): in this case we want to ship a working package, but stop doing so if and when the distro fixes things.

Example: python-uinput broken in Fedora 24 to Fedora 29! (~3 years without a fix!)

comment:7 Changed 7 months ago by Antoine Martin

Out of the 40+ builds, one slipped through with the wrong ffmpeg dependency: Error: Package: ffmpeg-xpra-4.1-1.el7_6.x86_64 (Xpra)..

comment:8 Changed 7 months ago by Antoine Martin

new constraint: some versions may require a package (ie: versions before 2.5 require websockify and python-websocket-client - see #2121), but not after. Yet we need to keep the RPM specfile updated and update the repository from that branch - until the distribution ships it / updates that version.

comment:9 Changed 7 months ago by Antoine Martin

See also #1910

comment:10 Changed 6 months ago by Antoine Martin

Milestone: 2.53.0

Out of time. See also #1830

comment:11 Changed 6 months ago by Antoine Martin

Just when you think that things can't possibly get more interesting: Ubuntu ships two incompatible versions of Xorg for the same distro version, see ticket:2190#comment:7.

comment:12 Changed 5 months ago by Antoine Martin

Priority: majorcritical

comment:13 Changed 5 months ago by Antoine Martin

Another one that should have been spotted: #2240, xpra 1.0.13 builds for centos 6.9 are MIA.

comment:14 Changed 3 months ago by Antoine Martin

New one: rencode needs a newer cython version on centos.

comment:15 Changed 2 months ago by Antoine Martin

Owner: changed from Antoine Martin to Smo
Status: assignednew

Some dockerfiles and build scripts added by smo in r22880 + r22881.
@smo: can you add details here?

comment:16 Changed 3 weeks ago by Smo

Yes I've been working on a few instances of docker to build these. I have committed the stuff that I was working on for centos 7.6 which works mostly well.

I'm not too happy with the results as every time I build I have to go through all the packages that I need and build them in the correct order.

This has a few issues.

1) No caching of source tarballs which is quite wasteful
2) Rebuilding all rpms even when we don't need to

Should probably change this so we can call the shell script with a name of spec file to build that way we can do one offs.

That doesn't solve the problem completely but at least we can built each one 1 at a time.

The problem is that some things need to be installed in the docker container for building other things. Which is possible but do we want to install all these every time we build 1 rpm that would be costly for time.

Note: See TracTickets for help on using tickets.