build minimal x264 / libav on win32 rather than fat binaries

[Antoine Martin]

Split from #103, see comment 10 - just for the record.

win32 native build from within a mingw shell:

wget http://www.tortall.net/projects/yasm/releases/yasm-1.2.0.tar.gz
tar -zxvf yasm-1.2.0.tar.gz
cd yasm-1.2.0
./configure
make && make install

wget ftp://ftp.videolan.org/pub/x264/snapshots/x264-snapshot-20130205-2245-stable.tar.bz2
tar -jxf x264-snapshot-20130205-2245-stable.tar.bz2
cd x264-snapshot-20130205-2245-stable
./configure --enable-shared --enable-static --prefix=/c/MinGW
make && make install

wget http://libav.org/releases/libav-9.2.tar.gz
tar -zxvf libav-9.2.tar.gz
cd libav-9.2
./configure --enable-runtime-cpudetect --enable-static --enable-shared --enable-gpl \
    --enable-memalign-hack --disable-avdevice \
    --disable-dxva2 --disable-everything --enable-decoder=h264 \
    --disable-decoders --enable-decoder=h264 --enable-decoder=h263
    --prefix=/c/libav-win32/win32/usr

Net result:

• libav-win32 binary directory from libav releases is 120MB
• our trimmed one is 29MB

(obviously this is a lot less significant once py2exe picks only the bits we really need - but still)

Notes:

• the command lines in comment:10 needed cleaning up (see edit diffs)
• also were missing --enable-x264
• from googling, I'm not sure if the cross-compile would have the same threading support or not. (and "win32threads" is faster)

[Antoine Martin]

works-for-me, please re-open if you find problems

[Antoine Martin]

And here is how one gets the libx264.lib needed for linking against (see #281). This is based on the DLL instructions found ​here(vlc) and ​here(mingw)):

dumpbin /exports libx264-129.dll > libx264.def
#edit def file..
lib /def:libx264.def /out:libx264.lib /machine:x86

See the example def file attached to this ticket. (you can use this regular expression to trim the line prefix: .*[0-9A-F]*)

Note: the build uses the canonical DLL name libx264.dll so after building you will need:

COPY libx264-NNNN.dll libx264.dll

(where NNNN is the version number)

[Antoine Martin]

Note: latest libav is now 9.3 and there may well be a newer one by the time you read this.
Always use the latest.
Same for the x264 snapshots.

See also r2773

[Antoine Martin]

example edited def file

[Smo]

Found some hints from here

​http://doom10.org/index.php?topic=26.0

On how to generate the libx264.def file and the resulting .lib

When building x264

./configure --enable-shared --enable-static --prefix=/c/MinGW \
--extra-ldflags=-Wl,--output-def=libx264.def

After you build x264 and then copy the directory to c:\x264 for building with xpra

Rename libx264-120.dll for example to libx264.dll

Then run

LIB /DEF:libx264.def

to generate the .lib

[ahuillet]

For ffmpeg:

wget https://ffmpeg.org/releases/ffmpeg-1.2.1.tar.bz2
tar jxvf ffmpeg-1.2.1.tar.bz2
cd ffmpeg-1.2.1
./configure --enable-runtime-cpudetect --enable-static --enable-shared --enable-gpl \
    --enable-memalign-hack --disable-avdevice \
    --disable-dxva2 --disable-everything\
    --disable-decoders --enable-decoder=h264 \
    --disable-avfilter \
    --prefix=/c/libav-win32/win32/usr

[ahuillet]

Avfilter doesn't build on Windows with ffmpeg1.2.1. We don't need it, so disable it.

[ahuillet]

0.9.2 builds just fine, starting with 1.0.7 avfilter doesn't build with the following error messages (here extracted from 1.2.1):

LD	libavfilter/avfilter-3.dll
libavfilter/avcodec.o: In function `avfilter_copy_frame_props':
e:\ffmpeg-1.2.1/libavfilter/avcodec.c:33: undefined reference to `av_frame_get_pkt_pos'
e:\ffmpeg-1.2.1/libavfilter/avcodec.c:37: undefined reference to `av_frame_get_metadata'
libavfilter/avcodec.o: In function `avfilter_get_audio_buffer_ref_from_frame':
e:\ffmpeg-1.2.1/libavfilter/avcodec.c:96: undefined reference to `av_frame_get_channels'
e:\ffmpeg-1.2.1/libavfilter/avcodec.c:97: undefined reference to `av_frame_get_channel_layout'
e:\ffmpeg-1.2.1/libavfilter/avcodec.c:99: undefined reference to `av_frame_get_channels'
libavfilter/avcodec.o: In function `avfilter_copy_buf_props':
e:\ffmpeg-1.2.1/libavfilter/avcodec.c:144: undefined reference to `av_frame_set_pkt_pos'
e:\ffmpeg-1.2.1/libavfilter/avcodec.c:171: undefined reference to `av_frame_set_sample_rate'
e:\ffmpeg-1.2.1/libavfilter/avcodec.c:172: undefined reference to `av_frame_set_channel_layout'
e:\ffmpeg-1.2.1/libavfilter/avcodec.c:173: undefined reference to `av_frame_set_channels'
libavfilter/avfiltergraph.o: In function `pick_format':
e:\ffmpeg-1.2.1/libavfilter/avfiltergraph.c:386: undefined reference to `avcodec_find_best_pix_fmt_of_2'
libavfilter/lavfutils.o: In function `ff_load_image':
e:\ffmpeg-1.2.1/libavfilter/lavfutils.c:36: undefined reference to `av_register_all'
e:\ffmpeg-1.2.1/libavfilter/lavfutils.c:38: undefined reference to `av_find_input_format'
e:\ffmpeg-1.2.1/libavfilter/lavfutils.c:39: undefined reference to `avformat_open_input'
e:\ffmpeg-1.2.1/libavfilter/lavfutils.c:46: undefined reference to `avcodec_find_decoder'
e:\ffmpeg-1.2.1/libavfilter/lavfutils.c:53: undefined reference to `avcodec_open2'
e:\ffmpeg-1.2.1/libavfilter/lavfutils.c:58: undefined reference to `avcodec_alloc_frame'
e:\ffmpeg-1.2.1/libavfilter/lavfutils.c:64: undefined reference to `av_read_frame'
e:\ffmpeg-1.2.1/libavfilter/lavfutils.c:70: undefined reference to `avcodec_decode_video2'
e:\ffmpeg-1.2.1/libavfilter/lavfutils.c:88: undefined reference to `avcodec_close'
e:\ffmpeg-1.2.1/libavfilter/lavfutils.c:89: undefined reference to `avformat_close_input'
e:\ffmpeg-1.2.1/libavfilter/lavfutils.c:88: undefined reference to `avcodec_close'
e:\ffmpeg-1.2.1/libavfilter/lavfutils.c:89: undefined reference to `avformat_close_input'
e:\ffmpeg-1.2.1/libavfilter/lavfutils.c:88: undefined reference to `avcodec_close'
e:\ffmpeg-1.2.1/libavfilter/lavfutils.c:89: undefined reference to `avformat_close_input'
libavfilter/lswsutils.o: In function `ff_scale_image':
e:\ffmpeg-1.2.1/libavfilter/lswsutils.c:29: undefined reference to `sws_getContext'
e:\ffmpeg-1.2.1/libavfilter/lswsutils.c:45: undefined reference to `sws_scale'
e:\ffmpeg-1.2.1/libavfilter/lswsutils.c:48: undefined reference to `sws_freeContext'
collect2.exe: error: ld returned 1 exit status
make: *** [libavfilter/avfilter-3.dll] Error 1

Build is fine with the following switches:

./configure --enable-runtime-cpudetect --enable-static --enable-shared \
    --enable-gpl    --prefix=/c/libav-win32/win32/usr --enable-memalign-hack \
    --disable-avdevice --disable-dxva2 --disable-decoders

[ahuillet]

Found the problem:

--disable-everything

should not be used because it breaks the build.

[ahuillet]

What I used to build ffmpeg:

--enable-swscale --cpu=i686 --enable-runtime-cpudetect --enable-static \
--enable-shared --enable-gpl --enable-memalign-hack --disable-avdevice \
--disable-decoders --disable-dxva2 --disable-encoders --enable-decoder=h264

[Antoine Martin]

Just upgraded to 1.2.2 without problems.

---

Note: in order to install it in the location our build scripts expect (as of 0.10.x), add:

--prefix=/c/ffmpeg-win32-bin

[Antoine Martin]

Should we be using

--enable-w32threads

or even pthread on win32?

Questions remain about thread safety:

• ​Thread Safety of LibAv/FFMpeg?
• ​multithreaded problems
• ​ffmpeg utility functions: int av_lockmgr_register(int(*)(void **mutex, enum AVLockOp op) cb)}

[Antoine Martin]

AFAICT, from ​this response, we only need locking around avcodec_open2 and since we always create the codec from the decoding thread, we're OK.

[Antoine Martin]

recap (tested with ffmpeg 1.2.4 and x264 stable 20131205):

• for building x264, see comment:4 (I think you can ignore the prefix since we don't do make install - it would be better to install with the right prefix rather than copying everything including the source... oh well, nvm)
• we don't need the avcodec lock, see comment:11
• w32threads is enabled automatically (at least for ffmpeg)
• we now support decoding vpx via avcodec (see r4533), so include it
• the build instructions from comment:5 onwards sort of work, but they are far from minimal (which is what this ticket is about) - here is a proper minimal build:

  ./configure --enable-swscale --cpu=i686 --enable-runtime-cpudetect \
      --enable-static --enable-shared --enable-gpl --enable-memalign-hack \
      --disable-avdevice --disable-decoders --disable-dxva2 --disable-encoders \
      --enable-decoder=h264 --enable-decoder=vp8 --enable-libvpx \
      --enable-libx264 --disable-devices --disable-muxers --disable-demuxers \
      --disable-postproc --disable-avfilter  --disable-filters --disable-protocols \
      --disable-bsfs --disable-parsers  --prefix=/c/ffmpeg-win32-bin
  

The same command line can also be used with ffmpeg 2.x (#415), and to prepare for vp9 (#464), we can add:

    --enable-decoder=vp9

Updated build instructions including vp9 and h265 can be found in ticket:445#comment:4

[migration script]

this ticket has been moved to: ​https://github.com/Xpra-org/xpra/issues/270