xpra icon
Bug tracker and wiki

Opened 4 years ago

Closed 3 years ago

Last modified 8 weeks ago

#727 closed defect (fixed)

building with clang - measure difference in performance

Reported by: Antoine Martin Owned by: Nick Centanni
Priority: trivial Milestone: 0.15
Component: platforms Version: trunk
Keywords: Cc:

Description (last modified by Antoine Martin)

http://clang.llvm.org/ is an alternative to GCC.

It is possible to build the cython modules with the script attached (based on this filter). To use it:

CC=clang-wrapper setup.py build

It would be interesting to run the automated tests to compare and see if we get any difference in performance. Probably should be tested with something that tests lots of instantiations rather than x264 encoding as this will still use the same binary code.
That said, it would be worth trying to build x264 and other libs with clang too...

See: Clang vs GCC - which produces better binaries? and LLVM Clang vs. GCC On Fedora 21 With A High-End Xeon

Attachments (6)

clang-wrapper (271 bytes) - added by Antoine Martin 4 years ago.
wrapper script which strips arguments that clang chokes on
gcc-vs-clang.csv (54.8 KB) - added by Antoine Martin 3 years ago.
test data output
clang-vs-gcc-mmap-pixelspersecond.png (15.8 KB) - added by Antoine Martin 3 years ago.
one of the most telling graphs I generated
perf_charts.tar.gz (425.6 KB) - added by Nick Centanni 3 years ago.
Chart generator (beta) for examining performance results.
clang-wrapper.2 (285 bytes) - added by Antoine Martin 3 years ago.
updated wrapper - works with Fedora 23
clang-wrapper.3 (347 bytes) - added by Antoine Martin 8 weeks ago.
updated wrapper for clang-6.0

Download all attachments as: .zip

Change History (24)

Changed 4 years ago by Antoine Martin

Attachment: clang-wrapper added

wrapper script which strips arguments that clang chokes on

comment:1 Changed 4 years ago by Antoine Martin

Description: modified (diff)

comment:2 Changed 4 years ago by Antoine Martin

Description: modified (diff)

comment:3 Changed 4 years ago by Antoine Martin

Description: modified (diff)

Changed 3 years ago by Antoine Martin

Attachment: gcc-vs-clang.csv added

test data output

Changed 3 years ago by Antoine Martin

one of the most telling graphs I generated

comment:4 Changed 3 years ago by Antoine Martin

Resolution: wontfix
Status: newclosed

As can be seen on this graph:
one of the most telling graphs I generated

gcc beats clang in most cases, it looks like clang adds an overhead to even the most basic case (mmap).
CPU utilization is about the same, but gcc encodes more pixels/s, sends more packets, etc.

I did find one case where clang does better than gcc: it seems that clang compresses png better, at least for the "glxgears" and "glxspheres" tests. No idea why as we don't rebuild the libpng library with clang.. it should end up being more or less the same. Odd!

We may want to revisit this later, once clang matures a bit more and once projects start making more of an effort to optimize for it. At present, x264 is always faster with gcc it seems.

comment:5 Changed 3 years ago by Nick Centanni

I've re-run the clang vs gcc tests on a more massive scale -- this is 40 seconds per test, the suite run 9 times for each variable, and each cell averaged across the 9 reps.

I've attached the results as an archive. After extracting the archive, view the HTML file.

Alternately look at how the python script is used, and change which combinations of metrics, encodings, and apps are graphed.

After making changes run the python script with no arguments, and a new HTML file will be written.

Last edited 3 years ago by Nick Centanni (previous) (diff)

comment:6 Changed 3 years ago by Nick Centanni

I've reopened the tickets because I think the results should be examined to make sure the conclusions we reached last time are still reflected in the results.

comment:7 Changed 3 years ago by Nick Centanni

Resolution: wontfix
Status: closedreopened

comment:8 Changed 3 years ago by Nick Centanni

Owner: changed from Smo to Nick Centanni
Status: reopenednew

comment:9 Changed 3 years ago by Nick Centanni

Owner: changed from Nick Centanni to Antoine Martin

Changed 3 years ago by Nick Centanni

Attachment: perf_charts.tar.gz added

Chart generator (beta) for examining performance results.

comment:10 Changed 3 years ago by Antoine Martin

Owner: changed from Antoine Martin to Nick Centanni

Thanks, that chart generator is great - much easier to use than sofastats which is a PITA to install.
Can you submit it or commit it to svn? And maybe edit the wiki to refer to it?

As for clang vs gcc: the gcc win is confirmed.

comment:11 Changed 3 years ago by Nick Centanni

I've added a section to the wiki at https://www.xpra.org/trac/wiki/Testing documenting the changes to the performance script, as well as instructions generating charts from the data using test_measure_perf_charts.py.

comment:12 Changed 3 years ago by Nick Centanni

Resolution: fixed
Status: newclosed

Changed 3 years ago by Antoine Martin

Attachment: clang-wrapper.2 added

updated wrapper - works with Fedora 23

comment:13 Changed 3 years ago by Antoine Martin

comment:14 Changed 3 years ago by Nick Centanni

Or maybe experiment with the -falign-* flags. The discussion in that second link seemed to suggest that setting alignment flags produces the equivalent speed improvement that you get when optimizing for size.

comment:15 Changed 2 years ago by Antoine Martin

Clang getting more competitive apparently: Intel Xeon Skylake Compilers: Clang Showing Strong Performance Against GCC. No x264 data, only lame.
Not quite there yet IMO.

comment:16 Changed 11 months ago by Antoine Martin

On Fedora 26 (clang 4), the clang wrapper script is no longer needed and with Cython 0.26 we can now un-disable more warnings: r16464.

comment:17 Changed 11 months ago by Antoine Martin

This is just one test, and with a specific CPU, but it seems that clang could give us better FPS with x264 by about 5%:
Ryzen Compiler Performance: Clang 4/5 vs. GCC 6/7/8 Benchmarks.

Changed 8 weeks ago by Antoine Martin

Attachment: clang-wrapper.3 added

updated wrapper for clang-6.0

comment:18 Changed 8 weeks ago by Antoine Martin

Building with clang-6.0 (ie: Fedora 28) hits this error:

clang-6.0: error: unknown argument: '-mcet'
clang-6.0: error: unknown argument: '-fcf-protection'

To fix this, use r19079 and the updated wrapper.

Note: See TracTickets for help on using tickets.