gcc build time benchmarks on huge-ass dedis

willie

I'm interested in finding out how long it takes to build GCC7 on various big dedis, let's say 6+ cores and 10K+ passmark. It's about 30 minutes on my i7-3770 (4 cores, 9k passmark) but I'd like to see the real monsters, E5-2697, Epyc 32-core and all that. I can supply a test script that runs under Debian and maybe could generate scripts for a few other popular distros, or if you're willing to give me temporary ssh access (non-root, can firewall almost everything) I could run it myself and post the results.

Test duration including setup: under 1 hour

Disk space needed: on order of 1GB

Server should be unloaded during test.

Budget: $0 but I can possibly help you out with some test or dev issue of your own.

Let me know if you want to try this.

WSS

I'd help ya out, we willie, but I have no dedis left, and my 8 core months-to-end-of-contract VPS is burning away at golum precious razorcoins.

eva2000

I no longer build GCC myself on CentOS as SCL devtoolset yum repos are available. It's how my Centmin Mod LEMP stack can install Nginx and PHP-FPM on CentOS 6/7 with GCC 4.4,4.8, 5.3.1, 6.3.1. 7.2.1 and Clang 3.4 and 4.0.1. Only need to compile for Clang 5.0.1 right now https://community.centminmod.com/threads/testing-gcc-7-1-7-2-clang-4-0-1-scl-yum-packages.12815/

Not that familiar with Debian, but would they have a similar 3rd party repos for such GCC versions as well ?

My OVH MC-64-OC 7700K @ 4.7-5.0Ghz expires in next ~3hrs so can give an Debian OS reload if you can link me to the scripts/instructions to compile and which version of Debian etc

willie

Eva, thanks! Please try the following:

sudo apt-get update
sudo apt-get install build-essential libgmp-dev libmpfr-dev libmpc-dev libc6-dev-i386

wget http://www.netgull.com/gcc/releases/gcc-7.2.0/gcc-7.2.0.tar.xz 
tar xf gcc-7.2.0.tar.xz
# do NOT cd into the gcc-7.2.0 directory here

mkdir test
cd test
../gcc-7.2.0/configure
time make -j8

Thanks also to exception0x876, who lent me a 2xE5-2670 16-core VM for the past hour. Build time for the above with -j20 (20 threads, slightly higher than the physical core count) was:

real 25m43.241s
user 213m16.832s
sys 6m17.300s

Note that's just over 800% cpu utilization, about half what I'd hope for on a 16 core machine. I saw a lot of the build time was spent re-running configuration scripts or linking .o's. At those times, the make was running essentially single threaded. There was very little time when all 16 cores were in use, and I never saw the 15 minute load average get above 8. The total time beats my i7-3770 but not by a huge amount. Eva2000, your overclocked game machine will probably do better than the E5 because of its higher single thread performance.

I have some other tests that could get better parallelism, if anyone wants to try them sometime (not now).

I'd been wanting to do this test for a while, and it was interesting!

telephone

eva2000 said: Not that familiar with Debian, but would they have a similar 3rd party repos for such GCC versions as well ?

gcc-7 is in buster (testing) and sid on Debian.

But I don't think that's the purpose of this thread

eva2000

@willie get an error on debian 9.3 64bit

checking for default BUILD_CONFIG... bootstrap-debug
checking for --enable-vtable-verify... no
/usr/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-linux-gnu/6/libgcc.a when searching for -lgcc
/usr/bin/ld: cannot find -lgcc
/usr/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-linux-gnu/6/libgcc_s.so.1 when searching for libgcc_s.so.1
/usr/bin/ld: cannot find libgcc_s.so.1
/usr/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-linux-gnu/6/libgcc.a when searching for -lgcc
/usr/bin/ld: cannot find -lgcc
collect2: error: ld returned 1 exit status
configure: error: I suspect your system does not have 32-bit development libraries (libc and headers). If you have them, rerun configure with --enable-multilib. If you do not have them, and want to build a 64-bit-only compiler, rerun configure with --disable-multilib.

from

apt-get install build-essential libgmp-dev libmpfr-dev libmpc-dev libc6-dev-i386
Reading package lists... Done
Building dependency tree       
Reading state information... Done
build-essential is already the newest version (12.3).
libc6-dev-i386 is already the newest version (2.24-11+deb9u1).
libgmp-dev is already the newest version (2:6.1.2+dfsg-1).
libmpc-dev is already the newest version (1.0.3-1+b2).
libmpfr-dev is already the newest version (3.1.5-1).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

GCC

gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/6/lto-wrapper
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Debian 6.3.0-18' --with-bugurl=file:///usr/share/doc/gcc-6/README.Bugs --enable-languages=c,ada,c++,java,go,d,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-6 --program-prefix=x86_64-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-libmpx --enable-plugin --enable-default-pie --with-system-zlib --disable-browser-plugin --enable-java-awt=gtk --enable-gtk-cairo --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-6-amd64/jre --enable-java-home --with-jvm-root-dir=/usr/lib/jvm/java-1.5.0-gcj-6-amd64 --with-jvm-jar-dir=/usr/lib/jvm-exports/java-1.5.0-gcj-6-amd64 --with-arch-directory=amd64 --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --with-target-system-zlib --enable-objc-gc=auto --enable-multiarch --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 6.3.0 20170516 (Debian 6.3.0-18) 

seems this works with

../gcc-7.2.0/configure --enable-multilib

but make fails

Makefile:23803: recipe for target 'stage1-bubble' failed
make[1]: *** [stage1-bubble] Error 2
make[1]: Leaving directory '/home/test'
Makefile:933: recipe for target 'all' failed
make: *** [all] Error 2

willie

Hmm, thanks. Can you try with --disable-multilib? It's odd, libc6-dev-i386 is supposed to supply the 32-bit lib. I wonder if there's some regression in Debian 9.3.

eva2000

willie said: Can you try with --disable-multilib?

it's compiling with that flag so will see

eva2000

gcc 7.2.0 built on i7 7700K @ 4.7-5.0Ghz with 2x450GB NVMe raid 1

real    19m31.112s
user    116m12.640s
sys     1m46.704s

willie

Thanks!

xyz

Yeah gets blocked by config scripts etc a lot of the time.

Not a server, but a machine I have here which is a i7 7820X (8C/16T 3.6-4.5GHz), 32GB RAM, Samsung 960 Pro 512GB, Ubuntu 17.10 amd64 (so GCC 7.2.0), make -j20:

real    17m45.548s
user    134m43.578s
sys     4m12.850s

The compilation software does kinda matter, and really, a proper test needs to use the same version of everything, but anyway...

EveNeko

12170.12user 416.28system 22:47.02elapsed 920%CPU (0avgtext+0avgdata 649044maxresident)k
143080inputs+16899952outputs (2766major+192128013minor)pagefaults 0swaps

Build on Dual Xeon E5-2680v4 (28C56T, 2.4G/3.3G Max) with 256GB RAM DDR4 2400MT/s
Build with command /usr/bin/time make -j64

willie

Thanks xyz and EveNeko. It sounds like clock speed is still king for this test.

netpioneer

E3-1231 (v3 @ 3.40GHz - 3.80GHz)

make -j1

real 131m22.912s
user 121m47.648s
sys 2m9.024s

make -j2

real 69m28.986s
user 124m0.876s
sys 2m17.944s

make -j3

real 49m33.381s
user 128m48.904s
sys 2m26.100s

make -j4

real 40m5.422s
user 133m25.224s
sys 2m27.528s

make -j8

real 35m16.522s
user 207m14.400s
sys 3m35.968s

eva2000

willie said: It sounds like clock speed is still king for this test.

yes it is. A i7 8700K 6C/12T processor would be ideal