Monday, September 29, 2014

glibtop: This machine has 56 CPUs, 32 are being monitored

Fully building TF2 dedicated server was taking around 13 minutes, 10 seconds on this:

Intel(R) Core(TM) i7-4930K CPU @ 3.40GHz
mikesart@mikesart-petra:~$ grep -c model.name /proc/cpuinfo
12

I tend to muck with the lower level libraries which meant I was spending quite a bit of my day waiting for these builds to finish.

TF2 dedicated server has:
 - 1347 cpp files
 - 13 DSOs (with 13 .dbg files)
 - 12 static libraries

I looked at several things: ccache, precompiled headers, distcc, unity builds. We are using ccache fulltime, but when I switch between debug / release or add a new define here and there, it wasn't helping as much as I'd like. Distcc meant I'd have to manage several other machines, and the unity builds seemed like a whole bunch of work also. The precompiled headers gave a decent win (down to around 7 minutes), but (from what I could tell), the .gch file really wants to be where the main header file lives. We do debug/release/client/dedicated server builds all out of the same tree and this just won't work.

So I got one of these:

(2) Intel(R) Xeon(R) CPU E5-2697 v3 @ 2.60GHz
mikesart@mikesart-mint:~/valvesrc/ValveGames/staging/game$ grep -c model.name /proc/cpuinfo
56

The build time is now 3 minutes, 30 seconds.

I tried creating a ramdisk and putting the entire source tree and tmp directory there, and the build time was still 3m30s (compared to building from a Samsung SSD 850 PRO 512GB). I think a decent chunk of time right now is going towards linking - so I'm going to investigate the gold linker and see if I can get a bit more time back from that.

We are currently using gcc 4.6.1 crosstools w/ glibc-2.11.1. I'm going to investigate newer versions of gcc and clang when I get more time to play with this stuff. Have some bugs and features before I get to play more - hopefully turnaround will go a bit faster now though...