I’ve recently seen GCC processes consuming as much as 1GB RAM for it’s resident set size (ie, what it actually consumes in memory). I also have an i7 processor, which means I’d like to run compiles at -j8 and higher (to compensate slightly for disk IO). I have 4GB of RAM. Now, at 8 processes, each consuming 512MB of RSS, I’m starting to push into swap quite considerably, resulting in much slower actual compiles, and I can just as well be using -j2 wasting a LOT of resources by it just sitting there and not getting utilized. A plan had to be made.
The default setup has a few problems for me. I run Gentoo. I compile many programs from scratch, and honestly, having to sit and stare at an unusable system that feels unresponsive and that has >50% of it’s CPU idle is sickening. top revealed that it was mostly gcc consuming in the range of 600-800MB of RAM _resident_ at -j5, not to mention the virtual size (portion of which is obviously swapped out). So the obvious culprit is GCC consuming too much ram. A quick search revealed that people using GCC under certain VPS systems were having major problems because the RAM was being reported as 4GB, but only a commit of 64 or 256MB was actually available. Specifically this post was particularly insightful:
ggc-min-expand
GCC uses a garbage collector to manage its own memory allocation. This parameter specifies the minimum percentage by which the garbage collector’s heap should be allowed to expand between collections. Tuning this may improve compilation speed; it has no effect on code generation.
The default is 30% + 70% * (RAM/1GB) with an upper bound of 100% when RAM >= 1GB. If ‘getrlimit’ is available, the notion of “RAM” is the smallest of actual RAM, RLIMIT_RSS, RLIMIT_DATA and RLIMIT_AS. If GCC is not able to calculate RAM on a particular platform, the lower bound of 30% is used. Setting this parameter and ‘ggc-min-heapsize’ to zero causes a full collection to occur at every opportunity. This is extremely slow, but can be useful for debugging.
and
ggc-min-heapsize
Minimum size of the garbage collector’s heap before it begins bothering to collect garbage. The first collection occurs after the heap expands by ‘ggc-min-expand’% beyond ‘ggc-min-heapsize’. Again, tuning this may improve compilation speed, and has no effect on code generation.
The default is RAM/8, with a lower bound of 4096 (four megabytes) and an upper bound of 131072 (128 megabytes). If ‘getrlimit’ is available, the notion of “RAM” is the smallest of actual RAM, RLIMIT_RSS, RLIMIT_DATA and RLIMIT_AS. If GCC is not able to calculate RAM on a particular platform, the lower bound is used. Setting this parameter very large effectively disables garbage collection. Setting this parameter and ‘ggc-min-expand’ to zero causes a full collection to occur at every opportunity.
What this means is that gcc will do garbage collection whenever it’s heapsize is greater than ‘gcc-min-heapsize’, and the heap has expanded by at least ‘gcc-min-expand’. Note that gcc-min-expand for systems with >1GB of RAM is 100% – thus not very often. Whilst the post above sets this value to 0 to force garbage collection as often as possible, I’m not quite on a 64MB system, so I’ve opted to set it to 20. The upper bound on the min-heapsize value should be OK for systems with 4GB of RAM but I’ve opted to lower this to 64MB in any case. I’ve done this by adding:
--param ggc-min-expand=20 --param ggc-min-heapsize=32768
To my CFLAGS variables in /etc/make.conf.
Proof that it works? Well, I’m able to merge qemu-kvm on my system now with -j8 instead of -j1, and am continuing on with other packages whilst typing this. Something I would not have been able to do an hour ago. This may slow down compiles ever so slightly, but on the other hand, the increases concurrency combined with the fact that it is not trashing probably results in a much faster overall compile.
The other mention was to drop -pipe, however, I find that this doesn’t actually help as the assembler being run is invoked as a sub-process of the GCC that just consumed (and continues to do so) all the RAM.
