Using ccache for Fun and Profit by Peter Steinberger
Our PSPDFKit project is over 600,000 lines of code and growing. Although we were committed to writing clean and efficient code, the project was large and had many boundary cases that required special attention. On the PSPDFKit 5 for iOS project, compile time in particular became a headache: every compile was slow.
Our android SDK has the same problem, and I started using ccache a few months ago when our android lead introduced ccache in the tech stack to handle lengthy C++ NDK compilation times.
What’s a ccache?
Ccache is a compiler cache that checks the cache before actually compiling. It has both direct and pre-processed modes, and since the ccache plugin was not supported prior to Clang 3.2, there were some issues prior to Clang 3.2, but now Clang is 3.2.3, so there are no Clang unsupported issues. Ccache is a project with a long history and its main focus is fast and correct.
The information of “CCache Xcode” found on the Internet is outdated and invalid. After I tried online methods quickly, it could not be configured to make it work normally. As our code base became more complex and our Jenkins work cluster was 10 Macs, the testing time went from almost unbearable to literally unbearable. Facebook’s Christian Legnitto, who previously worked on OS X versioning at Apple, suggested we try ccache after Twitter complained that it was now all about managing the Jenkins cluster.
Let’s get started
Use the following command to install ccache:
brew install ccache
Copy the codeIf you don’t have Homebrew installed, go here and install Homebrew first. If you don’t want to move, install Homebrew directly with the following command:
/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
Copy the codeIn order for Xcode to call ccache, we need a little script to configure some environment variables and then call ccache. Save this script somewhere in your project and name it ccache-clang.
#! /bin/sh if type -p ccache >/dev/null 2>&1; then export CCACHE_MAXSIZE=10G export CCACHE_CPP2=true export CCACHE_HARDLINK=true export CCACHE_SLOPPINESS=file_macro,time_macros,include_file_mtime,include_file_ctime,file_stat_matches exec ccache /usr/bin/clang "$@" else exec clang "$@" fiCopy the codeDepending on your situation, if you have C++ files in your project, you may also need a script named ccache-clang++ and write it like this:
#! /bin/sh if type -p ccache >/dev/null 2>&1; then export CCACHE_MAXSIZE=10G export CCACHE_CPP2=true export CCACHE_HARDLINK=true export CCACHE_SLOPPINESS=file_macro,time_macros,include_file_mtime,include_file_ctime,file_stat_matches exec ccache /usr/bin/clang "$@" else exec clang++ "$@" fiCopy the codeIf the cache is not hit, it will compile as it was compiled before, rather than reporting ccache not found (ccache has a built-in shell script, so checking the cache is fast).
Create a shell script:
Create touch ccache-clang Open the script open-a xcode ccache-clang Paste the script content run the script chmod 755 ccache-clangCopy the codeIf you learn about ccache configuration, you’ll find that there are many options to choose from. We used a fairly aggressive caching strategy above, and it worked fine. For your own project, you might start without CCACHE_SLOPPINESS, and then add the cache all at once if everything is working fine.
The most important argument here is CCACHE_CPP2, which is used to address the fact that Clang will handle file output from the preprocessor and may find many potential problems that you didn’t notice, such as unnecessary parentheses due to macro extensions. Using this option will slow down the compile time slightly, but it will be much faster than not using ccache at all. Peter Eisentraut has written a good article on this issue.
You also need to define CC variables in Xcode. In PSPDFKit, we do this in the.xcconfig file, which is shared across all our projects (this is a nice uniform project configuration, and easy to read and find). In the meantime, you can configure it directly in Xcode project Settings:
CC = "$(SRCROOT)/.. /Resources/ccache-clang"Copy the code
That’s it! The next time you compile it will be slower than normal. You can use ccache -s in the terminal to check whether ccache is working properly. There should be a lot of missed caches to start with, but as caches start to replace later compilations, compilations will get faster.
To the pit
Where the road is rough, there are potholes: Ccache has some disadvantages.
Modules for Clang are not supported, and ccache will be disabled if -fmodules is detected. So, to be ccache compatible, you need to replace all the elegant @import UIKit in your project with the old # import
, and all the problems that come with using ccache, such as macros. In the PSPDFKit project we used objective-c ++, and when we used a lot of C++ code, we couldn’t use modules, so it didn’t affect us. Modules automatically links to the used framework, but after disabling Modules, you need to manually add the used framework, which is tedious but quick to do.
You also need to stop using.pch. Apple does not recommend the use of.pch, and it is generally considered bad programming style to use.pch, and importing it wherever it is used is better than. It’s easy for us to delete those.pch. Of course, ccache doesn’t cache Swift files for you. Although Swift also uses Clang, ccache does nothing with Swift files. Maybe Ccache will eventually support Swift, but I can’t count on it. Since Swift is not stable so far, and we even need to make binary compatibility between the two versions of Swift, we cannot write our SDK with Swift, so the problem that CCache does not support Swift is not a problem for us.
At compile time, we should always monitor the project to see if it throws incompatible warnings. See the Unsupported compiler option. I spent a fair amount of time dealing with these incompatibilities. Setting the CCACHE_LOGFILE temporary environment variable will help us pinpoint errors: CCache will indicate which flags are problematic, as well as the specifics of cache hits and misses.
steipete@steipete-rmbp ~ $ ccache -s cache directory /Users/steipete/.ccache primary config / Users/steipete. Ccache/ccache. Conf secondary config (readonly)/usr/local/Cellar/ccache 3.2.3 / etc/ccache. Conf cache hit (direct) 42530 cache hit (preprocessed) 18147 cache miss 28379 called for link 1344 called for preprocessing 645 compile failed 1 preprocessor error 2 can't use precompiled header 2567 unsupported source language 12 unsupported Compiler Option 11564 no input file 2 files in cache 124223 Cache size 8.7 GB Max cache size 15.0 GBCopy the codeIs it worth it?
To give you an idea of what we did with ccache, our compilation time averaged 8 minutes with ccache, up from 14 minutes when we didn’t use ccache. Before using Ccache, compiling and packaging the entire PSPDFKit on the fastest MacBook Pro took 50 minutes. After using ccache, it took 15 minutes. Adding ccache to our tech stack was a huge step forward and I wish I hadn’t known about this great tool sooner!
Precompiled Header problem
Anton Bukov says to handle this by disabling GCC_PRECOMPILE_PREFIX_HEADER and enabling GCC_PREFIX_HEADER.