How to compile iOS projects 5 times faster

Preface

Beiliao currently develops two apps, Beiliao Parent Edition and Beiliao Teacher Edition. Recently, due to the rapid iteration of new features, the project scale has grown rapidly. The business code of a single end is about 230,000 lines, the private library is about 60,000 lines, the third-party library code is about 150,000 lines, and the number of lines of code of a single client is about 600,000. It now takes 11 to 12 minutes to package once. Although it is far less than Facebook's 40 minutes, we often release the beta version two or three times a day during internal testing. The CPU usage during packaging is basically 100%. Because there is no dedicated CI machine, it takes up a lot of work time for the colleague responsible for packaging (actually myself), so I have been looking for a solution to speed up packaging recently.

Current project structure

Our project uses CocoaPods to manage dependencies of third-party libraries and private libraries, which should be standard for most projects. It is still a pure Objective-C project and has not introduced Swift.

Researched solutions

The following are some of the mainstream solutions I have studied and the reasons why I did not adopt them in the end. These solutions have their own limitations, but they also gave me a lot of inspiration. The thinking process is as valuable as the final solution.

cocoapods-packager

Cocoapods-packager can package any pod into a Static Library, saving the time of repeated compilation and speeding up the compilation time to a certain extent, but it also has its own disadvantages:

1. The optimization is not thorough, and can only optimize the compilation speed of third-party and private Pods, but is powerless for other business codes that are frequently changed.
2. Subsequent updates of private libraries and third-party libraries are very troublesome. When the source code is modified, it needs to be repackaged and uploaded to the internal Git repository.
3. Too many binary files will slow down Git operations (Git LFS has not yet been deployed)
4. Difficult to debug source code

Carthage

This solution is similar to cocoapods-packager, with similar advantages and disadvantages, but Carthage can debug source code more conveniently. Since we are currently using CocoaPods on a large scale, switching to Carthage for package management requires a lot of conversion work, so we don't consider this solution.

Buck

Buck is a general-purpose build system, open sourced by Facebook. Its biggest feature is intelligent incremental compilation, which can greatly improve the build speed. When I first heard about Buck, it could only be used on Android, but now it has been adapted to iOS.

The main reason it can speed up the build is that it caches the compilation results. By continuously monitoring the file changes in the project directory, only the modified files are compiled each time. Another feature that inspired me is the HTTP Cache Server, which uses a cache file server to save everyone's compilation results, so that as long as one person in the team has compiled a file, others do not need to compile it again and can download it directly.

Buck is a fairly complete solution, and many large foreign companies such as Uber have already used it. I also spent a lot of time studying it, but ultimately I think it is not very suitable for our project and team at the moment, mainly for the following reasons:

1. Buck abandoned Xcode's project files and needed to manually write configuration files to specify compilation rules, which required major adjustments to existing projects. We are currently iterating new features rapidly and do not have the time or manpower to implement them.
2. The development and debugging processes had to be greatly changed. Because Buck took over the project compilation process, if you want to debug the project, you can't simply press ?+R in Xcode. You have to let Buck generate Xcode project files first. Uber engineers even recommended using Nuclide to replace Xcode as the development environment. Although it is feasible in principle, the team needs to spend a lot of time to adapt, and the efficiency reduction in the short term is inevitable.
3. Using Xcode to debug code does not benefit from faster compilation. Although you can use the buck command to start the App and then start lldb in the command line to debug, you will not be able to use Xcode's debugging tools such as View Debugging and Memory Graph Debugger.

Bazel

Bazel is very similar to Buck and is open source from Google. Its advantages and disadvantages are similar to those of Buck, so I will not go into details.

distcc distributed compilation

The principle is to send part of the files to be compiled to the server, and after the server completes the compilation, it will send back the compiled products. I tried the more famous distcc, the setup process was relatively simple, and finally I was able to successfully dispatch the compilation tasks to multiple servers in the intranet. However, the CPU usage of other compilation servers is always very low, only about 20%; that is to say, the speed of dispatching tasks is not even as fast as the speed of server compilation, and the time consumed in the process of dispatching tasks and sending back the compiled products exceeds the local direct compilation. I kept adjusting the parameters and repeated the test many times, and finally found that the compilation time did not get faster at all, and even got a little slower. Maybe the scale of our current project is not suitable for distributed compilation.

Final Solution: CCache

Let’s first look at my requirements for a solution:

1. Can significantly improve compilation speed, at least reduce compilation time by 50%
2. No major adjustments to the project are required
3. No need to change the development tool chain

CCache is a tool that can cache the intermediate products of compilation. It has been widely used in other fields, but it is less used in the iOS world. Through my practice, it can meet my three requirements. I first learned about it by searching this article: Using ccache for Fun and Profit | Inside PSPDFKit

If you don't use CocoaPods, just refer to the above article. Because some additional adjustments need to be made for CocoaPods, I will explain it here. Next, let's talk about how to apply CCache in an iOS project that uses CocoaPods as a package manager.

Installation steps:

Note: The project path cannot contain Chinese characters, otherwise it will affect the normal operation of CCache.

Install CCache

First, you need to install Homebrew on your computer. This should be standard for programmers using macOS, so you can skip it.

Install CCache via Homebrew and execute in the command line

 $ brew install ccache

After the command is run, the installation is successful.

Create CCache compilation script

In order to allow CCache to intervene in the entire compilation process, we need to use CCache as the C compiler of the project. When CCache cannot find the compilation cache, it will pass the compilation instructions to the real compiler clang.

Create a new file named ccache-clang, and put the following script in your project

ccache-clang

 #!/bin/sh
 if type -p ccache >/dev/ null 2>&1; then  
 export CCACHE_MAXSIZE=10G
 export CCACHE_CPP2= true  
 export CCACHE_HARDLINK= true  
 export CCACHE_SLOPPINESS=file_macro,time_macros,include_file_mtime,include_file_ctime,file_stat_matches
 # Specify the log file path to the desktop. It will be useful for troubleshooting integration problems later. Delete it after the integration is successful. Otherwise, it will take up a lot of disk space.
 export CCACHE_LOGFILE= '~/Desktop/CCache.log'  
 exec ccache /usr/bin/clang "$@"  
 else  
 exec clang "$@"  
 fi

In the command line, cd to the directory of the ccache-clang file and change its permissions to executable

 $ chmod 777 ccache-clang

If your code or third-party library code uses C++, copy the ccache-clang file and rename it to ccache-clang++. The corresponding calls to clang should also be changed to clang++, otherwise CCache will not be applied to C++ code.

 #!/bin/sh
 if type -p ccache >/dev/ null 2>&1; then  
 export CCACHE_MAXSIZE=10G
 export CCACHE_CPP2= true  
 export CCACHE_HARDLINK= true  
 export CCACHE_SLOPPINESS=file_macro,time_macros,include_file_mtime,include_file_ctime,file_stat_matches
 # Specify the log file path to the desktop. It will be useful for troubleshooting integration problems later. Delete it after the integration is successful. Otherwise, it will take up a lot of disk space.
 export CCACHE_LOGFILE= '~/Desktop/CCache.log'  
 exec ccache /usr/bin/clang++ "$@"  
 else  
 exec clang++ "$@"  
 fi

After completion, the project should have these two files

Xcode project adjustments

Define CC constants

In your project's build settings, add a constant CC. This value will cause Xcode to use the executable file in the execution path as the C compiler when compiling.

The value of the CC constant is $(SRCROOT)/ccache-clang. If your script is not placed in the project root directory, adjust the path yourself. If an error occurs when you run the project, check if the path is filled in incorrectly.

Disable Clang Modules

Because CCache does not support Clang Modules, you need to turn off the Enable Modules option. How to handle this problem on CocoaPods will be discussed later.

Adjustments required after turning off Enable Modules

Because Enable Modules is turned off, all @import statements must be deleted and replaced with #import syntax.

For example, replace @import UIKit with #import. After that, if you use other system frameworks such as AVFoundation, CoreLocation, etc., Xcode will no longer automatically import them for you. You have to manually import them in the Build Phrase -> Link Binary With Libraries of the project Target.

Test results

Try compiling it again, and then type cache -s in the command line to see ccache running statistics similar to the following:

 cache directory /Users/mac/.ccache  
 primary config /Users/mac/.ccache/ccache.conf  
 secondary config (readonly) /usr/ local /Cellar/ccache/3.3.4_1/etc/ccache.conf  
 cache hit (direct) 14378  
 cache hit (preprocessed) 1029  
 cache miss 7875  
 cache hit rate 66.18 %  
 called for link 61  
 called for preprocessing 48  
 compile failed 2  
 preprocessor error 4  
 can't use precompiled header 70  
 unsupported compiler option 2332  
 No input file 11  
 cleanups performed 0  
 files in cache 35495  
 cache size 1.3 GB  
 max cache size 5.0 GB

If the connection is successful, you can see that the cache miss is not 0. Because there is no cache in the first compilation, it must be a full miss. Then compile a second time. If you can see the cache hit number start to soar, congratulations, the connection is successful.

CocoaPods handling

If your project does not use CocoaPods for package management, then you have already fully integrated it and do not need to perform the following operations.

Because CocoaPods will package third-party libraries into a Static Library (or Dynamic Framework if the use_frameworks! option is used), the Static Library generated by CocoaPods also needs to turn off the Enable Modules option. However, because CocoaPods will regenerate the Pods project every time it executes pod update, if you modify the Enable Modules option in the Pods project directly in Xcode, it will be changed back the next time you execute pod update. We need to add the following code to the Podfile to turn off the Enable Modules option for the generated project and add the CC parameter, otherwise the pod will not be able to use CCache acceleration when compiling:

 post_install do |installer_representation|
 installer_representation.pods_project.targets.each do |target|
 target.build_configurations.each do |config|
 #Turn off Enable Modules
 config.build_settings[ 'CLANG_ENABLE_MODULES' ] = 'NO'  
 # Add the CC parameter to the generated Pods project file. Modify the path value according to your own project.
 config.build_settings[ 'CC' ] = '$(PODS_ROOT)/../ccache-clang'  
 end  
 end  
 end  

It should be noted that if a Pod you use references a system framework, such as AFNetworking references System Configuration, you need to import it in your own project's Build Phrase -> Link Binary With Libraries, otherwise you may receive errors such as Undefined symbols xxx for architecture yyy when compiling. It feels a bit like going back to the primitive era, but considering the great improvement in compilation speed, this price is acceptable.

Integration Troubleshooting

Pay special attention to the output of the log file and the statistics of the ccache -s command. If you see words like unsupported compiler option -fmodules in the log, it means that your Enable Modules is not turned off. Check carefully according to the previous steps. For other problems, refer to the Troubleshooting of the official document.

Further optimization

Remove Precompiled Header File

The content of PCH will be appended to the front of each file, and CCache searches for cache based on the MD4 digest of the file content. Therefore, when you modify the content of PCH or the header file referenced by PCH, all caches will be invalid and all can only be recompiled. CCache will cause the compilation time to be longer because the cache needs to be updated during the first compilation. For Beiliao's project, it has been almost doubled. Therefore, if PCH or the files introduced by PCH are frequently modified, the cache will miss frequently. In this case, it is better not to use CCache.

To avoid the above situation, I suggest importing as few header files as possible in PCH, and only retaining the header files of system frameworks and third-party libraries that are rarely changed. It is best to completely delete PCH. Anyway, Apple does not recommend using PCH now, and new projects created in Xcode do not have PCH by default.

Share cache folders within a team

I have tried this optimization method, but the final effect was not very good, so I did not adopt it. There is a description about shared cache folders in the official document of CCache, which describes how to modify the configuration of CCache so that the compilation cache can be shared among multiple computers. In theory, as long as one person has compiled the file, others can directly download it, saving time for the entire team. Because Buck also has a similar mechanism, I think it is worth a try, so I built an OwnCloud network disk in the company's LAN and let everyone put the CCache cache directory on their computer to share. Although the experiment was successful, the actual effect was not good. Because the cache directory with a size of several GB on multiple computers is synchronized, a lot of file comparison and transmission work needs to be performed in the background. Performing these operations while compiling will consume a lot of computing resources, which will slow down the compilation speed. In addition, after removing PCH, the cache hit rate is actually quite considerable, and there is no need to further improve the cache hit rate through shared cache, so I finally gave up the idea of ​​shared cache. If you are still not satisfied with the cache hit rate, you can consider trying in this direction.

Summarize

By integrating CCache, the time it takes to package our project in Xcode (select Product -> Archive in the menu) has been reduced from 11-12 minutes to 130 seconds, which is about a five-fold improvement. The results are impressive. The integration process is actually very simple. It took me only two hours from the beginning to the successful integration. If you are also troubled by the long compilation time, I recommend you give it a try.