The principles and applications of iOS compilation process

Preface

Programming languages ​​can generally be divided into two types: compiled languages ​​and interpreted languages.

C++ and Objective C are compiled languages. When a compiled language is executed, it must first be generated into machine code by a compiler. The machine code can be executed directly on the CPU, so the execution efficiency is higher.

JavaScript and Python are both interpreted languages. Interpreted languages ​​do not need to go through the compilation process, but when they are executed, an intermediate interpreter interprets the code into code that can be executed by the CPU. Therefore, compared with compiled languages, interpreted languages ​​are less efficient, but more flexible to write, which is why JS is so good.

The commonly used languages ​​for iOS development are Objective C and Swift. Both are compiled languages, in other words, they need to be compiled before they can be executed. The compilation of both depends on Clang + LLVM. Due to space limitations, this article only focuses on Objective C because the principles are similar.

Some students may ask, I don't understand the compilation process, so it's okay to write code? I don't deny this. However, a full understanding of the compilation process will be of great help to your development. At the end of this article, the following examples will be used to explain how to use XCode and compilation properly.

• __attribute__
• Clang warning handling
• Preprocessing
• Insert compile-time script
• Improve project compilation speed

For those who don’t want to read my long-winded explanation of a lot of principles, you can jump directly to the last chapter of this article.

iOS Compilation

Whether it is OC or Swift, Clang is used as the compiler front end and LLVM (Low level virtual machine) is used as the compiler back end. So the simple compilation process is as shown in the figure

Compiler front end

The task of the compiler front end is to perform syntax analysis, semantic analysis, and generate intermediate code (intermediate representation). In this process, type checking will be performed, and if errors or warnings are found, the line will be marked.

Compiler backend

The compiler backend will perform machine-independent code optimization, generate machine language, and perform machine-dependent code optimization. The backend processing of iOS compilation is as follows

• The LVVM optimizer will generate BitCode, perform link-time optimization, and so on.

[[188090]]

• The LLVM machine code generator will generate different machine codes for different architectures, such as arm64.

Execute an XCode build process

When you select build in XCode (shortcut command+B), the following process will be executed

• Compilation information is written to auxiliary files to create the compiled file structure (name.app)
• Process file packaging information, such as in a debug environment

 Entitlements:
 {
 "application-identifier" = "app's bundleid" ;
 "aps-environment" = development;
 }

• Execute CocoaPod pre-compilation script
  • For example, for projects using CocoaPod, CheckPods Manifest.lock will be executed
• Compile each .m file using the CompileC and clang commands.

 CompileC ClassName.o ClassName.m normal x86_64 objective-c com.apple.compilers.llvm.clang.1_0.compiler
 export LANG=en_US.US-ASCII
 export PATH= "..."
 clang -x objective-c -arch x86_64 -fmessage-length=0 -fobjc-arc... -Wno-missing-field-initializers ... -DDEBUG=1 ... -isysroot iPhoneSimulator10.1.sdk -fasm-blocks ... -I The files mentioned above -F The required Framework -iquote The required Framework ... -c ClassName.c -o ClassName.o

Through this compilation command, we can see

clang is the actual compilation command

-x objective-c specifies the language for compilation

-arch x86_64 specifies the architecture for compilation, similar to arm7, etc.

-fobjc-arc Some of the -f start, specify the use of arc and other information. This is why you can use non-ARC programming for a single .m file.

-Wno-missing-field-initializers A series of -W starting with the compiler warning options, through which you can customize the compilation options

-DDEBUG=1 Some of the -D prefixes refer to precompiled macros, which can be used to implement conditional compilation

-iPhoneSimulator10.1.sdk specifies the iOS SDK version used for compilation

-I Write compilation information to the specified auxiliary file

-F Link required Framework

-c ClassName.c compile file

-o ClassName.o compiled product

• Link required Frameworks, such as Foundation.framework, AFNetworking.framework, ALiPay.fframework
• Compile the xib file
• Copy xib, pictures and other resource files to the result directory
• Compiling ImageAssets
• Processing info.plist
• Execute CocoaPod script
• Copy the Swift standard library
• Creating and signing the .app file

Contents of the IPA package

For example, we download WeChat through the iTunes Store, then get the ipa installation package, and then actually look at the contents of its installation package.

• Right click ipa and rename it to .zip
• Double-click the zip file and you will get a folder after decompression. Therefore, the ipa package is an ordinary compressed package.

• Right-click WeChat in the picture, select Show Package Contents, and then you can see the actual ipa package contents.

Contents of the binary file

Through XCode's Link Map File, we can peek into the layout of the binary file.

In XCode -> Build Settings -> search for map -> turn on Write Link Map File

After enabling it, during compilation, we can see the corresponding link map text file in the corresponding Debug/Release directory.

The default directory is

~/Library/Developer/Xcode/DerivedData/TARGET-NAME>-Corresponding ID/Build/Intermediates/TARGET-NAME>.build/Debug-iphoneos/TARGET-NAME>.build/

For example, my TargetName is EPlusPan4Phone and the directory is as follows

/Users/huangwenchen/Library/Developer/Xcode/DerivedData/EPlusPan4Phone-eznmxzawtlhpmadnbyhafnpqpizo/Build/Intermediates/EPlusPan4Phone.build/Debug-iphonesimulator/EPlusPan4Phone.build

The main parts of this mapping file are as follows:

Object files

What this section includes

– .o file, which is the result of compiling the .m file mentioned above.

– .a file

– Framework that needs to be linked

 #! Arch: x86_64
 #Object files:
 [0] linker synthesized
 [1] /EPlusPan4Phone.build/EPlusPan4Phone.app.xcent
 [2]/EPlusPan4Phone.build/Objects-normal/x86_64/ULWBigResponseButton.o
 …
 [1175]/UMSocial_Sdk_4.4/libUMSocial_Sdk_4.4.a(UMSocialJob.o)
 [1188]/iPhoneSimulator10.1.sdk/System/Library/Frameworks//Foundation.framework/Foundation

The storage content of this area is relatively simple: the first part is the file number, followed by the file path. The file number will be used later.

Sections

This area provides the location and size of each segment and section in the executable file. This area fully describes the entire content of the executable file.

Among them, the segment is divided into two types

__TEXT snippet

__DATA data segment

For example, in an App I wrote before, the Sections area is as follows. You can see that the code segment

The address of the __text section is 0x1000021B0, the size is 0x0077EBC3, and the next location after adding the two is exactly the location of __stubs 0x100780D74.

 # Sections:
 # Position Size Segment Section
 # Address Size Segment Section
 0x1000021B0 0x0077EBC3 __TEXT __text //code
 0x100780D74 0x00000FD8 __TEXT __stubs
 0x100781D4C 0x00001A50 __TEXT __stub_helper
 0x1007837A0 0x0001AD78 __TEXT __const //Constant
 0x10079E518 0x00041EF7 __TEXT __objc_methname //OC method name
 0x1007E040F 0x00006E34 __TEXT __objc_classname //OC class name
 0x1007E7243 0x00010498 __TEXT __objc_methtype //OC method type
 0x1007F76DC 0x0000E760 __TEXT __gcc_except_tab
 0x100805E40 0x00071693 __TEXT __cstring // string
 0x1008774D4 0x00004A9A __TEXT __ustring
 0x10087BF6E 0x00000149 __TEXT __entitlements
 0x10087C0B8 0x0000D56C __TEXT __unwind_info
 0x100889628 0x000129C0 __TEXT __eh_frame
 0x10089C000 0x00000010 __DATA __nl_symbol_ptr
 0x10089C010 0x000012C8 __DATA __got
 0x10089D2D8 0x00001520 __DATA __la_symbol_ptr
 0x10089E7F8 0x00000038 __DATA __mod_init_func
 0x10089E840 0x0003E140 __DATA __const //Constant
 0x1008DC980 0x0002D840 __DATA __cfstring
 0x10090A1C0 0x000022D8 __DATA __objc_classlist // OC method list
 0x10090C498 0x00000010 __DATA __objc_nlclslist
 0x10090C4A8 0x00000218 __DATA __objc_catlist
 0x10090C6C0 0x00000008 __DATA __objc_nlcatlist
 0x10090C6C8 0x00000510 __DATA __objc_protolist // OC protocol list
 0x10090CBD8 0x00000008 __DATA __objc_imageinfo
 0x10090CBE0 0x00129280 __DATA __objc_const // OC constant
 0x100A35E60 0x00010908 __DATA __objc_selrefs
 0x100A46768 0x00000038 __DATA __objc_protorefs
 0x100A467A0 0x000020E8 __DATA __objc_classrefs
 0x100A48888 0x000019C0 __DATA __objc_superrefs // OC parent class reference
 0x100A4A248 0x0000A500 __DATA __objc_ivar // OC iar
 0x100A54748 0x00015CC0 __DATA __objc_data
 0x100A6A420 0x00007A30 __DATA __data
 0x100A71E60 0x0005AF70 __DATA __bss
 0x100ACCDE0 0x00053A4C __DATA __common

Symbols

The Section part divides the binary file into the first level. However, Symbols divides each segment in the Section into the second level.

For example, __TEXT __text represents the code content in the code segment.

 0x1000021B0 0x0077EBC3 __TEXT __text //code

The corresponding Symbols also start at 0x1000021B0. The file number corresponds to the number above.

 [2]/EPlusPan4Phone.build/Objects-normal/x86_64/ULWBigResponseButton.o

The specific contents are as follows

 # Symbols:
 Address Size File Number Method Name
 # Address Size File Name
 0x1000021B0 0x00000109 [2] -[ULWBigResponseButton pointInside:withEvent:]
 0x1000022C0 0x00000080 [3] -[ULWCategoryController liveAPI]
 0x100002340 0x00000080 [3] -[ULWCategoryController categories]
 ....

Now that we know how OC methods are stored, let's take a look at how ivar is stored.

First find __DATA __objc_ivar in the data stack

 0x100A4A248 0x0000A500 __DATA __objc_ivar

Then, by searching for the address 0x100A4A248, we can find the storage area of ​​​​ivar.

 0x100A4A248 0x00000008 [3] _OBJC_IVAR_$_ULWCategoryController._liveAPI

It is worth mentioning that for String, it will be explicitly stored in the data segment, for example,

 0x1008065C2 0x00000029 [11] literal string: http://sns.whalecloud.com/sina2/callback

Therefore, if your encryption key is written in a file in plain text, it is a very dangerous thing.

dSYM file

After each compilation, we will generate a dsym file, which stores the hexadecimal function address mapping.

In the binary file actually executed by the App, methods are called by address. When the App crashes, third-party tools (Fabric, Umeng, etc.) will help us capture the crash call stack, which will contain the call information of the crash address. Then, through the dSYM file, we can map the address to the specific function location.

In XCode, select Window -> Organizer to see the archier file we generated

Then,

• Right click -> Show in finder.
• Right click -> View Package Contents.

For more information on how to use dsym files to analyze crash locations, see one of my previous blogs.

• How to debug crash reports collected by third parties on iOS

http://blog.csdn.net/hello_hwc/article/details/50036323

Application scenarios you can think of and can't think of

__attribute__

More or less, you will have seen attributes in third-party libraries or iOS header files.

for example

 __attribute__ ((warn_unused_result)) //If the return value is not used, a warning will be given during compilation

__attribtue__ is an advanced compiler directive that allows developers to specify more compiler checks and some advanced compile-time optimizations.

There are three types:

• Function Attribute
• Variable Attribute
• Type Attribute

Grammatical structure

The syntax format of __attribute__ is: __attribute__ ((attribute-list))

Place it before the declaration semicolon ";".

For example, the most common thing in third-party libraries is to declare that a property or method is deprecated in the current version.

 @property (strong,nonatomic)CLASSNAME * property __deprecated;

The advantage of this is that it provides developers with a transitional version to let them know that this attribute is deprecated and that they should use the latest API, but the __deprecated attribute can still be used normally. If it is deprecated directly, the code will not run when the developer updates the Pod.

There are many usage scenarios for __attribtue__. This article only lists a few commonly used ones in iOS development:

 //Deprecated API, used for API update
 #define __deprecated __attribute__((deprecated))
 //Deprecation with description information
 #define __deprecated_msg(_msg) __attribute__((deprecated(_msg)))
 //When encountering __unavailable variables/methods, the compiler directly throws an Error
 #define __unavailable __attribute__((unavailable))
 //Tell the compiler not to throw a warning even if this variable/method is not used
 #define __unused __attribute__((unused))
 //The opposite of __unused
 #define __used __attribute__((used))
 //If the return value of the method is not used, warn
 #define __result_use_check __attribute__((__warn_unused_result__))
 //OC method is not available in Swift
 #define __swift_unavailable(_msg) __attribute__((__availability__(swift, unavailable, message=_msg)))

Clang warning handling

You must have seen the following code:

 #pragma clang diagnostic push
 #pragma clang diagnostic ignored "-Wundeclared-selector"
 ///Code
 #pragma clang diagnostic pop

This code snippet does the following:

• Push the current compilation environment onto the stack
• Ignore -Wundeclared-selector (undeclared) Selector warnings
• Compiling the code
• Pop the compilation environment

With clang diagnostic push/pop, you can flexibly control the compilation options of code blocks.

In my previous article, I introduced the XCode warnings in detail. Due to the length limitation of this article, I will not explain it in detail.

• iOS: Reasonable use of Clang warnings to improve code quality

http://blog.csdn.net/Hello_Hwc/article/details/46425503

Preprocessing

The so-called preprocessing is the processing before compilation. Preprocessing allows you to define compiler variables and implement conditional compilation.

For example, code like this is common

 #ifdef DEBUG
 //...
 # else
 //...
 #endif

Similarly, we can also define other preprocessor variables. In XCode-select Target-build settings, search for proprecess. Then click the blue plus sign in the figure to set preprocessor macros for debug and release modes respectively.

For example, we add: TestServer, which means that the code in this macro runs on the test server.

Then, with multiple Targets (right-click Target and select Duplicate), a single Target is responsible for the test server. This way we don't have to modify the code every time we switch the test server.

 #ifdef TESTMODE
 //Test server related code
 # else
 //Production server related code
 #endif

Insert Script

Typically, if you use CocoaPod to manage third-party libraries, your Build Phase would look like this:

Among them, the ones starting with [CP] are scripts inserted by CocoaPod.

• Check Pods Manifest.lock, used to check whether the third-party libraries managed by cocoapod need to be updated
• Embed Pods Framework, run the script to link the static/dynamic libraries of the three-party library
• Copy Pods Resources, run the script to copy the resource files of the tripartite library

And all these configuration information is stored in this file (.xcodeprog)

At this point, the principle of CocoaPod is roughly understood. By modifying the xcodeproject and configuring the compile-time script, we can ensure that the three-party library can be compiled and connected correctly.

Similarly, we can also insert our own scripts to do some extra things. For example, every time we archive, we have to manually adjust the target build version, and if we are not careful, we will forget. This process can be automated by inserting a script.

 buildNumber=$(/usr/libexec/PlistBuddy -c "Print CFBundleVersion" "${PROJECT_DIR}/${INFOPLIST_FILE}" )
 buildNumber=$(($buildNumber + 1))
 /usr/libexec/PlistBuddy -c "Set :CFBundleVersion $buildNumber" "${PROJECT_DIR}/${INFOPLIST_FILE}"

This script is actually very simple. It reads the current pist build version number, then adds one to it and rewrites it.

It is also very simple to use:

• Xcode – Select Target – Select build phase
• Select Add Run Script Phase

Then copy this script and check Run Script Only When installing to ensure that this script will only be executed when we install it on the device. Rename the script to Auto Increase build number

Then, drag this script under Link Binary With Libraries

Script compilation and packaging

Scripted compilation and packaging is very useful for CI (continuous integration). In iOS development, the two necessary commands for compilation and packaging are:

 //Compile into .app
 xcodebuild -workspace $projectName.xcworkspace -scheme $projectName -configuration $buildConfig clean build SYMROOT=$buildAppToDir
 //Pack
 xcrun -sdk iphoneos PackageApplication -v $appDir/$projectName.app -o $appDir/$ipaName.ipa
 You can view detailed documentation through the info command.
 info xcodebuild

In the appendix at the end of this article, an automatic packaging script that I used before is provided.

Improve project compilation speed

Usually, when the project is large and there are many source codes and third-party libraries, we will find that the compilation speed is very slow. After understanding the compilation process of XCode, we can optimize the compilation speed from the following perspectives:

Check the compile time

We need a way to see the compilation time so that we can make a comparison and know whether our optimization is effective.

For XCode 8, close XCode and enter the following command in the terminal:

 $ defaults write com.apple.dt.Xcode ShowBuildOperationDuration YES

Then, restart XCode, and compile, you will see the compile time here.

Code-level optimization

forward declaration

The so-called forward declaration is @class CLASSNAME, not #import CLASSNAME.h. In this way, the compiler can greatly improve the replacement speed of #import.

Package commonly used tool classes (Framework/.a)

Package it into a Framework or static library so that this part of the code does not need to be recompiled during compilation.

Common header files are placed in precompiled files

XCode's pch file is a precompiled file. The content here has been precompiled before executing XCode build and introduced into each .m file.

Compiler options optimization

In Debug mode, no dsym file is generated

As mentioned above, the dysm file stores debugging information, and in Debug mode, we can use XCode and LLDB for debugging. Therefore, there is no need to generate additional dsym files to slow down compilation.

Debug turns on Build Active Architecture Only

In XCode -> Build Settings -> Build Active Architecture Only, change it to YES. This way, you can only compile the current version, such as arm7/arm64, etc. Remember to only enable Debug mode. This option is automatically enabled in higher versions of XCode.

In Debug mode, turn off compiler optimization

Compiler Optimizations

Follow-up

There is a lot more I want to write about this article, but due to the length limit, I will leave it at this. A lot of unhappy things happened recently, so I remind myself: every failure is a learning experience.

When I have time later, I will introduce some black technologies during compilation:

• Write additional compilation information
• The function calling process and finding the address of the function in the binary file at runtime
• …

appendix

Automatically compile packaging scripts

 export LC_ALL=zh_CN.GB2312;
 export LANG=zh_CN.GB2312
 buildConfig = "Release" //This is the build mode
 projectName=`find . - name *.xcodeproj | awk -F "[/.]" '{print $(NF-1)}' `
 projectDir=`pwd`
 wwwIPADir=~/Desktop/$projectName-IPA
 isWorkSpace= true
 echo "~~~~~~~~~~~~~~~~~~~~~Start compiling~~~~~~~~~~~~~~~~~~~~"
 if [ -d "$wwwIPADir" ]; then
 echo $wwwIPADir
 echo "file directory exists"
 else
 echo "The file directory does not exist"
 mkdir -pv $wwwIPADir
 echo "${wwwIPADir} directory created successfully"
 fi
 cd $projectDir
 rm -rf ./build
 buildAppToDir=$projectDir/build
 infoPlist= "$projectName/Info.plist"
 bundleVersion=`/usr/libexec/PlistBuddy -c "Print CFBundleShortVersionString" $infoPlist`
 bundleIdentifier=`/usr/libexec/PlistBuddy -c "Print CFBundleIdentifier" $infoPlist`
 bundleBuildVersion=`/usr/libexec/PlistBuddy -c "Print CFBundleVersion" $infoPlist`
 if $isWorkSpace ; then #Whether to use CocoaPod
 echo "Start compiling workspace...."
 xcodebuild -workspace $projectName.xcworkspace -scheme $projectName -configuration $buildConfig clean build SYMROOT=$buildAppToDir
 else
 echo "Start compiling target...."
 xcodebuild -target $projectName -configuration $buildConfig clean build SYMROOT=$buildAppToDir
 fi
 if test $? -eq 0
 then
 echo "~~~~~~~~~~~~~~~~~~~~~Compilation successful~~~~~~~~~~~~~~~~~~~~"
 else
 echo "~~~~~~~~~~~~~~~~~~~~~Compilation failed~~~~~~~~~~~~~~~~~~~~"
 exit 1
 fi
 ipaName=`echo $projectName | tr "[:upper:]" "[:lower:]" ` #Convert the project name to lowercase
 findFolderName=`find . - name "$buildConfig-*" -type d |xargs basename` #Find directory
 appDir=$buildAppToDir/$findFolderName/ #app location path
 echo "Start packaging $projectName.app into $projectName.ipa....."
 xcrun -sdk iphoneos PackageApplication -v $appDir/$projectName.app -o $appDir/$ipaName.ipa
 if [ -f "$appDir/$ipaName.ipa" ]
 then
 echo "Packaging $ipaName.ipa successfully."
 else
 echo "Failed to package $ipaName.ipa."
 exit 1
 fi
 path=$wwwIPADir/$projectName$( date +%Y%m%d%H%M%S).ipa
 cp -f -p $appDir/$ipaName.ipa $path #Copy ipa file
 echo "Copy $ipaName.ipa to ${wwwIPADir} successfully"
 echo "~~~~~~~~~~~~~~~~~~~~~Compilation completed, processing successful~~~~~~~~~~~~~~~~~~~~"