Ctrip Train Ticket iOS Project Development Experience Optimization Practice

​Author | Dong Hai, Ctrip mobile development expert, focusing on mobile framework and mobile performance.

Yuanshuai, a senior software engineer at Ctrip, is committed to platform infrastructure development.

1. Background

Now all the apps of major companies adopt componentized architecture, which brings many advantages such as high cohesion, low coupling, and platformization, making the project structure clearer and project management easier. Most iOS projects use CocoaPod for componentized management. Some large projects need packaging platforms to perform tasks such as component bundles and app test packages. In terms of development, binary and source code switching is used to improve compilation speed.

Although componentization has brought great benefits to the engineering management of APP projects, there are some cumbersome problems for developers:

During development, if you need to debug a component whose source code has not been unlocked, you need to re-execute the command to unlock the source code of the corresponding component before debugging.

Every time you switch the source code of a component, you need to enter a string of commands with various parameters in the terminal to execute pod install. Manual input is slow and prone to errors.

Componentization makes the component granularity become finer and finer, and the number of components managed by each person will increase. Each component update needs to be packaged on the packaging platform. After the component bundle is packaged, the test package is packaged for verification.

Although these allow the work to proceed normally, the tedious and repetitive operations affect the development efficiency of developers.

2. Current Situation

Ctrip's train ticket APP has always adopted component management. Last year, it switched to CocoaPod for component management. With the iteration of business and continuous improvement of infrastructure, pod components have become more and more refined. Currently, the number of pod components has exceeded 60+.

The larger the number of pod components, the higher the maintenance cost for developers. Not only do they have to manage and maintain the updates of each pod component, but they also have to deal with the pod component bundle issues and the long test package packaging time. The above tedious, repetitive, and time-consuming operations bother our iOS developers. If these development experience issues can be optimized as much as possible, it will inevitably lead to improved developer efficiency.

3. Optimization plan

In order to make it easier for developers to debug code, provide a better packaging and testing experience, and make the development process more focused, we have made many operational optimizations and technical practices, mainly including:

3.1 Implementing binary debugging through technical means

During the development process, you will inevitably encounter a situation where the component you want to debug has not been decompressed into source code. The program crashes during operation but crashes on the component that has not been decompressed into source code. All you see is a bunch of incomprehensible assembly code (Figure 1), and you cannot see rich enough debugging information like source code debugging.

Figure 1

3.1.1 Binary File Analysis

How to debug the binary without understanding the open source code, and locate the specific line when the binary component crashes became our new problem. We searched for various materials and found that Meituan has a CocoaPod plug-in called zsource that can perform binary debugging. Although it is not open source, the general logic is clearly listed in the article. The general principle is:

Take libXXXX.a binary file as an example, use MachOView to view the binary file to obtain more friendly binary information. We can see that the "debug_str" section is stored in the binary. debug_str will record the source code address internally during compilation:

Figure 2

Use the command to enter in the terminal:

 dwarfdump ./libXXXX.a | grep 'XXXX'

Noting the name of the AT_name field, we can find out from the DWARF 1.1.0 Reference document that:

• A DW_AT_name attribute whose value is a null-terminated string containing the full or relative path name of the primary source file from which the compilation unit was derived.
• A DW_AT_comp_dir attribute whose value is a null-terminated string containing the current working directory of the compile command that produced this compilation unit in some form that treats Forelax as a host system.

The XXXX.swift source file is located at this address: /Users/marshal/Desktop/XXXX/XXXX/XXXX.swift

This address is the address where the source code is located during compilation. During debugging, the compiler will first use the corresponding mapping address to load the source code file. If there is a source code file at the corresponding address, you can enter source code debugging.

3.1.2 Script Development

After understanding the basic principles, the next thing is to solve various problems and obstacles:

1) Get the source code of the static library.

2) Get the path where the source code file is stored when compiling the static library.

3) Create the path obtained above locally and associate the source code of the static library with the path.

• Question 1: When we created the binary package, in order to facilitate switching source code debugging, the source code + .a were downloaded to the local computer at the same time during pod install.
• Question 2: From the article by Meituan, we can learn that the dwarfdump command can be used to obtain the path where the source code file is located when compiling the static library stored in the static library.
• Question 3: For this question, I think most people’s first idea is to copy the source code of the static library to the static library compilation directory created locally, but we use a more lightweight way: associate the two targets through the soft link command ln.

Finally, we solved the above problems by developing scripts. We inserted the scripts into the pod install process through Hook post_integrate to make the whole process smooth and natural.

The main script code is as follows:

 #Link, .a file location, source directory, project name
 def link(lib_file,target_path,basename)
 #Query the source code location
 dir = (`dwarfdump "#{lib_file}" | grep "AT_comp_dir" | head -1 | cut -d \\" -f2 `)
 #Create a directory
 FileUtils.mkdir_p(dir)
 #Link
 FileUtils.rm_rf(File.join(dir,basename))
 `ln -s #{target_path} #{dir}`


 end 

 #Integrate script via pod post_integrate
 post_integrate do |installer|
 openStaticLibDebug(installer,"project")
 end
 def openStaticLibDebug(installer,project)
 if !ENV["DEBUGLIB"]
 return
 end
 #Script directory
 path_root = "#{Pathname.new(File.dirname(__FILE__)).realpath}"
 #The pod directory of the current project
 pod_path = "#{path_root}/#{project}/Pods"
 installer.pods_project.targets.each do |target|
 bunlde_name = target.name
 #Here you can selectively enable source code debugging based on environment variables
 enableDebug = ENV["#{bunlde_name}_DEBUGLIB"]
 enableDebug = true
 if enableDebug
 DebugLibCode.new().link(lib_file,target_path,basename)
 end
 end
 end

The whole process is shown in Figure 3:

Figure 3

3.1.3 Solution Optimization

Although the above scripts have achieved the debugging of binary static libraries, new problems have been encountered during their promotion and use:

1) Every developer will get an error when executing the binary debugging script for the first time because of permission issues. The developer needs to manually create a cbuilder user directory under Users.

2) The time for each pod install has become much longer. After multiple measurements, on a computer with an M1 chip, the time from executing pod install without binary debugging to executing pod install with binary debugging increased by more than 60%; on a computer with an Inter chip, the time increased by more than 70%, as shown in Figure 4:

Figure 4

For question 1, it is unreasonable for developers to manually create the cbuilder user directory. We integrated this operation into ZTPodTool (ZTPodTool is a podfile management tool we developed, which will be introduced in detail below), and let ZTPodTool create the cbuilder user directory, so that developers can develop without noticing. However, after trying various directory creation APIs, we found that none of them could create this directory. This problem has troubled us for a long time.

After searching a lot of information, I found that AppleScript is a scripting language that is closely integrated with macOS. Its notable feature is that it can control other applications on macOS. By using it, some tedious and repetitive tasks can be completed. The code is as follows:

 NSString *script = @"do shell script \" /bin/mkdir -m 777 /Users/cbuilder\" with administrator privileges";
 NSError *errorInfo = nil;
 NSAppleScript *appleScript = [[NSAppleScript new] initWithSource:script];
 NSAppleEventDescriptor * eventResult = [appleScript executeAndReturnError:&errorInfo];

For question 2, we found that using the dwarfdump command to parse the binary file to obtain the source code path first loads the entire binary into memory, and then uses grep, head, cut and other commands to parse the source code path directory. This process is very time-consuming. The more pod component libraries there are in the project, the more time-consuming this process is.

These time-consuming commands are just to get a path. If we can get the path through other means, we can save these time and save a lot of time. So we thought, since it is the path on the packager, let the packager save the package related information in the product directory with json when packaging. When installing, we can get the package source path by reading the json file in the product.

After optimizing the script, we measured that the time it takes to run pod install is almost the same as before (Figure 5). In this way, our developers can debug the code of each component without any difference.

Figure 5

3.2 Another way to solve the M1 computer iOS simulator clipboard problem

People who use M1 series computers to develop on iOS simulators will basically encounter a very tricky problem, that is, the simulator's clipboard cannot communicate with the computer's clipboard, and developers cannot assign values ​​to the clipboard. Once a value is assigned, an error will be reported:

 [CoreServices] _LSSchemaConfigureForStore failed with error Error Domain=NSOSStatusErrorDomain
 Code=-10817 "(null)" UserInfo={_LSFunction=_LSSchemaConfigureForStore, ExpectedSimulatorHash=
 {length = 32, bytes =0x4014b70c 8322afc9 dfb06ed8 13148b48 ... b6adae0d b2637192 }, _LSLine=
 405, WrongSimulatorHash={length = 32, bytes = 0x073253e6 9a9b67cc 089d6640 ca4fdb3e ...
 46b00d8b bca98999 }}

When I reported this issue on Apple's official forum, the response I got was this:

//https://developer.apple.com/forums/thread/682395

So far I've been ignoring this thread because it started out with folks running Xcode and the simulator under Rosetta. This isn't a supported configuration and I recommend that you switch to running these natively.

However, it's now clear that multiple folks are hitting this while running Xcode and the simulator natively. Have any of you filed a bug about this? If so, please post your bug number?

If the clipboard is not available, entering addresses or long text in the simulator is very time-consuming and painful for iOS, RN and H5 developers. For this reason, we have come up with a simple way to bypass this system bug and perfectly solve this problem. The main process is as follows:

Figure 6

We have customized the shortcut key Ctrl + V for our APP to trigger the user to paste the content:

 - (NSArray<UIKeyCommand *> *)keyCommands {
 NSArray *a = [super keyCommands];
 if (a) {
 NSMutableArray *commands = [NSMutableArray arrayWithArray:a];
 [commands addObject:[UIKeyCommand keyCommandWithInput:@"v"
 modifierFlags:UIKeyModifierControl
 action:@selector(posteboardCommand:)
 discoverabilityTitle:@""]];
 return commands;
 }
 return @[
 [UIKeyCommand keyCommandWithInput:@"v"
 modifierFlags:UIKeyModifierControl
 action:@selector(posteboardCommand:)
 discoverabilityTitle:@""]
 ];
 } 

We have developed a Mac client for local services. Its main function is to start a local Http service to handle requests for obtaining the current computer clipboard content. It is displayed on the system status bar, which is convenient for controlling the start, stop and exit of the service, and supports modifying the port number (Figure 7). Click here to download and use it.

Figure 7

The code to get the current input box is as follows:

 @interface UIResponder (FirstResponder)
 + (id)currentFirstResponder;
 @end


 static __weak id currentFirstResponder;


 @implementation UIResponder (firstResponder)


 +(id)currentFirstResponder {
 currentFirstResponder = nil;
 [[UIApplication sharedApplication] sendAction:@selector(findFirstResponder:) to:nil from:nil forEvent:nil];
 return currentFirstResponder;
 }


 -(void)findFirstResponder:(id)sender {
 currentFirstResponder = self;
 }
 @end
 //The operation to trigger the acquisition of the clipboard is as follows:
 - (void)posteboardCommand:(UIKeyCommand *)command
 {
 #if TARGET_IPHONE_SIMULATOR
 NSURLSession *session = [NSURLSession sharedSession];
 NSURL *url = [NSURL URLWithString:@"http://127.0.0.1:8123/getPasteboardString"];
 NSMutableURLRequest *request = [NSMutableURLRequest requestWithURL:url];
 request.HTTPMethod = @"GET";
 NSURLSessionDataTask *task = [session dataTaskWithRequest:request completionHandler:^(NSData * _Nullable data, NSURLResponse * _Nullable response, NSError * _Nullable error) {
 dispatch_async(dispatch_get_main_queue(), ^{
 if (error) {
 NSLog(@"Read error, please check whether the service is open");
 } else {
 NSString *pasteString = [[NSString alloc]initWithData:data encoding:NSUTF8StringEncoding];
 UIResponder* aFirstResponder = [UIResponder currentFirstResponder];
 if ([aFirstResponder isKindOfClass:[UITextField class]]) {
 [(UITextField *)aFirstResponder setText:pasteString];
 } else if ([aFirstResponder isKindOfClass:[UITextView class]]) {
 [(UITextView *)aFirstResponder setText:pasteString];
 } else {
 }
 }
 });
 }];
 [task resume];
 #endif
 }

After adding this optimization, developers only need to use Ctrl + V to paste the contents of the Mac's clipboard into the input box of the iOS simulator, which is exactly the same as the normal copy and paste function experience.

3.3 Develop visualization tools and integrate various functions

The increasing number of components makes podfile operations more complicated, component bundles need to be packaged more frequently, and test packages take longer to create. In order to simplify the tedious operations of terminal command input, component package creation, and APP test packages, we developed a visualization tool, ZTPodTool, Figure 8. This tool can not only directly display the dependency hierarchy between components, but also directly submit component package creation requests on the tool, without having to frequently switch pages on the browser packaging platform.

Figure 8

On ZTPodTool, you can not only conveniently switch between source code and binary of each component and build component packages, but also support building test packages (Figure 9).

Fig. 9

When the developer clicks the install button, ZTPodTool will assemble the command according to the user's source code settings, and then automatically open a terminal that is more friendly to display logs, allowing the terminal to execute the command. Although the pod install command can also be executed through NSTask and NSPipe, the obtained StandardOutput log cannot be highlighted, which looks very painful. If it can be executed directly in the terminal, it will be more friendly to developers. After consulting Apple documents, it was found that the official did not provide a "terminal" SDK for developers to use. At that time, how to invoke the terminal to execute commands through other means became a problem that must be solved.

Finally, I solved this problem by using the AppleScript mentioned above. Here are two ways to call AppleScript:

 //Method 1
 NSTask* task = [[NSTask alloc] init];
 task.launchPath = @"/usr/bin/osascript";
 task.arguments = @"tell application \"Terminal\" to do script \" pod install --repo-update";
 task.currentDirectoryPath = @"/Users/zhangsan/iosWorkSpace";
 task.environment = @{
 @"LANG":@"zh_CN.UTF-8",
 @"PATH":@"/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/git/bin:/usr/local/"
 };
 [task launch]; 

 //Method 2
 NSError *err = nil;
 NSAppleScript *appleScript = [[NSAppleScript new] initWithSource:@"tell application \"Terminal\" to do script \" pod install --repo-update"];
 NSAppleEventDescriptor * eventResult = [appleScript executeAndReturnError:&err];

We have added more user-friendly features including:

• After receiving the test package packaging completion message, developers usually send the installation package QR code to testers for verification. On ZTPodTool, we support automatic sending of the package QR code to the selected testers after packaging, without the need for developers to notify again.
• As the number of components in the list increases, it takes more time for developers to find and choose the components they maintain. For this reason, we support the watch list function, so that developers can only see the components they are concerned about.
• Enter the version number on ZTPodTool to update the version of each pod component.
• Automatically open the project after installation is complete

3.4 Optimize the packaging process to generate test packages faster

The biggest pain point of binary packaging is that you need to package the independent component binary before packaging the ipa. Multiple components are dependent, and you need to package the dependent bundles in series. The overall packaging process is time-consuming. In the testing phase, if the test package can be quickly printed, it will undoubtedly significantly improve the efficiency of bug acceptance. Every time we submit the code, the packaging process is like this (Figure 10):

Fig.10

No matter which packaging method is used in the above process, the test package can only be generated after the component package is completed. The time to generate the test package depends on the number of component packages and the dependencies between components. Is there any way to shorten this process? When we develop locally, the compilation is very fast, but when it comes to generating the test package, we have to generate the component package before generating the test package. If the packager can also customize part of the source code compilation, then there is no need to wait for the component to be compiled first. This directly saves the time of generating the component package and enables faster packaging.

To enable the packager to support partial source code packaging, the podfile file must be configured first, but developers cannot submit changes to the podfile, which will cause git conflicts. So we took a different approach and used the component names that need to be turned into source code dependencies as part of the parameters of the packaging network request. The packaging platform writes these parameters into the environment variables when packaging, and then modifies the packaging script to read these parameters before starting to execute pod install. If there are components that need to be compiled from source code, modify the podfile according to the parameters. This operation allows the packager to support it perfectly.

To improve this function, after the developer clicks on the package, they can choose whether to package the components at the same time. Combined with the function of automatically notifying the tester after packaging mentioned above, the current process is as follows (Figure 11):

Fig.11

From the simplified process above, we can see that we have changed the original serial tasks into tasks that can be executed in parallel. After multiple experimental comparisons, excluding the interference of packaging queues, the average packaging time for all component bundles is 203 seconds, the time for testing the entire bundle is 367 seconds, and the time for packaging some source codes is 384 seconds. Therefore, the packaging efficiency in an ideal environment is improved by 32.6%.

• The original total packaging time is: 203 + 367 = 570
• Packaging efficiency is improved to: (570-384)/570 = 0.326315

Considering the actual situation, after the component bundle is packaged, the developer or tester will receive a notification before packaging the test package on the packaging platform, and will also need to check some configuration information. If multiple component bundles are to be packaged, and there are dependencies between the components, it will take more time to package the test package, while source code packaging is basically not affected by component dependencies. Therefore, this optimization makes the packaging efficiency far exceed 32.6%.

IV. Conclusion

Whether it is architecture evolution, process optimization or tool production, engineers always hope to use technical means to reduce duplication of work and improve labor efficiency. Due to space constraints, many problems and solutions encountered in the process of these optimizations are not listed. There are still some known problems that have not been solved. These known problems are the driving force for our continuous optimization, and we believe that we can bring developers a better development experience. ​