Use Jenkins to build iOS/Android continuous integration packaging platform

Background Description

According to the project requirements, we need to build a unified packaging platform within the team to package iOS and Android projects. In order to facilitate the distribution of test packages within the team, we hope to generate a QR code after packaging is completed, so that experience users (product, operation, test, etc.) can directly install the test package by scanning the QR code with their mobile phones.

This requirement is universal, and basically all teams developing apps may use it. Therefore, I organized the entire process of implementing the requirement into this article, and truly achieved zero-based start, ready to fly and use out of the box. I hope it will be helpful to everyone.

GitHub address: https://github.com/debugtalk/JenkinsTemplateForApp

First of all, let me show you the overall effect after the platform construction is completed:

The platform mainly implements three functions:

• Regularly check the GitHub repository and automatically build and package if there is any update;
• After the build is successful, a QR code is generated based on the ipa/apk, and the QR codes of each version can be displayed in the historical build list. The corresponding version can be directly installed by scanning the QR code with a mobile phone;
• The build results (Artifacts, such as .ipa, .app, .apk, info.plist and other files) of the current build are displayed on the build results page for users in need to download.

Next, this article begins to give a detailed introduction to the complete implementation process of platform construction.

Install Jenkins

Jenkins relies on the Java runtime environment, so you need to install Java first.

There are many ways to install Jenkins. You can run the installation package for the corresponding system type, obtain the image through Docker, or run the war package directly.

I personally prefer to run the war package directly. After downloading jenkins.war, run the following command to start Jenkins.

 $ nohup java -jar jenkins_located_path/jenkins.war --httpPort=88 &  

If you do not specify httpPort, Jenkins uses port 8080 by default.

Jenkins plugin

Jenkins has many plug-ins that can extend various functions.

For the iOS/Android continuous integration packaging platform I built, I used the following plug-ins.

• GIT plugin
• SSH Credentials Plugin
• Git Changelog Plugin: Get the commit log submitted to the repository
• build-name-setter: used to modify the Build name
• description setter plugin: used to modify the Build description information and add a QR Code to the description information
• Post-Build Script Plug-in: Implement some additional functions by executing scripts after compilation is completed
• Xcode integration: iOS only (optional)
• Gradle plugin: Android only (optional)

The installation method is also relatively simple. Just search for the above plug-in on the Jenkins plug-in management page and click to install.

Create a Job

In Jenkins, build projects exist in the form of jobs, so you need to create a job for each project. Sometimes, there may be multiple branches in a project being developed at the same time. In order to build them separately, you can also create a job for each branch.

There are many ways to create a job. This time, you only need to create a Freestyle project type.

 Main page -> New Item -> Freestyle project

For a continuous integration packaging platform, each packaging consists of four steps: triggering the build, pulling code, executing the build, and post-build processing. Correspondingly, these configurations are also included in each Job.

Configure Git code repository

To build the project, it is necessary to configure the project's code repository. Since our project is currently hosted in a GitHub private repository, Git needs to be configured here.

Under the [Source Code Management] configuration section, if the GIT plugin was installed successfully before, the Git option will appear.

When configuring a Git code repository, there are three items that must be configured: the repository URL, the repository permission verification method (Credentials), and the code branches that the current job needs to build (Branches to build).

When configuring the Repository URL, you can choose either HTTPS URL or SSH URL. However, it should be noted that the Credentials must correspond to the Repository URL, that is:

• If the Repository URL is in the form of an HTTPS URL, then Credentials must use the GitHub username and password verification method; and, if 2FA (two-factor authentication) is enabled in GitHub, you also need to create a Personal access token in GitHub and enter this Personal access token as the password when entering the password.
• If the Repository URL is in the form of an SSH URL, you need to first create an SSH key pair on the server where Jenkins is located and add the public key to GitHub's SSH keys. Then, when filling in Credentials, select the SSH Username with private key verification method, fill in the GitHub Username, SSH private key, and the Passphrase set when creating the SSH key pair.

If you are still confused about the concept of Git permission verification, you can refer to "Git Permission Verification in Simple Terms".

When configuring Branches to build, you can use a variety of forms, including branch name (branchName), tagName, commitId, etc. Among them, the branch name form is the most commonly used. For example, if you build the master branch, fill in refs/heads/master, and if you build the develop branch, fill in refs/heads/develop.

In addition to the above mandatory configuration items for Git, sometimes you may need to modify the default configuration items of Jenkins according to the actual situation of the project.

A common situation is to modify the configuration of the clone.

In the default configuration of Jenkins, when cloning code, all historical versions of the code will be pulled, and the default timeout is only 10 minutes. This results in some projects, due to the large amount of code itself, many historical versions, and poor network environment, Jenkins simply cannot pull all the code within 10 minutes, and the task will be automatically terminated after the timeout (error status code 143).

The solution to this problem is also very simple. There are only two ways to solve it: either pull less code (do not get historical versions) or increase the timeout limit. The corresponding configuration is in Advanced clone behaviours:

• Shallow clone: ​​Do not obtain historical versions after checking this option;
• Timeout (in minutes) for clone and fetch operation: Overrides the default timeout after configuration.

Configuring build triggers

The code repository is configured, which means that Jenkins has access to the GitHub code repository and can successfully pull the code.

So when does Jenkins execute the build?

This requires configuring the build trigger strategy, that is, the build trigger. The configuration item is located in the [Build Triggers] column.

Triggers support multiple types, the commonly used ones are:

• Build periodically
• Build when a change is pushed to GitHub
• Regularly check for code updates and build if there are any updates (Poll SCM)

The build trigger selection is a composite option. If you select multiple types, the build will be executed when any type meets the build condition. If none of the types are selected, the Jenkins Job will not perform automatic builds, but you can manually trigger the build by clicking [Build Now].

The format of the timer (Schedule) is briefly described as follows:

MINUTE HOUR DOM MONTH DOW

• MINUTE: Minutes within the hour (0-59)
• HOUR: The hour of the day (0-23)
• DOM: The day of the month (1-31)
• MONTH: The month (1-12)
• DOW: The day of the week (0-7) where 0 and 7 are Sunday.

Usually you need to specify multiple values, in which case you can use the following operator (priority from top to bottom):

• *Adapts to all valid values. If an item is not specified, a * is used as a placeholder;
• MN adaptation value range, for example, 7-9 means 7/8/9 are all satisfied;
• MN/X or */X: use X as the interval;
• A,B,C: enumerate multiple values.

In addition, to avoid multiple tasks triggering builds at the same time, you can use the H character when specifying a time period. After adding the H character, Jenkins will randomly select a time point in the specified time period as the starting time, and then add the set time interval to calculate the subsequent time points. Until the next cycle, Jenkins will randomly select a time point as the starting time again, and so on.

To make it easier to understand, here are a few examples:

• H/15 * * * *: represents every 15 minutes, and the start time is uncertain. This hour may be :07, :22, :37, :52, and the next hour may be :03, :18, :33, :48;
• H(0-29)/10 * * * *: represents every 10 minutes in the first half hour, and the start time is uncertain. This hour may be :04, :14, :24, and the next hour may be :09, :19, :29;
• H 23 * * 1-5: Sometime between 23:00 and 23:59 every night on weekdays;

Configuration build mode

After the trigger strategy is configured, Jenkins will automatically execute the build according to the set strategy. However, how to execute the build operation still requires us to set it by configuring the build method.

The common construction method is to install the corresponding plug-in according to the specific type of the construction object, and then use the corresponding construction method. For example, if you are building an Android application, after installing the Gradle plugin, you can select Invoke Gradle script, and then use Gradle to build; if you are building an iOS application, after installing the Xcode integration plug-in, you can select Xcode, and then select Xcode to build.

The advantages of this method are simple operation, UI visualization, and it can quickly meet the needs when the scene is not complicated. However, the disadvantage is that it relies on the existing functions of the plug-in. If the scene is more complicated, a single plug-in may not be able to meet the needs, and other plug-ins need to be installed. In addition, some plug-ins may still have some problems, such as poor compatibility with certain operating system versions or XCode versions, which will make us passive when problems arise.

I personally prefer another way, which is to write a packaging script by myself, customize all the build functions in the script, and then execute it in Execute Shell. This method is more flexible, can meet the build requirements of various scenarios, and can quickly fix problems when they occur.

In addition, when building iOS applications, there is another point that requires extra attention, which is the configuration of the developer certificate.

If you use the Xcode integration plug-in to build, the configuration will be more complicated. You need to import the development certificate in Jenkins and fill in multiple configuration items. However, if you use the packaging script to build, the situation will be much simpler. As long as the developer certificate is installed on the computer where Jenkins runs, and the packaging command can work normally in the Shell, there will be no problem executing the packaging script in Jenkins.

Post-build processing

After the build is completed, the generated compiled product (ipa/apk) will be located in the specified directory. However, it is quite troublesome to install the ipa/apk file directly on the mobile phone. Not only do you need to transfer the installation package of several tens of megabytes when distributing the test package, but the experience user also needs to connect the mobile phone to the computer via a data cable when installing, and then use tools such as PP Assistant or Wandoujia to install it.

Currently, a more elegant way is to use platforms such as pgyer or fir.im. After uploading the ipa/apk file to the platform, the platform will generate a QR code. Then you only need to distribute the QR code link. After the experience user scans the QR code with their mobile phone, they can quickly install it, which greatly improves efficiency.

Upload the installation package file and generate a QR code

Both Dandelion and fir.im have corresponding Jenkins plug-ins. After installing the plug-in, you can upload the installation package in Post-build.

In addition to using the Jenkins plug-in, fir.im also supports command upload, and Dandelion also supports HTTP Post interface upload.

I personally recommend using the command or interface upload method and calling it in the build script. Flexibility is one aspect, and the greater advantage is that if the upload fails, you can retry, which is extremely necessary when the network environment is not very stable.

After Jenkins successfully uploads the installation package, the pgyer/fir.im platform will generate a QR code image and return the URL link address of the image in the response.

Display QR code image

Now that we have the URL link of the QR code image, how can we display the QR code image in the historical build list of the Jenkins project?

Here we need to use another plugin, description setter plugin. After installing this plugin, there will be a description setter function in the [Post-build Actions] column, which can set the description information of the current build after the build is completed. This description information will not only be displayed on the build page, but also in the historical build list.

With this premise, it seems that it is possible to display the QR code image in the historical build list. The most intuitive way is to use the HTML img tag and write <img url='qr_code_url'> into the build description information.

The idea of ​​this method is correct, but doing so will not achieve the desired effect.

This is because for security reasons, Jenkins uses the Plain text mode for all description information's Markup Formatter by default. In this mode, the HTML code in the build description information will not be parsed.

It’s easy to change, just go to Manage Jenkins -> Configure Global Security and change the Markup Formatter setting to Safe HTML.

After changing the configuration, we can use the HTML img tag to insert images in the build description.

There is one more point to add. If you use the Pyger platform, you will find that the QR code image returned after uploading the installation package is a short link. The amazing thing is that this short link is fixed (for the same account). This short link always points to the most recently generated QR code image, but the platform does not return the ***URL address of the QR code image in the response. In this case, it makes no sense to save the URL link of the QR code image after each build.

The solution is to download and save the QR code image locally through the returned QR code image short link every time the installation package is uploaded, and then reference the Jenkins address of the image in the build description information.

Collecting Compilation Artifacts

After each build is completed, many files are compiled and generated, but not all files are what we need.

Normally, we may only need some of the files, such as .ipa/.app/.plist/.apk, etc. At this time, we can collect these files separately and display them on the build page so that they can be downloaded when needed.

To implement this function, you need to add Archive the artifacts in the Post-build Actions column, and then specify the path of the artifact file using a regular expression in Files to archive.

After the settings are completed, after each build is completed, Jenkins will use the set regular expression to search and match in the Console Output. If the file is successfully matched, it will be collected.

Summarize

This article mainly introduces the basic concepts and implementation process of how to use Jenkins to build an iOS/Android continuous integration packaging platform. For the sake of length and reading experience, the execution commands, build scripts (build.py), and detailed configuration of Jenkins involved are not elaborated in detail in this article.

In order to achieve true out-of-the-box use, I extracted Jenkins' configuration files and build scripts to form a set of templates. You only need to import them into Jenkins, and then modify a small amount of configuration information for specific projects to run this continuous integration packaging platform and achieve the same functional effects as in the illustration at the beginning of the article.