Ctrip flight cross-terminal cross-framework UI automation testing solution Flybirds

Author: Liang, senior development expert at Ctrip, author of open source projects, focusing on DevOps, automation frameworks, tool platforms, etc.

1. Background

Multi-terminal R&D is an unavoidable topic for front-end development in today's era. In order to solve these problems, many development solutions have been launched in the industry, but there are not many solutions for cross-terminal UI automation testing.

Flybirds has been open source for more than three months since the beginning of 2022. Through communication and feedback from active users in the community, it has launched the v0.2 version of the cross-terminal and cross-framework testing solution. A set of scripts can run on multiple terminals, and the plug-in architecture design also makes it convenient for community developers to freely join in the expansion and grow together.

2. New features in v0.2

• Support multi-terminal script reuse
• Supports multiple browser rendering engines: Chromium, WebKit and Firefox
• Support compatibility testing in multiple browsers concurrent mode

3. What kind of multi-terminal testing solution do we need?

In recent years, many new development frameworks have appeared every once in a while, which have brought better development experience and performance, but also created many challenges for automated testing.

What kind of multi-terminal testing solution do we really need? From Flybirds' perspective, we hope that multi-terminal testing will not become an obstacle in the R&D process, especially when the multi-terminal ecosystem as a whole is thriving, the automated testing solution should grow together with the development solution.

Whether it is the Web, React Native, or Native side, the ideal solution should be multi-end adaptation, retain good extensions, take into account more frameworks, and be jointly built by the community to promote the prosperity of the overall ecology. Therefore, Flybirds provides the community with a cross-end and cross-framework testing solution.

4. Plug-in architecture

The plug-in architecture helps us separate the capabilities of each end. The plug-in provides the components, APIs, and configurations required for runtime, and Flybirds injects them into the corresponding life cycle respectively.

4.1 File Structure

 ├─ cli scaffolding
 ├─ core
 | ├─ config_manage.py configuration management
 | ├─ dsl
 | │ ├─ globalization ​
 | │ └─ step Step list
 | ├─ global_resource.py global configuration
 | ├─ launch_cycle lifecycle management
 | └─ plugin
 | ├─ event management
 | ├─ plugin_manager.py plugin management
 | └─ plugins
 | ├─ android Andriod related processing
 | ├─ ios iOS related processing
 | └─ web Web related processing
 ├─ report ​
 ├─ template processing
 └─ utils

4.2 DSL step list

Of course, in this architecture, each end is slightly different, mainly due to the platform differences of each end. The following is a list of DSL steps supported by each end. Most steps can be applied to multiple ends.

5. Multi-terminal application examples

5.1 Test Cases

 Function : Passenger module

 @ p1 @android @web ​ ​
 Scenario : Exposed passengers_Select passengers on list page
 When the page jumps to [ One-way filling page ]
 Then the page rendering is complete and the element [ selected passenger name ] appears
 Then the text of [ Select Passenger Text ] is [ Select Passenger ]
 Then the text for [ Selected Passenger Name ] is [ Li Yifeng ]
 Then the text for [ Selected Passenger ID Type ] is [ Passport ]
 Then the text of [ Selected Passenger ID Number ] is [ YHE77 ]
 Then there is an element of [ passenger type label children ]
 Then return to the previous page

5.2 Page Object Management

Page object management in multi-terminal projects is unified through json files. There are usually two situations:

1) When the two ends are the same, refer to the following configuration

 // Element positioning configuration ele_locator.json
 {
 "Select the boarding document" : "testid = passger_check" ,
 "Selected Passenger Name" : "testid = passger_name_checked" ,
 "Selected Passenger ID Type" : "testid = passger_ct_checked" ,
 "Selected Passenger ID Number" : "testid = passger_cn_checked"
 }

2) Different terminals can be distinguished by Android, iOS, and web

 // scheme configuration schema_url.json
 {
 "One-way filling page" : {
 "android" : "urlschemel://auth_activity" ,
 "ios" : "urlschemel://ios_auth_activity" ,
 "web" : "https://address"
 }
 }

 // Element positioning configuration ele_locator.json
 {
 "passenger_type_label_children" : {
 "android" : "textid=passger_type_child" ,
 "ios" : "lableid=passger_type_child" ,
 "web" : "xpath=//html/body/div"
 }
 }

6. Data-driven parameterization

In actual projects, most automated tests are based on data-driven parameterization, so they also need to be used in conjunction with "scenario outline + examples". Here we modify the above example:

 Function : Passenger module

 @ p1 @android @web ​ ​
 Scenario outline : Exposed passenger_Select passenger on list page
 When the page jumps to [ One-way filling page ]
 Then the page rendering is complete and the element [ selected passenger name ] appears
 Then the text of <element> is <title> ​
 Then there is an element of [ passenger type label children ]
 Then return to the previous page

 example :
 | element | title |
 | Select passenger copy | Select passenger |
 | Selected passenger name | Li Yifeng |
 | Selected Passenger ID Type | Passport |
 | Selected passenger ID number | YHE77 |

7. Multi-browser concurrency

Relying on PlayWright's cross-browser capabilities, Flybirds supports all modern rendering engines, including Chromium, WebKit and Firefox.

Flybirds supports multi-browser concurrent mode, which makes browser compatibility testing convenient and efficient.

7.1 Configuration Parameters

 // browserType: configure browser kernel
 "web_info" : {
 "headless" : true ,
 "browserType" : [ "firefox" , "chromium" , "webkit" ],
 "defaultTimeout" : 30
 },

7.2 Executing commands

 #You can specify the browser kernel through parameters
 flybirds run -D browserType = webkit , firefox , chromium

8. Pre-operation inspection

Next, before starting to run, please check the running environment

8.1 Android, iOS

1) Make sure the configured test device can be connected normally

• Android: Execute the command adb devices to check whether the device list contains the test device.
• iOS: Take the tidevice library as an example, execute the tidevice list command to check whether the device list contains the test device.

2) Download and install the test package

Android: The framework will automatically download and install the test package through the packagePath configured in config (make sure that the phone has turned on "Allow installation from unknown sources"). You can also download and install it manually.

iOS:

• Please manually download the demo APP to install
• Open wdaproxy shell tidevice --udid $udid wdaproxy -B $web_driver_angnt_bundle_id -p $port

8.2 Web

Before executing the Web project, check whether the browser environment has been installed

 #View supported browsers
 playwright install --help ​

 #Running without parameters will install all default browsers
 playwright install

 #Install a specific browser by providing a parameter
 playwright install webkit

IX. Operation

You can easily switch execution environments, filter cases, and control the number of concurrent operations through the cli. Some common commands are listed below.

 #View the operations supported when running the project
 flybirds run -- help

 #Execute all feature files in the features directory
 cd { PATH_TO_PROJECT_FOLDER }
 flybirds run

 #Specify the feature set to be executed, which can be a directory or a specific feature file
 flybirds run - P ./features/test/demo .feature ​ ​ ​ ​ ​

 #Run scenes with specific tags. Separate multiple scenes with commas. A scene starting with ' - ' does not run scenes containing this tag.
 flybirds run - T tag1 , tag2 , - tag3 , tag4

 #Switch to the Web environment to execute the use case
 flybirds run -D platform = web ​

 #Switch to Android environment to execute the use case
 flybirds run -D platform = Android ​

 #Switch to iOS environment to execute the use case
 flybirds run -D platform = iOS ​

 #Execute the use case with tag android
 flybirds run -T android -D platform = Android ​ ​

 #Specify the maximum number of processes to be opened during concurrent execution. The default value is 4, which is only effective in the web environment
 flybirds run -- path features - p 5

10. Configuration

The following are the configuration items required before each end runs

• Android configuration items: deviceId, packageName
• IOS configuration items: platform, deviceId, packageName, webDriverAgent,
• Web configuration items: platform, browserType, headless

11. Custom Framework Extension

Flybirds' plug-in design mode retains good expansion, and we will open more in the future.

11.1 Modifying Extensions

If you want to modify the current extension in your project, you can replace the (app, device, element, app, step, screen, screen_record) below plugin with your local files and make corresponding configuration in "plugin_info.json".

For example, if you want to modify the screen.py file in the web:

• Create a py file locally and name it screen.py
• Add the following configuration in plugin_info.json web:

 "screen" : {
 "path" : "{local_path}/screen.py" ,
 "ns" : "screen.plugin.myextend"
 }

{local_path} is the local path, "ns" is the package name, pay attention to the uniqueness of the package name (the above package names are just examples and are not mandatory)

12. Internal Enhancement Package

For custom functions commonly used within the team, you can consider creating an extend package. Flybirds supports dynamic loading, and the package name contains "-flybirds-plugin". Ctrip Air Ticket has internally connected various DevOps tools in the enhancement package, which can be used after installation.

13. Continuous Integration

The command line execution mode provided by cli makes it very convenient to add various continuous integration tools.

Take Jenkins as an example:

 # Inside the jenkins shell command
 cd { PATH_TO_PROJECT_FOLDER }
 # Run
 flybirds run -P ./features/test/everything .feature -define platform = Android ​ ​ ​ ​ ​ ​ ​
 cp - R reports $WORKSPACE

14. Release Plan

We will release according to the SemVer version control specification. We will gradually add new features and optimize the code. You are very welcome to join our co-construction plan and put forward your valuable suggestions on GitHub, as well as any problems you encounter during use. We will also iterate a small version every week. You can also give us spiritual support here by clicking a Star.