Learn more about IOS9 every day 2: UI testing

Automated testing of user interface tools is very useful for software development. It can help you locate problems quickly. A successful testing process can give you confidence in the final release of the software. On the iOS platform, we use Automation to complete this work. This requires opening a separate application Instruments, and then writing and running JavaScript scripts. The whole process is painful and long.

UI Testing

In Xcode7, Apple introduced a new way to manage your app interface testing. UI testing allows you to find, interact with, and verify properties and states of UI elements. In Xcode7, UI testing is accompanied by test reports and runs together with unit tests. XCTest was integrated into the testing framework in Xcode 5, and in Xcode7, new testing capabilities for UI were added. It allows you to set assertions at specific points to view the state of the UI at that time.

Accessibility

In order for UI Testing to work, the framework needs to establish direct connections with many of your elements and then arrange the operations. You can define special points, or create tweaks on a certain UI, and then specify click or slide operations. But this will fail on devices of different sizes.

This is where accessibility comes in handy. Accessibility is a framework that Apple has long released, and is intended for people with certain physical disabilities (such as blindness) to use your app. It presents your UI to these users in a semantic way, allowing them to perform rich operations. You can (and should) make your elements Accessible. There are many reasons, such as custom controls, that cannot be automatically discovered.

UI Testing has the ability to provide solutions for testing devices of different sizes by providing accessibility features to your app. It also ensures that you don’t have to rewrite all the tests after reorganizing your UI. It can not only help you test your own UI, but also help your app better support people with certain physical disabilities.

UI Recording

Once you have your accessible UI set up, you will want to create tests for your UI. Writing UI tests is time-consuming, boring, and difficult if your UI is complex. Thanks to Xcode 7, Apple introduced UI Recording. It allows you to create tests in new projects or in existing ones. When you open it, test code is automatically created as you operate on the device or simulator. Ok, that's enough of the introduction, it's time to see how it works with an example.

Creating UI test examples

We will create an example using the UI Testing suite to show how UI Testing works. The final demo can be downloaded from Github, and you can follow along to practice and view the results.

create

In Xcode 7, when you create a new project, if you choose to include UI Tests, a new target will be created for you and you can set up all the configurations you want in the popup box.

This project is very simple, but it is enough to help us demonstrate how UI Testing works in Xcode 7.

Here is a menuViewController, which contains a switch and a button. Clicking the button can push to the detailViewController page. When the switch is off, push is prohibited. The detail page has a button and a label. Clicking the button can increase the value of the label.

Using UI Recording

Once the UI controls are created and the methods are written, we can write unit tests to ensure that changes to the code do not affect the effectiveness of the methods.

The XCTest UI Testing API

Before we record the test action, we need to decide where to put the assertions. In order to be able to test our UI, we can use the XCTest Framework, which has now been extended with three new APIs.

• XCUIApplication This is the proxy for the application you want to test. It can start your application and start it in a new process every time. This may take a little time, but it means that every time you test your application, it will complete the work that needs to be processed and maintain a clean, brand new state.
• XCUIElement This is a proxy for the UI element of the application you want to test. Elements have types and unique identifiers. You can use them together to find where your elements are. These elements are combined in a tree structure to form the presentation of your application.
• XCUIElementQuery is used when you want to find an element. Each XCUIElement is based on a query. The query must find the corresponding element in your element tree, otherwise it will fail. The exception message will indicate that it does not exist. You can check whether it is displayed in the tree. XCUIElementQuery is universal. If you look for an element supported by accessibility, the query will return a set of results.

Now we are ready to write a test that will further explain the mentioned APIs.

Test 1 - Ensure navigation does not work when the switch is off

First we define a test method.

 func testTapViewDetailWhenSwitchIsOffDoesNothing() { 
 
 }

After defining the method, we move the cursor into the method and click the record button at the bottom of the Xcode window.

Now that the app is running, tap the switch to turn it off, then tap the View Detail button. The following code will be automatically inserted into the testTapViewDetailWhenSwitchIsOffDoesNothing method.

 let app = XCUIApplication() 
 
 app.switches[ "View Detail Enabled Switch" ].tap() 
 
 app.buttons[ "View Detail" ].tap()

Now click the record button again and the recording will stop. You can see that the detailViewController page is not actually pushed. But the test doesn't know this at this time, so we need to add an assertion to check that there is no change. We can compare the title value of the navigation bar, which is not elegant, but it is enough for the current demonstration.

 XCTAssertEqual(app.navigationBars.element.identifier, "Menu" )

After adding this assertion, you will find that the test still passes. If you change the title of the navigation bar to "Detail", you will find that the test will not pass. Here is the final test code, with some comments explaining the behavior.

#p#

Test 2 - Ensure navigation works properly when the switch is on

The second test is very similar to the previous one, so we won't go into detail. The only difference is that the switch is enabled, so the app loads the details page to the screen. The XCTAssertEqual method is used to verify whether it is correct.

Test 3 - Ensure that the increase button actually increases the value of the label

In this test, we want to verify that clicking the increase button will increase the value of the label by 1. The first two lines of code are very similar to the previous example, so we copy them.

 let app = XCUIApplication() 
 
 // Tap the view detail button to open the detail page.   
 
 app.buttons[ "View Detail" ].tap()

Next we need to get the button, we will click it a few times. So we need to make the button a variable. We don't have to write the code and debug it manually. Record again and click the increase button, which will automatically add the following code for you.

 app.buttons[ "Increment Value" ].tap()

We stop recording and change the code to the following.

 let incrementButton = app.buttons[ "Increment Value" ]

This approach saves us from having to write code manually, in the same way we get the label variable.

 let valueLabel = app.staticTexts[ "Number Value Label" ]

Now we have the element of interest that we can interact with. In the following test, we test clicking the button 10 times and see if the value of the label increases. We could record it 10 times, but since we have the variable, we can write a loop to test it.

These three tests are far from a complete test, but they show you a good start that you can easily expand. Why not write a test of your own to practice, such as verifying that when the button is enabled, you can successfully navigate when the switch is disabled?

When a recording error occurs

Sometimes you find that you clicked on an element while recording, but the generated code doesn't look right. Usually this is because your element is not enabled for Accessibility. To determine if this is the case, you can open Xcode's Accessibility Inspector.

Once the Accessibility Inspector is open, if you press CMD+F7 and hover over an element, you can see the full element information below the hotspot. This can give you some clues when you can't find the element.

Once you have found the problem, you can open Interface Builder. In the Properties pane, find the Accessibility tab. It allows you to set the accessibility of an element. This is a powerful tool for setting the accessibility properties of your graphical interface.

When a test fails

When a test fails, if you are not sure why? There are many ways to fix the error. First, you can take a look at the test report.

When you open this view and hover your mouse over a step, you will find a small eye icon on the right side of the method. Clicking on this eye icon will give you a screenshot at that time, so you can clearly see the state of your UI at that time to find errors.

Just like unit testing, you can set breakpoints, allowing you to find problems more easily. You can output the UI hierarchy, element attributes, etc., and then find the cause.

Why UI Testing?

UI automated testing is a great way to increase confidence and provide quality assurance when making changes to your app. We have seen how easy it is to add UI tests and run them in Xcode. Not only does it help you find problems, but it can also help people with disabilities use your app.

One particularly nice feature of Xcode is the ability to test your app from a continuous integration server. This allows you to leverage Xcode's robots to run tests, and from the command line means that if a test fails, you'll be notified immediately.