iOS view, exploration of animation rendering mechanism

When developing a terminal, the first thing to consider is the rendering of views and animations, switching, etc. The most direct experience of users when using an app is whether the interface looks good, whether the animations are dazzling, and whether the sliding flow is smooth. The UI is the facade of the app, and its experience accompanies the entire process of the user using the app. If the UI fails, the user will not have the desire to open the app a second time.

iOS provides developers with a rich framework (UIKit, Core Animation, Core Graphic, OpenGL, etc.) to meet various development needs from top to bottom. I have to say that Apple is awesome. You don't need to understand the principles behind many interfaces to get started and meet most of your needs. However, if you encounter performance problems, it's easy to be at a loss. Ease of use and optimization are contradictory. Just like ARC, it's very enjoyable to use when you don't encounter memory problems. Once you encounter them, you will be required to understand the memory mechanism of iOS better than when using MRC. The same is true for UI. Moreover, as an employee of Tencent, of course we can't just know how to use it. We must know the whys and the whys. Okay, without further ado, let's get to the point: see how iOS renders views and animations, and how to optimize when we encounter rendering performance problems.

(Note: The following content is some of the author's experience and summary, welcome to discuss!)

Let's first take a look at the official description of Core Animation:

It can be seen that the core of iOS rendering view is Core Animation. From the bottom to the top, it is GPU->(OpenGL, Core Graphics)->Core Animation->UIKit.

On iOS, animation and view rendering are actually done in another process (hereinafter we call this process render server). Before iOS 5, this process was called SpringBoard, and after iOS 6 it was called BackBoard.

The following picture shows the progress of the entire system when using the project to record a video (a large number of view rendering):

You can clearly see the status of the BackBoard process.

The various stages of view or animation rendering on iOS:

There are 4 stages inside the APP:

• Layout: In this stage, the program sets the View/Layer hierarchy information and sets the layer properties, such as frame, background color, etc.
• Create backing image: At this stage, the program will create the backing image of the layer, whether it is passed to the layer through setContents or drawn through drawRect: or drawLayer:inContext:. So drawRect: and other functions are called at this stage.
• Preparation: In this phase, the Core Animation framework prepares various attribute data of the layer to be rendered, as well as the parameters of the animation to be done, and prepares to pass them to the render server. At the same time, the image to be rendered is also decompressed in this phase. (Except for images loaded from a bundle using the imageNamed: method, which will be decompressed immediately, other images, such as those read directly from the hard disk or downloaded from the Internet, will not be decompressed immediately, and will only be decompressed when they are actually rendered).
• Submission: In this stage, Core Animation packages the layer information and the parameters of the animation to be performed, and passes them to the render server through IPC (inter-process communication).

2 stages outside the APP:

When this data reaches the render server, it will be deserialized into a render tree. Then the render server will do the following two things:

• According to the various properties of the layer (if it is animated, the intermediate values ​​of the properties of the animation layer will be calculated), use OpenGL to prepare for rendering.
• Render these visible layers to the screen.

If you are doing an animation, the last two steps will be repeated until the animation is finished.

We all know that the screen refresh rate of iOS devices is 60HZ. If the above steps cannot be completed within one refresh cycle (1/60s), it will cause frame drops.

Let's see which operations may consume too much CPU or GPU and cause frame drops.

• There are too many layers or geometries on the view:
  If the view hierarchy is too complex, the CPU will spend more time recalculating the frame when some views are rendered or the frame is modified. This is especially true if autolayout is used. At the same time, too many geometric structures will greatly increase the number of OpenGL triangles that need to be rendered and the rasterization operations (converting OpenGL triangles into pixels).

• Too much overdraw: Overdraw means that a pixel is filled with color multiple times. This is mainly caused by some semi-transparent layers overlapping each other. The GPU fill-rate (the rate at which pixels are filled with color) is limited. If there is too much overdraw, it will inevitably reduce the performance of the GPU.

• Delayed loading of views:
  iOS will only load a view when it displays the view of a viewcontroller or accesses the view of a viewcontroller, such as someviewcontroller.view. If the user clicks a button and the button's response function does a lot of CPU-intensive work, then presenting a viewcontroller at this time will easily cause lag, especially if the viewcontroller needs to get data from the database, initialize the view from a nib file, or load an image.

• Off-screen drawing: There are two situations for off-screen drawing: 1. Some effects (such as rounded corners, layer masks, drop shadows and layer rasterization) cannot be drawn directly to the screen, and must first be drawn to an offscreen image context. This operation will introduce additional memory and CPU consumption. 2. After implementing drawRect or drawLayer:inContext:, in order to support arbitrary drawing, core graphic will create a backing image of the same size as the view to be drawn. And when the drawing is completed, it must be transferred to the render server for rendering. So don't overload drawRect and other functions without doing anything.

• Image decompression: Use imageNamed: to decompress images when loaded from a bundle. Generally, images are decompressed when they are assigned to a UIImageView image or layer contents or when they are drawn into a core graphic context.

Notes on rendering performance optimization:

Hidden drawing: Both CATextLayer and UILabel draw text into the backing image. If the frame of a view containing text is changed, the text will be redrawn.

Rasterize: When using the layer's shouldRasterize (remember to set the appropriate layer's rasterizationScale), the layer will be forced to draw to an offscreen image and cached. This method can be used to cache layers that are time-consuming to draw (such as those with more gorgeous effects) but are not often changed. If the layer changes frequently, it is not suitable.

Off-screen drawing: Use rounded corners, layer masks, and drop shadows effects by using stretchable images. For example, to achieve rounded corners, assign a circular image to the content property of the layer. And set the contentsCenter and contentScale properties.

Blending and Overdraw: If a layer is completely covered by another layer, the GPU will optimize and not render the covered layer, but calculating whether a layer is completely covered by another layer is very CPU-intensive. Blending the colors of several semi-transparent layers together is also very CPU-intensive.

What we are going to do:

1. Sets the view’s backgroundColor to a solid, opaque color.

2. If a view is opaque, set the opaque property to YES. (Directly tell the program that it is opaque, rather than asking the program to calculate it)

This will reduce blending and overdraw.

If you use an image, try to avoid setting the image's alpha to transparent. If some effects require the fusion of several images, let the designer draw it with one picture, and don't let the program merge them dynamically when running.

OK, now that we have introduced some points to note about rendering optimization, let’s look at some specific examples using the Core Animation instrument and the GPU driver.

Core Animation:

• Color Blended Layers: Check the coverage of the semi-transparent layer. From green to red, the redder the color, the greater the coverage.

The following two pictures show the measurement of the semi-transparent layer of the project details page. You can see that there are still many semi-transparent layers on the details page, but it does not mean that there are many semi-transparent layers and a large range of them needs to be optimized. You need to refer to the measurement of the GPU driver to see, which will be introduced below.

Color Hits Green and Misses Red: When using shouldRasterize, the layer drawing will be cached. If the rasterized layer needs to be redrawn, it will be marked in red.

Below is the case where the cell of the Kugou message list is set to shouldRasterize = YES. The backing image of each cell's layer will be cached. If the tableview is scrolled down, the cell will be reused, so the layer will be redrawn and marked in red.

Color Offscreen-Rendered Yellow: If you want to do offscreen drawing, it will be marked in yellow.

Below is the homepage of the project. Since the circular avatar is implemented by setting the property of roundedCorner, it will trigger offscreen drawing.

The Core Animation template only allows developers to intuitively see which areas may need optimization, but whether to optimize or not still depends on the performance of the GPU driver.

GPU Driver

Renderer Utilization - If this value is greater than 50%, it means that the GPU performance is limited by fill-rate, there may be too much Offscreen rendering, overdraw, and blending.

Tiler Utilization - If this value is greater than 50%, it means there may be too many layers.

Let’s take the project details page above as an example and look at the GPU driver measurements:

You can see that the Renderer Utilization is about 20%, and the Tiler Utilization is 15%, so there is no need for optimization. (Of course, we are not considering the CPU usage here, but only based on some measurements of the core animation template above, we can say that there is no need for targeted optimization.)

What I want to say here is that the above points only provide us with optimization directions and ideas when we encounter rendering performance problems. Optimization often means more complex and difficult to understand code. Don't over-optimize when you don't encounter rendering performance problems.

I hope this helps you all!