Mobile app performance issues determine user retention

——China’s first in-depth analysis report on mobile app performance data

Fully open for download: Click (http://bbs.tingyun.com/forum.php?mod=viewthread&tid=45) to get the full version of the 2014 China Mobile Application Performance Management White Paper!

According to research statistics from Tingyun (tingyun.com): more than 74% of users will choose to remain silent, endure, or leave when faced with performance issues, and nearly 5% of real users will give up using the app after a performance issue in a mobile app causes a 10-second response delay. Generally speaking, the frequency and types of application performance issues are beyond imagination. Due to the existence of various factors such as network environment, device model, and development environment, there are more than 107 million combinations of application performance issues (derived from an integrated analysis of 5,079 different models, 1,172 operating systems, and 18 operators). You never know what the next performance issue that will cause user loss will be.

Tingyun R&D personnel said: "Compared with user churn, mobile application performance issues will cause more losses to users. For example, when the application crashes or errors occur, it will cause key business interruptions and revenue declines, which will further affect the product life cycle value; if the application request response time is long, it will lead to slow end-user experience and reduced user retention rate; if the application interaction performance is slow, then the page elements will load slowly, causing freezes or incomplete layout chaos."

About Crash: 2014 Mobile App Performance Data Is Not All That Good

2014 was a year of rapid development of mobile Internet. According to Tingyun's performance monitoring of 350 million active devices, when the crash rate of iOS exceeded 8‰ and the crash rate of Android exceeded 4‰, the number of active users showed a significant downward trend, and the average crash rate throughout the year exceeded the standard value, indicating that there were generally minor hidden dangers.

In September 2014, Apple launched iOS8, which has become the operating system with the highest crash rate, with a crash rate of over 25‰. Compared with other systems, iOS7X has stable overall crash performance, with a crash rate of less than 10‰.

Compared with iOS system, Android has a lower overall crash rate, among which only Android2X system has a crash rate higher than 10‰. After Android4.X version, the crash rate is stable at 2‰-4‰.

According to analysis, the reasons for the low overall crash rate of the Android system are: the stability of Android 4.X version has been significantly improved compared with previous versions, while the iOS 8 version is relatively unstable; secondly, in terms of update strategy, the iOS update push cycle is longer, while Android will push updates instantly; thirdly, due to the particularity of the language/system architecture, OC needs to directly face the underlying API, with a higher possibility of error, and is affected by the OS version, and the impact of hardware differences is greater than that of Java; finally, the iOS system is more restricted, such as memory, background, API restrictions, etc.

About errors: Most of them are failure to establish a connection, request timeout, 500

Mobile application errors, as the name implies, are a collection of network errors and HTTP (HTTP status codes) that occur in the production environment of the application, which can cause the application page to fail to display, affecting the user experience. According to statistics, connection timeouts account for the largest proportion of network errors, accounting for 29%; connection failures rank second, accounting for 23%; illegal hosts (19%) and illegal URLs (13%) also account for a relatively high proportion. Among HTTP errors, "500" ranks first, accounting for 46%.

In terms of overall errors, connection timeout, connection establishment failure, 500 and unknown host are ranked high in mobile application errors, accounting for 6.3‰ of the overall error rate.

When faced with HTTP errors, Tingyun believes that the appropriate remedy should be given based on the error code, while network errors require analysis of the cause of the problem from the network level.

About application request response time: it will directly affect user retention

User retention rate is one of the key business indicators that both mobile Internet and Internet value the most. Retained users and retention rate directly reflect the quality of the application and the ability to retain users. When the application request response time is at different values, the application response status will be different. After Tingyun monitored the host request response time of more than 50,000 mobile applications, it was found that when the request response time exceeds 2000ms, users began to churn.

Regarding application interaction performance: it will directly cause users to lose patience

The time it takes for users to interact with the interface elements and content of mobile applications is called interactive performance time. The degradation of user experience caused by the long loading time of the application's page elements is called slow interaction. When the page loads slowly, it will cause the application to freeze. There are many reasons for the slow loading, such as interface layout, view loading, stored procedures, querying databases, program operations, etc. So how to find the bottleneck of interactive performance? Tingyun believes that it can be judged by querying the time used by the front-end UI thread or the background working thread. According to statistics, when the application interactive execution performance time reaches 400ms, the application enters the stage of slight performance risks.

Regarding operator network performance: There are large differences in different regions

With the development of 2G to 4G networks, users have put forward higher and higher requirements on the network performance of operators. According to Tingyun data statistics, when the response time of a 2G network reaches 4600ms, the network is judged to have a minor hidden danger. However, when the 3G and 4G network performance of the same operator enters a state of minor hidden danger, the corresponding network response time is shortened to 1200ms and 600ms.

It can also be seen that the network response time of different provinces under different networks of the same operator is also very different. The network performance of each segment of China Mobile is better in Heilongjiang and Tibet, but worse in southern cities such as Hong Kong and Fujian; China Unicom's 2G network performance is poor in Guangdong and Fujian, but 3G and 4G networks are better in northern cities such as Gansu, Beijing, and Hebei; the performance distribution of each segment of China Telecom is relatively uneven, and the provinces with better 2G and 3G performance are not as obvious as those of China Mobile and China Unicom, and are distributed in Tibet and Taiwan, while the better 4G areas are concentrated in central provinces such as Guizhou and Hubei.

From the response time itself, it can be analyzed that the best performance of 2G, 3G, and 4G networks are China Telecom, China Unicom, and China Telecom, respectively. Under the same 3G network, China Unicom and China Telecom are comparable, while under the 4G network, China Telecom's network performance is much better than China Unicom and China Mobile.