Product operation: retention and loss analysis!

The value of user retention is self-evident. Even if a product has high user growth, if users do not stay and complete the core behaviors defined by the product, it is ineffective growth. Only when users stay can the product continue to generate revenue. Products such as communities and games also need to retain a sufficient number of users to maintain the product ecosystem. Therefore, user retention needs to be reviewed and analyzed frequently.

So how can we conduct retention analysis more efficiently and systematically? This article summarizes the system ideas as follows:

Step 1: Observe retention data

Observe the retention data and look at the user's short-term, medium-term, and long-term retention rates, which correspond to the three stages of the user life cycle of the retention curve (oscillation period, selection period, and stable period).

Figure 1. User life cycle of the retention curve

The retention rate at different periods reflects the status of products and users at different stages.

How long the retention rate lasts in each of the three periods is related to the natural usage cycle of the product, that is, the frequency with which users use the product. For example, for products that are used every day, such as Meituan Waimai, Zhihu, and Douyin, the short-term retention rate mainly focuses on the next-day retention. For products that are only used every week or every few weeks, such as Maoyan Movies, Damai.com, and 12306, the weekly retention rate is focused.

There are three specific ways to determine the natural life cycle of a product:

1. Based on business experience

As described in the above example, this method is relatively simple and can be used to judge without historical data.

2. Analyze daily retention rate

The specific approach is to take new users over a period of time and observe their daily retention rate within 30 days. The first retention peak after the first-day retention rate is the product usage cycle, which can be verified by the periodicity of subsequent retention peaks.

Figure 1. Retention rate of new users on a certain day (virtual data)

Note:

N-day retention of new users added on a certain day = the number of active users on that day on the Nth day / the number of new users added on that day

For example, the number of new users on January 1 is 100, and the number of active users on the third day is 60. Then the active retention rate on the third day = 60/100 = 60%

Note that if the product's usage cycle is daily, there is no obvious retention peak, and the daily retention rate shows a trend of gradually decreasing and then tending to be flat.

The reason for observing the retention rate within 30 days is that if the usage cycle of a product exceeds 30 days, there is generally no need to focus on user value through retention. Instead, the focus is on the user's usage experience, and users are driven to become loyal users through membership mechanisms and building brand awareness, such as Beike and Huolala.

3. Churn regression rate curve

The churn return rate curve can help us reasonably define a user. It determines how long a user has to wait before they return to the product to be considered a churned user. The product usage cycle must be within this time span, which defines the upper limit of the product usage cycle.

Churn regression mainly refers to churned users logging into the product again. Churned users are users who have not logged into the product for a period of time. This period of time is called the churn period.

Lost return rate = number of returning users / number of lost users * 100%

By calculating the churn regression rate at different churn periods, we can derive a churn regression rate curve. As the churn period increases, the user regression rate decreases. When the churn period exceeds a certain point, the decline in the user regression rate will be greatly reduced and tend to be flat. This point is the inflection point of the curve. As shown in the figure below, the inflection point is the 10th day, which means that when a user has not logged in to the product for 10 consecutive days, it can be judged that the user has actually churned, and the product usage cycle does not exceed 10 days.

Figure churn regression rate curve

Step 2: Identify retention issues

The key to determining whether there is a retention problem is to do a comparative analysis and compare it with different retention standards. If it is lower than the relevant standard, there may be a problem and further analysis is needed. The comparison criteria can be divided into the following three categories:

1. Time Standard

Compared with its own historical performance, it should be noted that lower than historical performance is not necessarily a problem. It is necessary to consider whether the product retention has cyclical characteristics. For example, the retention of new users of game products on weekends is usually lower than the retention of new users on weekdays because the users are more widespread.

2. Program Criteria

Product retention plans are usually developed in conjunction with the product's business goals.

3. Specific standards

It is usually based on business experience or comparison with industry/competitor data.

In addition, it should be noted that even if the overall retention rate meets the standard, it does not mean that there are no problems. The retention data of different channels/users of different values ​​can be split for further investigation.

Step 3: Retention Problem Analysis

Retention problems mean user loss, so it is necessary to further analyze who lost, when, where, and how. In order to answer these questions, it is necessary to segment lost users and locate loss points.

1. Lost User Segmentation

To break down which users are lost and lead to low retention, there are three ideas for user segmentation.

(1) Segmentation by user source

User sources include channel type, mobile phone system, etc. For example, if a certain channel has a large user loss, it may be because there is a problem with the advertising strategy of the channel, resulting in the import of too many new users, who are not the target users of the product.

(2) Segmentation by user value

There are two common methods of segmenting by user value: the pyramid model and the RFM model.

• Pyramid model: sort by user importance: celebrity users > professional users > contributing users > active users > ordinary users.
• RFM model: It comprehensively considers the three dimensions of the most recent consumption (Recency), consumption frequency (Frequency) and consumption amount (Monetary) to evaluate and rank user value. RFM is relatively complex, and the analysis can also be simplified according to business needs. For example, game products are generally divided into several levels based on the user's recharge amount, distinguishing between large recharge users and ordinary recharge users.

(3) Segmentation by user attributes

User attributes include 4 categories:

• Demographic characteristics: such as gender, age, occupation, education, etc.
• Social relations: marriage, presence of children, presence of elderly people in the family, etc.
• Behavioral characteristics: basic behavior (registration time, average daily usage time, etc.), business behavior (purchase of discounted products, etc.).
• Business related: such as the height, body fat percentage, and average daily steps of fitness product users.

2. Locating the loss point

Loss point location is to identify exactly when, where, and how users lose. This information can be used to find clues to the reasons for user loss.

(1) When did it disappear?

There are three possible scenarios for observing when users churn:

The first type: Loss during a period of using the product

As shown in the figure below, the retention rate of new users added during the period of March 21-27 on the 7th day is significantly lower than the expected level. At the same time, it can be observed that the retention decay from the 3rd to the 7th day is significantly faster than the expected trend (the lower the ratio, the faster the decay). Although the retention rate on the 15th and 30th days is also lower than expected, the decay trend is consistent with expectations. Therefore, it can be basically determined that the churn problem occurs around the 7th day when the user uses the product.

Figure Retention Rate Heat Map (Dummy Data)

The second type: loss at a specific point in time

As shown in the figure below, the retention rate of new users from March 21 to 27 on March 27 and 28 was significantly lower than the daily level. Further investigation can be carried out to see if there were any abnormalities on these two days, such as the inability to log in to the APP or improper operation activities affecting the user experience.

Figure Retention Rate Heat Map (Dummy Data)

The third type: The retention rate of new users on a certain day/period of time is continuously low

As shown in the figure below, the overall retention rate of new users added on March 25 is significantly lower than that of new users added at other times. This type of situation is usually a problem with the import of new users, such as abnormal advertising delivery. It can be analyzed in combination with the segmentation of the source of lost users to further locate the problem.

Figure Retention Rate Heat Map (Dummy Data)

(2) Where is the loss?

It mainly refers to which functional module of the product, or which stage of the process of using the product the user churns. It usually depends on the functional module or process where the user stays before churn. If it is found that most of the churned users stay in a certain place before churn, then you can focus on checking the corresponding functions or processes.

For example, for game products, it is found that a high proportion of churned players stayed in a certain mission copy before churn, then this mission copy may be the reason for the player churn.

(3) How to lose

Usually, you can analyze the operation logs of lost users before they churn to see if there are any abnormalities.

Step 4: Exploring the causes of churn

In the third step, clues about the reasons for user churn can be obtained by segmenting lost users and locating churn points. The fourth step is to dig deeper into the reasons for churn based on the clues obtained.

There are two ways to dig into the causes of churn:

1. Research

Those who have the conditions and resources can give priority to conducting research, including:

• General research inquiries: such as uninstalling research pages and questionnaires.
• In-depth investigation of causes: such as telephone, interviews, communities, and forums.

2. Hypothesis Testing

Research is generally labor-intensive, financially intensive and time-consuming. In most business analysis scenarios, it is not realistic to conduct research every time. Therefore, when conditions are limited, the second method, "hypothesis verification", can be used. That is, based on the clues obtained in the third step, make causal assumptions based on business experience, and then verify them with data.

For example, in the example of player loss in the above-mentioned game product, it was found that a high proportion of the lost players stayed in a certain quest copy before losing, and these players were all level 5 players. Based on business experience, it can be assumed that the quest copy was too difficult for these players, leading to the loss. This can be verified through the pass rate data of level 5 players in this quest copy. If the pass rate is obviously low, the hypothesis can be verified.

Step 5: Improve retention

After understanding the reasons for churn, how do you start to improve retention? Specific problems require specific analysis, but there is a general principle, as stated in the article "The Core Secret of Building a Billion-Dollar Product: User Engagement Hierarchy Model", which is to continuously increase the user's "benefits from continued use of the product" and "losses from leaving the product."

1. Increase the benefits of continued product use

(1) Demand satisfaction

On the one hand, we strive to more accurately match user needs. For example, for short video products, if the recommended videos do not suit the preferences of certain users and the needs of these users are not met, these users are prone to churn. On the other hand, we strive to meet users' longer-term needs, such as beauty camera products, and continuously optimize products and launch new ways of playing.

(2) User experience

When there are other products on the market that can meet the same user needs and provide a better user experience, users will not switch to other products. For example, both taxi-hailing apps can meet daily taxi needs and their services are similar, but the service experience may be different. Didi has more driver resources and responds to orders faster, so I prefer Didi to other taxi-hailing apps. This is also the barrier advantage that enables Didi to occupy the top position in the market for a long time.

2. Increase the “loss of leaving the product”

User investment: guide users to complete more key behaviors within the product. The essential idea is to let users invest more, including time, energy, and emotions, that is, to maximize the user's sunk cost. When people decide whether to do something, they not only consider whether it is good for them, but also whether they have invested in it in the past. Therefore, the higher the sunk cost, the more reluctant users are to give up the product.

For example, Evernote guides users to create a certain number of notes. The more notes users create, the higher the loss if they switch to other note-taking products.

In conclusion, the above sharing is based on work experience, amateur learning and personal thinking. If there are any shortcomings, please feel free to give me your advice.