Silicon Valley Guide: How to systematically improve user retention?

Pinterest is one of the most popular social platforms abroad , recommending pictures, GIFs and videos to users in the form of waterfall flow. In the nine years since its establishment, its valuation has reached 12.3 billion US dollars, and its monthly active users exceed 200 million. The rise of this Silicon Valley unicorn was not due to luck – Pinterest has been adhering to the concept of growth hacking since its inception, constantly implementing " data-driven growth" and putting theory into practice.

Dannie Chu is a member of the founding team of Pinterest. For the past six years, he has led Pinterest's growth engineering team and has grown Pinterest's monthly active users from 20 million to over 200 million. In Dannie's view, user retention is crucial to building a long-term sustainable business, but it is often overlooked.

The following is Dannie’s personal account of his experience in improving user retention .

Over the past five years, I have led Pinterest's growth engineering team and spent a lot of effort to improve user retention , increasing Pinterest's monthly active users from 20 million to 200 million. Recently, I have started helping some startups , both very early and later stages, understand and achieve long-term sustainable growth. By helping these companies in different situations, I found that there is an important factor for startups to continue to grow that is often overlooked - healthy user retention . Most companies don’t realize how critical retention is to building a long-term business, and even if they do realize it, it’s difficult to actually improve user retention.

In this article, I will discuss why user retention is so important and some tools that are crucial to improving user retention. These tools can basically be directly applied at all stages of a company’s development.

Brian Balfour once wrote a great blog post titled “Why User Churn Is the Invisible Killer of Product Growth” (see Note 1 at the end of the article). This is true whether you are in the early stages of a startup and want to achieve large-scale user growth , or you already have hundreds of millions of users like Pinterest.

Problems with retention will prevent your product from growing truly fast, or will severely limit the long-term development space of your product. Twitter is a typical example of the second situation. Despite having a large number of registered users, their active user growth has stagnated. The main reason behind this is poor user retention.

Twitter's quarterly active user data, data from Credit Statistica

User retention is also very important for a company to achieve commercial success. For most toC or toB companies, a typical business model is as follows:

Total revenue = customer acquisition x retention x average order value

The company initially acquires customers through various paid or free methods, but only a portion of these customers will remain for the long term and ultimately generate profits for the company. For this reason, user retention is the core of business operations. Improving user retention can reduce the cost of acquiring a profitable customer and increase the user's lifetime value, which will ultimately bring overall profit growth to the company. You can imagine that if the user retention of a business is very low or even close to zero, then... Game Over.

Since user retention is so important to your business, how can you improve user retention in your product?

The first thing you need to determine is what kind of users are effective retained users for your business.

This truly effective and important indicator needs to be related to the business process of your product and be able to accurately measure whether the user has obtained useful value from your product. Only when users gain value will they continue to use your product at a certain frequency, thus forming retention.

For Pinterest, we choose to use weekly usage as the retention indicator because we think it is more reasonable for users to log in once a week to discover new things in their lives.

If you are making a communication tool like WhatsApp, then the number of daily uses would be a more appropriate long-term retention indicator. So you need to start from your own business and find the most appropriate retention measurement indicator.

Next, you need to understand what percentage of newly registered users remain on your app weeks or months later, and how that percentage changes over time.

A simple approach is to draw a new user retention path diagram like the one below. There are many tools like Mixpanel that can help you draw this kind of picture. Of course, this picture is not difficult, you can also draw it yourself.

An example of a new user retention cohort

The value of this chart is that it can help you understand how your new users are retained, whether retention is healthy, and where the key points to focus on if you want to improve retention.

For example, some key insights you can extract from this graph are:

1. Identify whether your product has long-term retention

If your retention curve eventually becomes flat like the one above, it will be a strong proof that your product has long-term retention. But if this curve continues to approach zero, or even remains flat close to zero for a long time, then you need to improve your core product as soon as possible. Without a healthy long-term retention rate, you can't build a sustainable business because one day, all your users will churn.

2. Determine the changing trend of retention.

In the above picture, you can clearly see that the blue curve representing the new user group has a lower retention rate than the previous red curve, which means that the retention rate of new users is getting worse over time. If you encounter a similar situation, you should start to be vigilant. In this case, your user retention may also gradually approach zero.

Many times this may be due to you introducing a new group of users, if so, you need to improve your new user experience accordingly.

3. Find your window of opportunity

The above chart shows that after the first week, half of the new users churned away. This sends a strong signal: you should spend more energy on improving the user's first landing and usage experience . Of course, retention is different for every product, but in general, it becomes increasingly difficult to retain a user over time, so it is very important to deliver core product value to your users as quickly as possible.

As mentioned earlier, the user retention of each product has its own characteristics. Drawing it out can help you better understand it and find where to focus your efforts.

Now you should understand how your product retains users and where your window of opportunity is. Next you might be wondering how to improve your user retention.

There are many common and effective methods worth considering, such as: making the user's first experience as simple as possible, delivering value based on the user's motivation or needs, cultivating user usage habits as early as possible, etc.

Apart from that, what I recommend most is to seek answers from those users who have stayed. If your product is a product that can retain users, then these users who stay must have done something to experience the charm of your product. You can improve your product based on these things that make users realize the value of your product.

1. Communicate more with your users

When you don’t have much data in the early stages, the easiest thing to do is to chat with your users and try to understand what it is about the product that doesn’t appeal to them.

I recommend that you start by talking to the users who have stayed because they are often the ones most willing to share their experiences and ideas. User research is essential for understanding what your users do to see the value in your product, and it can also help you understand the difficulties they encounter in using it. Communicate with users regularly and combine the results of communication with user behavior data, which can provide you with valuable ideas for improving your product. This approach works at any stage of a product.

2. Analyze the path data of those retained users

As you have more and more user behavior data, you can work backwards to find out what those users who stayed did in your product. With this data, you can optimize your product and guide more users to experience the value of your product in the same way, thereby improving retention (forming a positive cycle):

First, you need to identify a set of core behaviors that are closely related to long-term retention , especially those behaviors that allow users to understand your product and get value from your product. For Pinterest, these actions may be browsing Pin pages (Translator's note: If you have not used Pinterest, you can understand Pin this way: a Pin is a picture/video uploaded by a user, and browsing a Pin is equivalent to watching a video in Tik Tok ), clicking on the content of other people's Pins, saving a Pin, creating a board (a private collection of Pins), etc. I suggest you start with a few key actions, which will be simpler and clearer.

Second, for a specific group of users (such as new users who registered in a certain week), calculate the proportion of users who performed each action . Think of this action as a new user’s journey from signup to retention, e.g. from not saving any Pins, to saving one Pin, to saving more than one Pin, and so on.

Then, for each such step, calculate the percentage of retained users . Taking the action of “saving a Pin” as an example, the data may be plotted similar to the following figure, where the size of the circle represents the proportion of users entering each step.

Finally, use this data to guide development priorities and conduct experiments to verify the cause and effect of these actions .

In the above picture, you can see that as users take more actions, their retention rate will also increase, so working hard to get users to start saving at least one Pin is a good opportunity to improve retention.

Some people call the moment when users gain core value from a product the "Aha moment" , which can be reflected in the fact that the more core behaviors users have, the higher the retention rate . Based on this point of view, you can design experiments to verify whether the practice of "guiding users to save the first Pin" (Translator's note: "TikTok guides users to collect the first video") can indeed improve long-term retention rate.

When you try to analyze with different user groups or add more behavioral variables, you will find that the process will become more interesting. This article only describes a very simple model. You can go deeper, such as analyzing key behaviors to explain more complex problems, or automating the process of selecting key user behaviors.

However, even a simple model like this is enough to help you find many opportunities to optimize your product and improve your retention metrics.

Now you understand user retention and what you can do to improve it. After that, it’s time to put it into practice and iterate quickly by running as many experiments as possible to improve your user retention.

When running these experiments aimed at improving retention, you’ll find that it takes weeks or even months to see statistically significant changes in long-term retention, which makes rapid iteration impractical. To solve this problem, you need to look for some leading indicators that can predict long-term retention changes and use these indicators to capture signals of retention growth. This helps you make decisions faster, allowing you to iterate on changes more quickly.

1. Identify a set of leading indicators, especially those that reflect the behavior of users early in the user journey that can reflect the value that users can get from the product.

For example, if you are working to improve the user experience in the first stage, you should choose the user's [action] action in the next stage as the measurement indicator rather than a certain [action] action of the user in this stage - this can still be considered an early enough behavior, and it can better reflect whether a product improvement is effective for user retention value.

(Translator's note: I think there is an unspoken logic here: the behavior in the next stage is affected by the behavior in the current stage. For example, a business process: view product -> add to favorites. If you want to improve the user experience of viewing products, you may need to select many indicators of the viewing stage, such as PV, such as browsing time. In this case, it is better to directly use the behavior of the next process, i.e., adding to favorites, to indicate whether your experience in the previous stage is good enough. The assumption here is that only when the viewing link is done well, the data of adding to favorites will be improved.)

2. For those leading indicators you choose, you can select a group of users to calculate it:

• Coverage: The percentage of users (out of total users) who performed an action

• Precision: The percentage of users who performed an action that remained

• Recall rate: The percentage of retained users who actually perform an action

The data you get will look like this:

Comparison of coverage, precision and recall of three leading indicators

3. Choose leading indicators that have high coverage and a healthy balance between precision and recall. For example, in the example above, indicator 2 is the best because:

First, it has a higher coverage rate, so you can get more sample size in the experiment, which is crucial for shortening the experiment or decision cycle; indicator 1 is the most accurate leading indicator, but if you want to recruit the same number of users in an experiment, it will take you 7 times longer.

Secondly, indicator 2 has a relatively high average value of precision and recall (i.e., F1 score) (see Note 2 at the end of the article)

Third, although the F1 score of indicator 3 is also high, the accuracy of indicator 2 is higher, which is more important for a leading indicator. Definition of recall metric: Precision is the percentage of users who retain the action. Because your logic is to improve retention rate by guiding more users to perform these core behaviors, you should pay more attention to the accuracy of core behaviors under the same circumstances.

Leading indicators are not perfect, and sometimes you need to run experiments longer to prevent misjudgments, but they are important tools for increasing your learning and iteration speed, even at the cost of accuracy.

Healthy user retention is critical to the success of your business, but to truly improve it, you need to understand retention, use the right tools, and iterate quickly.

1. Understand how your users retain: You can do this by defining retention accurately and charting how your user retention changes over time.
2. Gain a deeper understanding of your users through user interviews and user behavior data to discover ways to improve user retention.
3. Experiment, learn, and iterate quickly with the goal of improving leading indicators that predict long-term retention.

In my experience, these three tools are incredibly valuable for dramatically improving user retention, and they can also help you get a better handle on this critical problem, thereby laying a solid foundation for building a long-term sustainable business.

Note 1: Brian Balfour is the founder and CEO of Reforge, a company that provides growth plans for professionals in marketing , product, data and other fields. He previously served as Vice President of Growth at HubSpot. Original link of this blog: https://www.reforge.com/blog/retention-engagement-growth-silent-killer

Note 2: The F1 score is an indicator used in statistics to measure the accuracy of a binary classification model. It takes into account both the accuracy and recall of the classification model. The F1 score can be regarded as a weighted average of the model's accuracy and recall, with a maximum value of 1 and a minimum value of 0.

The author of this article @增增黑盒 is compiled and published by (青瓜传媒). Please indicate the author information and source when reprinting!

Product promotion services: APP promotion services, advertising platform, Longyou Games