A senior product expert with 15 years of experience teaches you how to measure product iteration?
|
The topic I’m going to share with you today is data measurement of product iteration. Today's main content is divided into four parts. The first part is to share my personal views on products and product iterations. The second part will share with you why we should insist on measuring each product iteration. The third part is today’s main topic “How to measure product iteration”. It will introduce a relatively systematic method we have explored to measure iteration through data, and finally give a summary of the entire sharing content. The first question is, what is a product? What is product iteration? Regarding the question of "what is a product", I believe everyone may have their own understanding and cognition, just like there are 1,000 different versions of Hamlet in the eyes of 1,000 people. Each of us has our own understanding of the question "What is a product?" I personally believe that a product is a standardized solution to a problem. The so-called standardized solution to the problem: First of all, the product is a "solution" that solves the problem. As a product manager, we are actually always thinking about what problems we want to solve for users, or where exactly the users are feeling pain or itchiness? Then the solutions provided by our products should be able to solve these pain points or itch points well, which is the value of our products. Secondly, the product is also a "standardized" solution. For example, several years ago, Bell invented the telephone, which first solved the problem of long-distance communication between people. The solution uses telephones and telephone lines to transmit human voice from one place to another so that another person can hear it, allowing people to communicate over long distances via the phone. The "standardization" here includes two aspects: one is the abstraction of demand, that is, abstracting the demand for long-distance communication into the medium of sound, while ignoring the other four irrelevant senses; the second aspect is the standardization of solutions, solving the cross-distance transmission of sound with the following technical solution: use a telephone line to connect two telephones in two places. One of the telephones collects sound, converts it into electrical signals, transmits it through the telephone line to another telephone, and then uses the handset to restore the electrical signals back into sound. Through such a standardized solution, all long-distance communication needs are met. The above examples are standardized problem solutions. This is my concept about products - any product is a solution to one or more problems, and the solution to this problem is an abstract and standardized solution. In addition, the product is also a business model. As a business model, it is very important to be replicable and scalable. When it comes to business, there is an old Chinese saying that goes "business is business". When we look at products from a business perspective, we can't just make something and call it a product. It plays a key role in the development of the company and needs to obtain cash income from customers through the business model to provide financial support for the company's survival and development. Two very important points in a business model are replicability and scalability. At this point, some people may wonder, " Where has the user experience gone ?" There are many guests attending the lecture who are designers or product managers who are designers themselves, and they may feel uncomfortable about this. But from my personal point of view, user experience is very important, but compared with the previous two aspects, user experience is more like icing on the cake. User experience is one of the replicable and scalable components of a business model. If we only look at user experience, it is actually not that important. There is an old Chinese saying, "Without the skin, where will the hair be attached?" If we only talk about user experience without solving problems and without business, it will not stand. Of course, user experience is very important, and it is something that every product manager and designer, as well as everyone on the product line, needs to work hard to do, but it must be viewed in the context of the entire business model. Next, I will share about product development and product iteration. I believe that product development is a process. The purpose of this process is to find and discover three things. The first is what problem to solve; the second is what solution to use to solve the problem; the third is to seek commercial success, which must be replicable, scalable, and able to bring the expected benefits to the company. Up to now, most Internet products adopt an iterative approach to product development. I personally think that the reason why iterative development is widely adopted is that product development is a high-risk thing, and the iterative development method can help us control the risk of failure within a reasonable range. We can invest relatively little cost and obtain some feedback from users as quickly as possible to verify whether our products meet market demand. If it fits, keep going and explore new ways. The second is that the iterative approach can help us accelerate the entire product development process. Acceleration means that we find the right problems, the right solutions and shorten the business exploration cycle. The above are my personal views on products and product iteration. Let me share with you “Why do we need to do product measurement”. Internet companies, especially startups , face survival pressure every day. For large companies, they need to consider how the company can survive in the long run and achieve better results in this market. Taking a small company as an example, what is the life and death line? Reach product-market fit before investors ’ money runs out (product-market fit describes a state in which the number of product users may grow rapidly. If product-market fit is not reached, a lot of exploration work needs to be done before that). The process of product development, that is, our product iteration is actually a process of experimentation. Think back to the experiments we did in middle school and college. A very important step is to verify the experimental results. For product iteration, we also need to pay attention to whether the results of each round of experiments (iterations) meet our expectations. However, due to the great pressure of progress, the product team and the R&D team may often unintentionally avoid the verification process, which is very inappropriate. Let me share two pieces of data, one from Microsoft and one from Amazon. Among the ideas that were generally recognized as OK within the Microsoft team, after they were actually implemented: about one-third achieved the expected results, one-third had no significant impact, and one-third even had negative effects. Between 60% and 90% of Amazon’s ideas fail to improve their products. Therefore, when it comes to product verification, we should not be too blindly confident, but should return to reality from the distorted force field. As we all know, one of our product managers has a very important ability, which is to distort reality into what we imagine. Of course this is not a bad thing. If a product manager does not have this ability, he will not be able to make a good product. We must first distort the future real world in our minds into what we believe in, and build a complete set of solutions based on this, so that we can produce product design drafts and finally hand them over to the product R&D team. If you don’t have the ability and imagination, you actually can’t make a product. But if we just blindly immerse ourselves in our own world and don't step out of it, it will harm us. Therefore, product development will be divided into stages. During the stage of thinking about solutions, researching problems, and doing things, we must make full use of our ability to distort reality. However, after the version is actually released and launched, we must return to reality and take a look at what the data is like. So we insist on measuring each iteration to validate our ideas. How to measure product iteration? We have conducted research on how to measure it, and found that many excellent teams are committed to doing this. In daily work, everyone has their own experience and has summarized some methods. We conducted research together with many teams, and after communication, we summarized a relatively comprehensive method. The core of this method is to ask ourselves five questions and then answer them through data. This method allows us to more comprehensively measure the actual effect of each revision. These five questions are ●Are the new features (or improved features) popular enough for users? ● Will users repeatedly use our new (or improved) features? ● What is the overall impact of this iteration on user retention? ● If the revision is to optimize a certain usage process, has the conversion rate of this process been improved? ● Gain a deeper understanding of how users are actually using our products, and improve (or add) features? Before we get into the measurement methods, let me briefly share some basic data concepts. If our product manager has no knowledge of data measurement before, then he needs to understand some data indicators first. If the earth is regarded as an APP, the users are like the people on the earth. 140 million people are born every year, 7 billion people are currently living in the world, and a total of 108 billion people have died. The number of births is like the number of new users, the existing 7 billion population is equivalent to active people, the cumulative number of deaths means loss, and the concept of "it is rare for a person to live to be 70 years old" is similar to the retention cycle, and people's reproduction is very similar to the word-of-mouth spread of a product. In this way, we can match these concepts in the user life cycle one by one: Acquisition, Activation, Retention, Referral, etc. For Internet products, there is also a link of commercial realization (Revenue). These five links constitute what we often call the A AR RR user life cycle model. 1. Is the new feature (or improved feature) popular enough for users? When it comes to iterative measurement systems, the first thing to consider is whether the new feature is popular. After the product is released, how many active users does this version have? Are some new features after the release popular? If we use an indicator to summarize it, this indicator is called the function activity ratio (the ratio of active users using a certain function to the number of active users in the same period). This indicator can reflect whether a new function is popular. We cannot look at it from this single aspect. It is insufficient to measure it from this single aspect alone. 2. Will users repeatedly use our new (or improved) features? The second aspect is to see whether new users will use the product repeatedly. Assuming that the product manager is good at networking, he can use various means to guide users to new features as much as possible. If the user is not happy with the product, he will use it on the first day, and then use it again on the second day, but find out on the third day that the product has bugs and the experience is not good, so he will stop using it. Not using it means that the user synchronization usage rate will decrease. However, when it comes to actually measuring it with data, the synchronization usage rate is quite difficult to measure. Because users will churn, the repeat usage rate must be viewed together with the retention and activity ratio to obtain a relatively objective result. 3. What is the overall impact of this iteration on user retention? The third aspect is the impact of the new version on retention. The current mainstream product data analysis or user behavior data analysis platforms actually have corresponding models and functions, which you can directly look at. From the perspective of retention, as shown in the figure, the red line is the dividing line between the two versions. The version below the red line is the old version, and the version above the red line is the new version. After launching the new version, we found that the user retention rate has increased significantly. This indicates that the revision has improved user retention. If there is no change, or the retention rate decreases, it means that we have not achieved the results we expected. 4. If the revision is to optimize a certain usage process, has the conversion rate of this process been improved? Next we need to see whether the conversion rate has increased. For example, a content product will publish UGC or PGC content, which can be articles, pictures or videos. Our expectation is that users will share the content on other social platforms if they like it. By examining the three steps from reading to sharing and then completing it again, we can compare and analyze whether the conversion rate has increased after the revision. If there is no improvement, we need to find the reasons behind it. 5. Gain a deeper understanding of how users are actually using our products and what improved (or new) features can be added? Finally, if your boss asks you for the results of a product revision, it is enough to show him data such as user retention rate and conversion rate. However, as product managers, we need to further dig into some more data so that we know which aspects of the subsequent iteration need to be deepened. What is the primary concern? How is the usage of the released or improved functions? It can be split from two aspects: activity and retention rate. Going further, what else do we need to pay attention to? For example, what do users do when they open the app? If the version change is relatively large, will there be any changes in what users do after entering the application? We need to compare a time period before the revision with a time period after the revision, the main purpose of the comparison is to find the differences. If there is not much difference before and after the revision, it is one case. If there are some changes before and after the revision, we need to analyze whether this change is what we expected. In addition to what users do after opening the app, we also need to analyze what users do before leaving the app, that is, at which stage the users end their use of the app. In this way, some problems can often be discovered. For example, for a web product, a user goes to the homepage through a search engine and then ends there. At this time, we need to analyze from the product perspective to see if further optimization is needed in the guidance of the homepage copy , button placement, color, layout, etc. Finally, we should focus on iteratively improved functions or newly added functions. The approach is to see what users did before using the improved function and where they came from. Also, what did you do after using this function? That is to say, put this function in the user's usage scenario or in the user's usage path to understand the user's background. At this point, we need to look at the overall situation of the overall revision from the perspective of major statistical indicators. As a product manager, it is still not enough to achieve this step, including the overall performance of users when using the functions. We also need to further examine the user's entire usage path. Speaking of this, actually when I talked about product data analysis before, I was a little embarrassed to talk about this. Because I think this seems a bit LOW, to check the user's usage path one by one, to see what he does in the first step and what he does in the second step. I think this seems a bit LOW, so I am a bit embarrassed to talk about it. It was not until later that I met some product managers from Baidu, who mentioned that an important part of their daily work is to check the user usage paths and even make plans. For example, if you target 100 users, you may spend a week or two to review their usage and develop corresponding plans. In this way, we can make up for our lack of understanding of users to a large extent. summary Here, we briefly review the entire method. I just mentioned that if you want to use data to measure product iteration, you need to see: whether the new function is popular; whether users are willing to use it repeatedly, which respectively corresponds to the product's function activity ratio and repeated usage rate; the third is to find out whether the new version has an impact on retention and whether it improves the product conversion rate as expected; the fourth is to have an in-depth understanding of how users actually use our products, including behavioral path statistics and detailed usage records of individual users. In my personal experience, if you can measure each product iteration according to this method, first of all, you will probably find that many revisions do not produce sufficiently positive effects as expected. Second, if you do some analysis for each iteration according to these methods, you may summarize some very important insights and experiences. Summarize Looking back, I talked about three aspects today. The first one was to share my personal understanding and opinions on products and product iterations. The second is to draw out my personal opinion on why we need to iterate our products based on my personal understanding of the product. Whether it is for the company or the product manager's personal career, the future is expected to be a relatively good development. The third aspect is to share with you a systematic method for measuring product revisions. Basically, we rely on answering these questions to examine the effectiveness of our revision measurement. Mobile application product promotion service: APP promotion service Qinggua Media information flow The author of this article @于晓松is compiled and published by (APP Top Promotion). Please indicate the author information and source when reprinting! |
>>: How to develop a Farmhouse mini program?
Recommend
Analysis of Himalaya's competitive products
Preface : From Himalaya , I have briefly analyzed...
Operations uncle: The circle of friends exploded! Let’s talk about why the constellation QR code of the Plato app can be all over the circle of friends overnight!
I'm sure all of you in your circle of friends...
The data and effects of major mobile game launches are all here!
Mobile game traffic has three main sources: chann...
Online promotion channels for product operation and promotion plans!
In the mobile Internet industry, the market for g...
DOU+ advertising and buying tips!
The biggest charm of Dou+ is to leverage natural ...
Summary of myofascial chain rehabilitation techniques (must be mastered by rehabilitation therapists)
Summary of myofascial chain rehabilitation techni...
20 suggestions for private domain traffic operations!
In the past, when people were doing brand marketi...
How to create a Kuaishou account from scratch? Here’s a how-to guide!
How to create a Kuaishou account from scratch? Af...
How to bid and match Google/Baidu SEM keywords?
Bidding and matching methods are the two most imp...
Experienced APP operators, is this how they attract users?
If you think of an APP as a car, then the operato...
Overseas promotion and marketing, Facebook advertising process!
With its huge user base and increasingly sophisti...
Sogou advertising promotion, Sogou advertising account opening fee
Sogou Search has many search platforms, such as S...
Changsha tea drinking and tasting high quality reliable recommendation high-end takeaway with its own studio and the whole venue
Changsha's high-quality tea drinking and reli...
Unity3D MOBA real-time competitive game development with engineering materials
Unity3D MOBA real-time competitive game developme...
What are the functions of Foshan WeChat blind date mini program? How much does it cost to make a matchmaking app?
Nowadays, the ratio of men to women is unbalanced...