How to build a product indicator system from 0-1?

Suppose Doudou opened a small supermarket near the community, and Huahua would come to the store to buy half a pound of pork head after get off work every Tuesday, rain or shine, without fail. Doudou thought to himself, "Huahua always comes at a fixed time, and has been coming here for several months. If I prepare the meat in advance, we can save both of us time." From then on, every Tuesday, as soon as Huahua came to the store, Doudou would tell him, "The pork head is seasoned, and I've added extra peanuts for you. Just take it away." Huahua smiled and thought to herself: "Hey, this waiter is quite considerate, that's nice."

In the above case, Doudou observed Huahua's behavior of "buying pig heads after get off work every Tuesday" and prepared "pig head meat with more chopped peanuts" in advance. Doudou's approach not only refreshed customers' favorability, but also improved user loyalty.

In the Internet industry, Doudou's observation of Huahua's behavior is called data analysis. To do a good job of data analysis and apply it to business, we first need to build a good indicator system. Next, I will talk about how to build an indicator system.

01 Data indicator system?

1.1 What is an indicator system?

Usually, the indicators we talk about are statistical data that are of reference value to the current business. In other words, not all data are called indicators. The core significance of indicators is that they make business goals describable, measurable and decomposable.

Indicators can be divided into atomic indicators and derived indicators. According to my understanding, atomic indicators are indicators without any modifiers, also called metrics, such as order volume, user volume, etc.; derived indicators are addition, subtraction, multiplication, division or limitation of modifiers on atomic indicators.

For example: yesterday’s imported cases from abroad, the number of visits to the website in the past week, etc.

The indicator system sorts out the business from different dimensions and organizes scattered, single-point, interrelated indicators in a systematic manner. Among them, dimensions are divided into qualitative dimensions and quantitative dimensions. Qualitative dimensions are mainly textual descriptions, such as name, place name, etc.; quantitative dimensions are mainly numerical descriptions, such as salary, age, etc.

Take a limited-time flash sale on an e-commerce site as an example (as shown below):

The upper red box represents market activity, the middle red box represents the current price fluctuation range, and the lower red box represents the price trend. The indicators in the three red boxes can constitute the simplest indicator system, which is used to describe the current situation of Yili pure milk flash sales, and belongs to the descriptive indicator system.

1.2 Why do we need an indicator system?

For data product managers, building an indicator system can better organize the business and improve the efficiency of problem analysis.

Therefore, the author believes that the main purpose of the indicator system is:

1. Provide guidance for business development;
2. Establish a common vision, unite the team, and motivate the team.

02 How to design an indicator system?

The following is divided into five parts (as shown in the figure below), which discusses how to design an indicator system.

2.1 Setting goals

This is the first and most important step. It is also the standard for evaluating many products after they are put into operation, thus forming a closed loop. Good goals have three characteristics:

1. Aligned with high-level goals;
2. Goals should be SMART;
3. Challenging.

Next, I will talk about the SMART principle:

1) S stands for Specific

The goals must be clear and specific, and must be aligned with specific work indicators; they cannot be general. Here is an example:

• Ineffective goals: I want to be a content manager.
• Specific goal: I want to master copywriting skills.

2) M stands for Measurable

Goals must be measurable, measurable indicators must be quantitative or behavioral, and data or information to verify these indicators must be available.

The above example is further refined to make the goal measurable.

Measurable goals: I want to master the skills of setting article topics, addressing pain points, finding creative ideas, and deciding titles.

3) A stands for Attainable

Goals must be achievable, meaning they can be achieved with effort. Avoid setting goals that are too high or too low.

If I had just started content marketing and set a goal of “I want to become an expert in copywriting within two months”, then this would be an unrealistic goal. A more down-to-earth goal is: I want to master the skills of setting article topics, addressing pain points, finding creative ideas, and deciding titles within three months.

4) R stands for Relevant

Goals are related to other goals in the work.

For example, my medium-term goal is to independently complete the creation and publication of a series of articles within one year, and my short-term goal is to master the basic article writing skills within three months. Only when medium-term goals are strongly related to short-term goals can they be easier to achieve.

5) T stands for Time-bound

The timeliness of a goal means that the goal is time-limited. For example, when I was operating the public account "A Data Person's Reserved Land", the goal I set for myself was "to create and publish 12 articles by December 31, 2021." The December 31, 2021 here is the fixed time limit.

2.2 Model Building

2.2.1 PLC Model

Product Life Cycle (PLC) refers to the market life of a product, that is, the entire process from the time a product enters the market to the time it is eliminated by the market. The product life cycle includes exploration phase, growth phase, maturity phase and decline phase (as shown in the figure below).

At different stages of the product life cycle, each business party has different focuses and pays attention to different data indicators.

1) Exploration period

The focus of the exploration period is to verify the core value of the product and whether it can meet market demand and make profits from it. What to do: hypothesise, verify, iterate, and execute. This stage will focus on target user portraits, key behaviors, and retention rates. Let’s take the early Tubatu as an example (as shown below).

Based on the above diagram of Tubatu's business process and its positioning (exploration phase), the current focus is on polishing service capabilities and understanding the matching degree between user group needs and product services. The key indicators are as follows:

1. Target user portrait: gender, age, education, region, and occupation.
2. Key behaviors: number of pictures and texts published, number of viewers, amount of dissemination, and amount of usage.
3. Quality: article conversion rate and completion rate.

2) Growth stage

After the exploration period of polishing the product, the product has a good retention rate, and now the product begins to enter the user growth period. During the growth stage, you need to focus on the first half of the user's entire life cycle, that is, improving retention, user activation, and self-propagation.

3) Maturity

As the market becomes saturated, the user growth rate slows down and gradually stabilizes. The core indicator of attention should be user activity, while paying attention to the commercial conversion path. In fact, if the market itself is an incremental market, you can consider making a different growth curve for a mature product through customer acquisition.

4) Decline

When new products or substitutes emerge, users turn to other products, causing the number of users of the original product to decline rapidly, thereby causing the original sales and profits to decline rapidly, and the product enters a period of decline. In the decline stage, you need to focus on user churn and retention.

2.2.2 OSM model

The OSM model (Objective, Strategy, Measurement) is an important method to assist in determining the core in the process of indicator system construction. It includes business objectives, business strategies, and business metrics, and is a horizontal thinking of indicator content.

1) Business objectives

The goals are determined mainly from the user and business perspectives, and the principles are that they should be feasible, understandable, interventionable, and positive.

1. What is the user's goal in using the product?
2. What needs of users does the product meet?
3. What is the purpose of the company/business/product etc.?

2) Business Strategy

Strategies adopted to achieve the above objectives. In other words, when do users feel that their needs have been met?

3) Business metrics

What changes in data indicators are brought about by these strategies? Whether the user's demands are effectively met and business goals are achieved.

Taking PMCAFF as an example, what are its indicators according to the OSM model?

1) Business objectives

What is the user's goal in using the PMCAFF product?

Two types of users need to be involved: content producers and content consumers. Next, we will briefly introduce the analysis ideas of content producers.

User needs: Publish articles or share opinions, establish industry influence or receive feedback on content.

So, how can we make users feel that their needs are met?

2) Business Strategy

The strategies used by PMCAFF are: applause, commenting, sharing, recognition, columnists, and good questions.

3) Business metrics

Next, we need to create indicators for these strategies. Here our indicators are result indicators and process indicators.

Remark:

1. Outcome indicators: indicators used to reflect certain business outputs or results, which are usually known with a delay and difficult to intervene. Result indicators are usually more about monitoring whether the data is abnormal, or monitoring whether the user's needs are met in a certain scenario.
2. Process indicators: indicators generated when users perform a certain operation. Certain strategies can be used to influence process indicators and thus affect result indicators. Process indicators are usually more focused on why user needs are or are not met.

Let's take the PMCAFF above as an example:

1. Result indicators: number of articles published, number of people who published articles, number of applause/comments on articles, amount of rewards received, number of columnists, number of new columnists, etc.
2. Process indicators: number of people using content import, content publishing conversion rate, article interaction rate, etc.

After selecting the indicators, the next step is to choose the analysis dimensions. The dimension selection level is mainly determined by combining the data analysis perspective with the actual business scenario. For example: user tag dimension, time dimension, channel dimension, etc.

2.2.3 Index classification

Indicator classification mainly involves breaking down indicators into different levels and analyzing them level by level. According to the enterprise strategy, enterprise organization and business, the indicators are graded from top to bottom and analyzed layer by layer. The OSM model can be combined to determine the indicators.

1) First-level indicator: corporate strategy level

It is used to measure the achievement of the company's overall goals, is closely integrated with the company's current business, and has core guiding significance for all employees. Level 1 indicators usually guide the company's strategic level.

The primary indicators are usually determined based on the market, product life cycle, product category and business model. There is only one most critical indicator (OMTM, One metric that matters) at a point in time.

For example: How did Xiaohongshu’s OMTM (North Star Metric) evolve?

2) Secondary indicators: business strategy layer

In order to achieve strategic goals, the company will further break it down into core indicators of business lines or business groups. Usually, in order to achieve the primary indicators, companies will make corresponding strategies, and the secondary indicators will also be related to these strategies.

For example, Xiaohongshu’s current primary indicator is sales, so the secondary indicator can be set as sales of goods in different categories, sales by region, etc. In this way, when there is a problem with the first-level indicator, we can quickly locate the problem.

3) Third-level indicators: business execution level

The third-level indicators are to expand the second-level indicators vertically and perform path decomposition, funnel decomposition, and formula decomposition. Tertiary indicators are usually used to locate problems with secondary indicators and usually guide the work of front-line operations or analysts. Level 3 indicators are the most common indicators in the business.

Path decomposition requires analysis of business processes, such as: opening the app, browsing the homepage, browsing the product details page, adding to the shopping cart, submitting the order, order payment, and successful payment.

Use the formula to break down monthly active users, as shown below.

2.2.4 AARRR

The AARRR model is the pirate model and is also a classic model for user analysis. It reflects that growth runs through all stages of the user life cycle, namely acquisition, activation, retention, revenue, and referral.

1) Get

Operations personnel promote through various channels, acquire target users by various means, evaluate the effectiveness of various marketing channels, and constantly adjust operations strategies to continuously reduce customer acquisition costs.

Key indicators: exposure, clicks, downloads, installations, activations, activation rate, installation rate, registration conversion rate, retention rate, payment rate, etc.

2) Active

Active users refer to target users who have started using the product. Product managers guide users to use the core functions of the product through new user rewards, product guidance, and other means. We need to grasp user behavior data and monitor the health of the product.

Key indicators: ratio of new and old users, DAU/WAU/MAU, average daily login times, average daily usage time, etc.

3) Retention

Usually the cost of maintaining an old user is much lower than the cost of acquiring a new user, so we need to not only attract new users, but also pay attention to user stickiness, and where and why users churn.

Key indicators: new user retention rate, old user retention rate, active user retention rate, daily, weekly, monthly retention rate, churn rate, etc.

4) Monetization

It is mainly used to measure the commercial value of products, which is also the essence of business.

Key indicators: ARPU, ARPPU, payment rate (distinguishing between new and old users), average order value, LTV, etc.

5) Self-propagation

It is mainly based on the attractiveness of products, marketing, celebrities and other events, which enables users to spread the word spontaneously.

Key indicators: fission coefficient, etc.

03 Burying points

Data collection is the collection of relevant data, which is the starting point of the data flow. Whether the collection is correct or not, and whether it is complete or not, directly determines the quality of the data and affects all subsequent links. So what kind of data can be considered high quality? This requires advance planning of the locations.

3.1 What is a buried point

Tracking is a term in the field of (user behavior) data collection. Its scientific name is Event Tracking. It mainly refers to the relevant technologies and implementation processes for capturing, processing and recording specific user behaviors or events, such as the number of times a user clicks a button, the length of time a user reads an article, etc.

Tracking is a commonly used data collection method that records user behavior processes and results to meet the needs of rich data applications. Data tracking is an important method of data collection and is the origin of data. The collected data is often used to analyze product usage, user behavior preferences, etc., thus extending to data products such as user portraits and user recommendation systems.

3.2 Tracking process

The business department proposes requirements based on the business, and the product manager organizes the requirements into data requirements and outputs a data requirements document (DRD, Data Requirements Document). Next, the product manager will conduct a requirements review with the data team. Regardless of whether the review is passed or not, a requirements review minutes email will be sent to relevant personnel after the meeting.

After the review is passed, the product manager needs to confirm the development time with the development engineer and send a scheduling email. After development is completed, test engineers, data analysts, and product managers need to verify whether it is complete and accurate and submit an acceptance report. After the function is launched, the product manager or development engineer needs to send an online email. As shown below.

3.2 How to bury points

How to bury it? From a business perspective, there are five perspectives:

• Who: Identifies the initiator of the behavior, usually using an account number or device number. Account number is a common method to distinguish users through information such as ID number, mobile phone number, account ID, etc.; device number is mostly used for products that do not require login, and users are distinguished by device code.
• When: records when the behavior occurred. Generally, the server time, that is, the Unix timestamp, is used to record the time when the behavior occurred. It is a global unified time and is not affected by any region.
• Where: records the location where the behavior occurs, usually located by GPS, or by device IP to determine the user's location.
• What: refers to the specific content of the user's behavior. For example, if a user reads a book, what is the title of the book he purchased? What is the price? Which publisher published it, etc.
• How: How the behavior occurs is usually included in the name of the behavior, such as submitting an order. There are also some behaviors that can be completed in multiple ways, such as unlocking an iPhone, which can be unlocked by entering a password or by scanning the face. No matter which method is used, it is a kind of information that can be recorded.

3.3 Case: Browsing the APP Home Page Behavior Tracking

The behavior of browsing the homepage of an e-commerce APP is analyzed from five perspectives and divided into two categories: unique indicators and public indicators, as shown in the figure below.

3.4 Case: Payment Order Behavior Tracking

Payment order behavior is analyzed from five perspectives and divided into two categories: unique indicators and common indicators, as shown in the figure below.

04 Data Analysis

4.1 What is Data Analysis?

Data analysis refers to the process of using appropriate statistical and analytical methods to analyze large amounts of collected data, extract useful information, form conclusions, and conduct detailed research and summarize the data. Simply put, it is to analyze the data.

The purpose of data analysis is to dig out the information hidden behind some seemingly chaotic data and extract the internal laws of the target object. For enterprises, the essence of data analysis is to create business value and drive business growth.

4.2 Data Analysis Methods

We take an e-commerce website as an example, use data products to collect data from the website, and then use common data analysis methods to analyze it, such as funnel analysis, retention analysis, time analysis, user portrait, channel analysis, distribution analysis and other analysis methods.

4.2.1 Funnel Analysis

Funnel analysis can scientifically reflect the user behavior status and the user conversion rate of each business process from the starting point to the end point. It is an important process data analysis model. The funnel analysis model has been widely used in user behavior analysis, such as channel quality assessment, product sales, and other daily data operations and data analysis work.

For example: For e-commerce products, the ultimate goal is to get users to buy goods, but the conversion rate of the entire process is determined by the conversion rate of each step. At this time, we can monitor it through the funnel analysis model. As shown in the figure below, we can observe the conversion rate of users at each level, find the weak points in the conversion path, optimize products, improve user experience, and ultimately improve the overall conversion rate.

4.2.2 Retention Analysis

Retention analysis is an analytical model used to analyze user engagement/activity, that is, the process of converting initial new users into active and loyal users. As the statistics change, relevant personnel can see the changes in users at different stages and thus judge the product's stickiness to users.

For example: For a certain e-commerce platform, the 7-day retention rate of users in the last 30 days (as shown in the figure below) shows that the user retention rate is low. Next, divide users into different groups according to region, age, behavior, etc., observe the differences in retention, and find areas where the product can be optimized.

4.2.3 Event Analysis

Event analysis is used to study the impact of an event on a company and the extent of the impact. Generally speaking, event analysis includes event definition and selection, drill-down analysis, and results.

An event is when a user completes a specific thing at a certain point in time, at a certain place, and in a certain way. Its key factors are Who, When, Where, What, and How.

For example: Operations staff found that the number of successful payments fluctuated greatly in the past 30 days (as shown below). Enterprises can first define events, filter the delivery method to "self-operated", and then drill down from multiple other dimensions, such as "product ID", "order amount", "whether coupons are used", etc. When segmentation filtering is performed, abnormal data has nowhere to hide.

The above are three commonly used data analysis methods. In different industries, they are often presented in different styles. When we face different problems, we need to know clearly which method or methods are most effective and apply them flexibly in combination with specific scenarios. There is no best analysis method, only the most suitable one.

Author: Cat ears

Source: Cat's Ear