Data Operations: How to build a data indicator system?

Many people use data indicators to measure project effectiveness, but the resulting data is often fragmented and key information cannot be seen. So how can we systematize the indicators and look at the overall problem through single point data? Let’s see what the author says?

1. What is the indicator system?

In our product and operations work, we come into contact with different data and different indicators. Many times, the data we collect are at the level of a single point, and the data that is finally displayed is often scattered and cannot be connected in series to discover global problems.

The systematization of indicators connects scattered data together, allowing you to see the overall picture through a single point and solve single point problems through the overall picture.

To describe it in one word, it is "one move affects the whole body". By seeing the changes in relevant indicators, we can see the changes in the overall business scenario, so as to quickly discover problems or monitor the effectiveness of corresponding operational strategies.

2. What are the benefits of the indicator system?

I believe that after reading the previous explanation, you will find that the biggest difference between systematic indicators and scattered indicators is whether they can discover some problems more quickly.

If you’re still a little confused, let’s look at the following case with conversion rate and retention rate:

Just do it, but 7 days later.

We often encounter this kind of problem when analyzing single-point data - we analyze whatever problem occurs without considering the user's entire scenario.

So let’s change our perspective and see what the difference is?

From the above two examples, we can see that non-systematic indicators are often single-point analyses. If the analysis fails, another point needs to be re-analyzed, and it is impossible to connect them in series for global analysis.

Systematic indicators are often analyzed in combination with user scenarios, and multiple different indicators and dimensions can be connected in series for comprehensive analysis. Just like a dimension in the sorting factor (such as price), you can analyze the page conversion rate, the product sales rate, etc. The cause of the problem can be found more quickly through analysis of the same dimension.

Having said so much, how to build an indicator system?

Below is the essence of practical content:

3. How to build an indicator system

Before we talk about how to build an indicator system, let’s first briefly talk about what indicators are.

An indicator is actually a kind of measurement. It can be used to monitor and evaluate the status of business processes or to measure the status of a certain functional module or the effectiveness of one's own activities.

From an operational perspective, a good indicator needs to have four characteristics: it must be valuable at the business level; it must be able to measure the true business situation; it must be simple and executable; and it must be recognized by everyone.

From a technical perspective, a good indicator has four characteristics: easy to collect and measure quickly; high accuracy; decomposable in multiple dimensions; and a single data source. Just like we often use to measure the number of people launching APP products, using UUID or COOKIE is often more accurate than using IP.

But many times, due to technical or business reasons, it is often difficult for us to find perfect indicators. So the most important thing for us at this time is to unify our analysis and observe the fluctuations of the data more.

1. What indicators to choose

There are two ways to select indicators: indicator classification and OSM model.

(1) Indicator classification: usually divided into three levels.

First-level indicators: indicators at the company’s strategic level.

Used to measure the achievement of the company's overall goals, usually set at 5-8 indicators. These indicators are closely integrated with the business, formulated in accordance with industry standards, and have industry standard indicators for reference. These indicators are of core guiding significance to all employees in the company.

For example, the first-level indicators of a game company include: new accounts, retention rate, DAU/MAU, number of paying users (rate), revenue amount, etc.

Secondary indicators: business strategy level indicators.

In order to achieve the primary indicators, companies will make some strategies, and the secondary indicators are usually related to these strategies. It can be simply understood as the implementation path of the first-level indicators, which is used to locate the problems of the first-level indicators more quickly.

For example, if the primary indicator of a game company is game revenue, then the secondary indicator can be set as the revenue from different game items. If the primary indicator is DAU, then the secondary indicator is set to the DAU of the sub-server, etc. In this way, when there is a problem with the first-level indicator, we can quickly find the location of the problem.

Level 3 indicators: business execution level indicators.

The third-level indicators are the path breakdown of the second-level indicators and are used to locate problems with the second-level indicators. The use of third-level indicators is usually to guide front-line personnel in their work. The requirement for the third-level indicators is that front-line personnel can quickly take corresponding actions after seeing the indicators.

For example, if the secondary indicator of a game company is the DAU of the XX server, then the tertiary indicator can be set as game duration, game frequency, game level distribution, game level loss, etc. By observing these data, you can make targeted adjustments. For example, if the number of users leaving a certain level is particularly high, then try to lower the difficulty.

Of course, if you want to be more detailed, you can continue to split it down, so I won’t go into details here.

Here is a note. When grading the entire indicator, we need to think about:

Whether the first- and second-level indicators can reflect the current operating status of the product; whether the third- and fourth-level indicators can help front-line personnel locate problems and guide operational work.

The above is the vertical content of indicator classification. Thinking laterally, how do we select appropriate indicators for different levels of indicators?

This is the second method of indicator classification: OSM model.

1. OSM Model

The OSM model (Obejective, Strategy, Measurement) represents business objectives, business strategies, and business measurements respectively.

O: What is the user's goal in using the product? What needs of users does the product meet? S: What strategies did I adopt to achieve the above goals? M: What changes in data indicators are brought about by these strategies?

We build an indicator system to better identify user problems and solve them. So we need to consider the overall content from the user's perspective.

Taking Zhihu as an example, according to the OSM model, what are its indicators?

O: What is the goal for users to use the Zhihu product?

There are two different users involved here - content sharers and content consumers. Here we briefly introduce the analysis ideas of content producers, and you can try to analyze content consumers yourself.

User needs: share knowledge and opinions (publish opinions), establish industry influence (content receives feedback).

So, how can we make users feel that their needs are met?

S: Zhihu’s strategy is: like and comment on content, reward content, increase salt value, and be an excellent answerer on XX topic.

M: Next, we need to create indicators based on these user actions. We will have two indicators here, namely result indicators and process indicators.

Result indicators: used to measure the results of a user's action. They are usually known after a delay and difficult to intervene. Process indicators: indicators generated when users perform a certain action. These process indicators can be influenced through certain operational strategies, thereby affecting the final result.

Let’s take content producers as an example:

Result indicators: number of articles published, number of people who published articles, number of likes/comments on articles, number of people who received rewards, amount of rewards received, number of excellent answerers, number of new excellent answerers, etc. Process indicators: number of people using content import, content publishing conversion rate, article interaction rate, comment folding rate, etc.

Usually we use the OSM model in the process of specifying indicators to target the actions taken by users in different scenarios, the possible results of these actions, and what kind of data changes will occur in the user's actions. We will then combine the data to make targeted adjustments to our operational strategies or product features.

Simply put: Result indicators are more about monitoring data anomalies, or monitoring whether user needs are met in a certain scenario. Process indicators, on the other hand, focus more on why user needs are met (or not met).

1. How to build

After selecting the indicators, we need to start building the indicator system. As we said before, the indicator system conducts comprehensive analysis through scenario processes, and the most important thing here is the choice of analysis dimensions.

A good indicator can be broken down and divided into multiple dimensions. When a good indicator is combined with complete dimensions, many problems can be easily solved. If there is no appropriate dimension, what you have established are still just multiple indicators. The indicators are only reasonable, but you cannot perform before-and-after scenario analysis. To put it simply, dimension is the line that connects points into a scene.

I usually use the following logic to build a system: select indicators - create possible dimensions for each indicator - recombine indicators and dimensions.

Usually when we select an indicator, the dimensions we think of are relatively simple. For example, when a user enters a product details page, I may only want to know which category of product details page the user has entered. When a product is sold, I may be concerned about the category and amount. Similarly, when we monitor user searches on the product list page, we will be concerned about what words the user has searched for, the search frequency, etc.

So if I let the user enter the product details page, what kind of scenario will the combination of search keywords and transaction indicators look like?

The user searched for a keyword and entered the product details page, and then the product was purchased.

At this time, when we analyze the search keywords, the efficiency will be very high. For example, if the user conversion rate for searching a certain keyword is very high, but the number of searches is relatively small, can we increase the overall conversion rate by setting this keyword as a hot search?

After breaking down the dimensions of the indicators, how do we reorganize them?

My principle is: in the same process, if user actions are related, cover the same available dimensions in multiple indicators as much as possible. Taking e-commerce as an example, the user's actions are: search category - search list page - click filter - product list page - submit order - purchase. Then, during the entire process, it is best to retain the search category keywords, screening conditions, product information, and other content.

Finally, once our indicators and dimensions are determined, all that remains is to do data collection. What kind of buried data will not be criticized by developers? The kind of development that requires no brainer.

The format I used before is this, you can refer to it:

Finally, all that’s left is data visualization. Data visualization is usually combined according to one's own business scenarios and uses appropriate data, which usually includes user data, channel data, business process data, etc. I won’t go into too much detail about the specific visualization. Just search on Baidu and I think you can tell what kind of picture is used for what kind of scene.

What is important is the combination of these processes, that is, combining different indicators and dimensions. The events derived from the correlation analysis are the most core. This is a third-level indicator that can fully guide your work and runs through the entire process.

IV. Conclusion

Data analysis is a basic skill of product and operation. Good operations and products will always discover and solve problems through data. A complete data system can make this task more efficient, discover problems at the source, and work can be done easily with the help of systematic tools.

Finally, let me introduce to you the existing platforms that can assist in building these data systems: There are several platforms currently providing data services in China: Growing IO, Sensors Data, Zhuge IO, etc.

If you do not have a complete data set, it is recommended to use a third party which will be faster. Then all that remains is to use the contents of this article, provide the required data indicators and dimensions, and find developers to carry out point placement.

Source: Tree Cat Says