How to quickly build a data operation and analysis system?

Regarding the question of how to quickly build a data analysis system, this article provides four construction principles: clarify the business functions to be statistically analyzed, split measurement indicators/propose analysis hypotheses, find comparative data, and clarify data acquisition channels.

Product design and optimization are based on data and go beyond data. Data is a powerful auxiliary means to reflect product effectiveness. Therefore, before designing products and iterating functions, it is best to plan the data statistical analysis system for this "update" in advance, and continue to observe after going online, and guide further product optimization based on data feedback. However, faced with complex data indicators and functional processes, how to quickly and clearly build a suitable data measurement system is a very important issue.

Whether you are building a complete system or simply measuring the online effect of a function/optimization, generally speaking, you can proceed in the following four steps:

1. Identify the business functions to be analyzed
2. Split metrics/formulate analytical hypotheses
3. Finding comparative data
4. Identify data acquisition channels

1. Identify the business functions to be analyzed

Data is a way to express product effects. Before building a data system, you must first clarify the business type and the verification goal:

A. In terms of business differentiation, different industry fields have very different focuses:

(1) In the field of Internet finance, what matters are the number of users with non-zero balance, the amount of funds held by users, the amount of subscriptions, and the user wealth index;

(2) In the e-commerce industry, what matters is the number of users who make purchases, the amount of money they spend, the frequency of purchases, the repurchase cycle, etc.

(3) Social products focus on user activity, such as daily active time, community activity (comments, posts), etc.

B. In terms of scenario differentiation, is it the effect of functional optimization and iterative verification? Is it a comparative analysis of differentiated competition? Is it about attracting new users, retaining users, and promoting activation based on user scenarios? Or is it to retain lost customers?

Product data systems can generally be divided into two categories: data statistics and data analysis.

(1) Data statistics are more closely related to product function effects and are used to measure a certain product indicator, such as the number of users, DAU, MAU, user purchase amount, etc.

(2) Data analysis is more used in scenarios such as product path process analysis, problem discovery, iteration guidance, and operational effect feedback.

Different businesses and different goals determine what data indicators we should choose for measurement.

2. Split the metrics/formulate analytical hypotheses

After clarifying the business scenarios and statistical/analysis goals, the next step is to continue to split the appropriate measurement indicators; for data analysis needs, it is also necessary to propose analysis hypotheses before this. The following examples illustrate:

A. Analyze the conversion rate of a product feature

Conversion rate can generally be divided into registration conversion rate, subscription conversion rate, scenario user conversion rate, entry conversion rate, etc., which is the process of "users becoming fans of a certain product."

Further breaking down, the key data in the conversion rate chain are: exposure UV → click UV → number of converted users, which corresponds to the behavior of "user sees → user is interested and tries → user is converted".

Put the conversion rate into the context for analysis. There are generally two goals:

(1) Look at the conversion effect of a certain product/operation process, count the conversion rate of new users from contact to final successful conversion, and use the funnel model to express the conversion rate data;

(2) For products that are accessed through multiple channels and multiple entrances, or for product features that are undergoing A/B testing, it is necessary to compare the conversion rates of multiple channels and multiple entrances to compare the effectiveness of each path.

B. Guide product optimization direction through user activity analysis

Activity indicators can be divided into user login/visit frequency, scene setting frequency, subscription/purchase frequency, interaction frequency, etc., which mainly look at the user's retention and activity on the product. For example, the user has logged in 10 times in the past 30 days, and the user has made 30 subscription behaviors in the past 90 days.

After some products/functions are launched, users are converted with a try-it-out attitude, but they churn away after using them a few times. It can be made clear that the focus of the next stage of work is to improve user retention rate. On the contrary, some products have a high user retention rate, but a closer look reveals that most of the users are inactive, which means that the focus of the next stage of work will be to activate users.

C. Monitoring user health

In a sense, the health of a product overlaps with its activity, and some broad concepts can include activity in health.

For example, ARPU value, user mobility, the speed of user upgrades and upgrades under the membership system... are all indicators to measure the health of a product.

Take the speed of user upgrades and upgrades under the membership system as an example: when designing a membership system, the construction of the data system must have three stages: prior planning and calculation, in-process verification and tracking, and post-adjustment. Prior planning and calculations generally require a lot of time and effort, because once the membership rules are released to the public, it is difficult to make adjustments easily. Therefore, adjustments in the third stage should be avoided if possible.

The membership system needs to fit the upgrade and demotion curve. The general effect is that the upgrade starts from easy to difficult, and there is a certain buffer value for downgrade. If the upgrade is too fast and the downgrade is too slow, there will be risks such as system breakdown and unsustainable costs. If the upgrade is too slow and the downgrade is too fast, users will not buy in and there will be no stickiness. The speed of upgrading and downgrading reflects the health of the product.

D. User churn node analysis

After many products have been online for a period of time, they find that the churn rate is getting higher and higher. At this time, you can pay attention to the churn nodes of users in the entire link: at which step do users mainly start to churn, when is the concentrated time point of user churn, and start from the churn nodes to optimize products, appropriately retain churn users and plug leaks, and other operations.

For example, when using the fixed investment function on the mutual finance platform, after observation and analysis, it may be found that the user churn rate is highest before and after the first deduction, and the peak of deductions is accompanied by a peak of churn. The possible reasons are as follows:

• Before deduction: Users may set up the plan with the intention of trying something new, and terminate the fixed investment to “stop loss” in a timely manner before the deduction (actual fund behavior) occurs; or they may lack a sense of security about the deduction.
• After the deduction: There are concerns about the safety and liquidity of funds; there is no guarantee that the bank card funds are sufficient on the deduction date, and the deduction fails due to insufficient funds.

Once the possible problem is found, corresponding user education and guidance can be carried out to reduce the churn rate. A water tank has multiple holes. If you block one or several of them, the rate of water loss will naturally slow down.

E. Launch and verify by assuming potential user portraits

When a product reaches out to users, it always selects potential target users to improve various indicators such as conversion rate. But what is the “target user” and how old are the users? Where is the user located? How often do users make online payments? It is necessary to summarize through multiple delivery attempts. Multiple variables can be assumed, and user packages can be extracted and reached by adjusting the portraits of potential target users. The data differences between multiple delivery channels can be compared to achieve the purpose of verification.

3. Find comparative data

Any evaluation of effect indicators without comparison is just rogue behavior. When launching a product, the product manager needs to see its pros and cons and find a suitable reference to compare the results before making an evaluation and conclusion.

For example: Since a community product was launched, it has a total of 1 million users and an average of 80,000 active users per day.

Is this data good or bad? We need to find a standard for comparison and measurement. Compared with our competitors, is our level of active users considered high? Compared with the average daily active users of 50,000 in the past, there has been a clear improvement.

Therefore, after obtaining the product's online performance data, you need to find a corresponding product as a target. This target can be a competitor, historical experience data, or the default standard in the industry.

4. Clarify the data acquisition channels

After planning the data measurement system, the next step is to collect data before the product goes online and obtain data sources after the product goes online. There is a series of tips: conversion data click stream, user attribute channel number, feedback sampling questionnaire, and broad and universal third party.

A. Conversion data clickstream

When looking at the conversion data of user login visits, purchases and other product paths, the number of users is often selected as the statistical analysis dimension. At this time, using relatively simple clickstream embedding can generally meet the needs; mainly counting the number of users at each step in the product process can form a funnel model.

B. User attribute channel number

In scenarios with user attributes such as subscription amount, purchase quantity and amount, comment interaction, etc., it is necessary to dig deeper. At this time, you can "label" the user with markers such as channel numbers to facilitate tracking and monitoring of the user's subsequent behavior.

C. Questionnaire for feedback sampling

Sometimes, we need to explore the reasons for user behavior, understand the user's subjective operating intentions, and obtain user feedback. It is difficult to draw appropriate guidance through the above purely objective data. In this case, we can choose to conduct a questionnaire; we can obtain sample data for sufficient feedback questions.

D. Broadly applicable third party

There are some third-party data platforms, such as Umeng, TalkingData, WeChat Index, Baidu Index and other data platforms, which are suitable for monitoring data in large industries and large fields. For example, through the WeChat Index, you can know the recent number of online searches for a certain word, the month-on-month increase or decrease, add comparative words, etc.

After explaining the methodology, do you still feel a little confused? Let me give you an example:

For Luckin Coffee, the brand positioning is more on "workplace coffee" and "social coffee". Therefore, at the initial stage of customer acquisition, offline pilot stores were set up in different business districts, and various preferential benefits were used to promote app downloads and stimulate user sharing to acquire customers. At this stage, if data analysis is to be performed, the analysis objectives must be clearly defined:

1. The pilot effects of different offline shopping districts
2. Users’ demands and expectations for workplace coffee
3. App promotion effect/social coffee marketing fission effect

Based on the above three points, we can further propose corresponding analysis hypotheses and find and split measurement indicators:

1. Stores in IT white-collar office areas perform better than stores in downtown business districts; sales at take-out stores are higher than at dine-in stores.
2. The peak hours for ordering are 13:00-14:00 at noon and 15:00-17:00 in the afternoon; takeaway orders delivered within 30-40 minutes are in line with users' psychological expectations, and within 30 minutes may exceed expectations; white-collar workers in the workplace prefer American style, while girls prefer Rena ice series, etc.
3. Social sharing promotion goals, such as the average old user sharing 3 times/week, the average conversion rate of 3 new users/week, the average purchase of 3 cups/week, and the repurchase rate of more than 60%.

In view of the above analysis assumptions, in order to make data comparison more objectively, the above assumptions can be further abstracted into data measurement indicators, such as:

1. The sales performance of each pilot store is measured by the average number of coffee cups sold per day, the sales volume of each type of coffee (price), and the sales volume of takeaway and dine-in stores in each business district, to assist in the decision-making of expansion/reduction of investment;
2. Collect user profile attribute data for coffee purchases, match gender, age, purchase time period, category preferences, customer feedback on delivery orders, etc., and then screen out the main customer groups for the business and improve the details of drinks and services;
3. By using data embedding methods, we can track customers’ app UV/PV, number of cups of coffee purchased, number of times coupon links are shared, number of new users who click on the shared link and register, percentage of users who place a second order within one week of registration, and other data. We can obtain data such as app order conversion rate, sharing rate, sharing click-through rate, number of conversions from old customers to new customers, and repurchase rate, which can be accurately and objectively compared.

Many people use Luckin Coffee to compare Starbucks and Lian Coffee, and use the three as a competitive comparison. Similarly, how to measure the quality of the data submitted by Luckin Coffee can be achieved by comparing the data of Starbucks and Lian Coffee's related businesses, such as Starbucks store daily sales volume and daily sales revenue, Starbucks' sales volume and gift card giveaway volume, etc., and then drawing conclusions by comparing the two data.

According to the four-step rule of building a data system, statistical analysis of product data can be done in advance before and after the product goes online, so that verification effects and function optimization are no longer difficult to start!

Author: The fish that slipped through the net

Source: The fish that slipped through the net