This article will focus on operational activities and talk about how to judge the quality of an operational activity through data analysis, as well as the problems and opportunities of the activity through data insights, to provide a guide to avoid pitfalls and growth directions for the next activity.

Without further ado, let’s get straight to the point.

1. What is a good operational activity?

What constitutes a good operational activity? If the activity objectives are achieved and the cost of achieving the objectives is appropriate, that is, the input-output ratio is relatively high, then we can conclude that this is a good operation activity.

1. Determine whether the goal has been achieved

All data analysis begins with a certain goal. The beginning of the operational activity data analysis is to go back to the beginning and ask ourselves: What is our goal in planning this activity?

The key point of measurement is to find the data indicators that quantify the effectiveness of the activity, which is often referred to as the "first key indicator" or "North Star indicator."

The North Star indicator comes from the goal of this activity. The most common goal of e-commerce promotion activities is to sell goods quickly to boost sales. The corresponding North Star indicator is GMV (total order amount).

Then select the appropriate comparison object and compare to determine whether the North Star indicator has achieved qualitative change. For example, compare with activities in the same period last year, or activities of the same scale and similar positioning.

It is worth noting that the North Star metric is often affected by multiple factors. For example, the growth of GMV may not be due to the effectiveness of the operating strategy, but rather the natural growth of overall traffic.

Therefore, sometimes in addition to looking at the North Star Indicator, it is also necessary to break down the North Star Indicator to determine whether the indicator that is most closely related to the core strategy of this activity has also achieved a qualitative change after the breakdown.

2. Is the input-output ratio appropriate to achieve the goal?

Common input costs in e-commerce activities include direct preferential benefit subsidies, such as red envelopes, coupons, etc., as well as advertising, KOL celebrity cooperation, etc. Through these investments, more traffic and conversions can be brought in, thereby increasing sales GMV.

The ultimate essence of promotional activities is inseparable from commercial value. The higher the input-output ratio, the higher the commercial value. This also explains why promotional activities do not directly reduce the prices of all goods to allow consumers to purchase in the simplest and most favorable way, but instead use coupons, full-discounts and other methods.

The core is actually to enable platforms and merchants to obtain the maximum value output at the lowest possible cost.

Therefore, when reviewing an activity, we also need to look at how much it cost to achieve a certain level of growth, and whether this cost is higher than in the past or similar activities.

2. How to find problems and opportunities through data analysis

When doing a data review report, a common situation is that we calculate whether the North Star indicator has increased or iterated, and give a conclusion that the activity goal has been achieved or not. But, going a step further, the boss asked, what caused such a change in the North Star indicator? Many people are often unable to give an exact answer with certainty, so it is necessary to dig deeper into the data.

1. What is roll-up and drill-down analysis?

Before discussing specific in-depth analysis, we need to first talk about roll-up and drill-down analysis. To simplify the roll-up and drill-down in data analysis, they can actually be transformed into "aggregation" and "segmentation" in the logic tree. For example, splitting the total sales amount into South China, Central China, North China and other regions for analysis is drilling down, and aggregating several major cities such as Guangzhou, Zhuhai, Shenzhen, Dongguan and others into the Pearl River Delta region for analysis is rolling up.

As dimensions are drilled down and rolled up, data will be continuously segmented and aggregated. In this process, we can often find the root cause of the problem.

2. Selectively perform dimension drill-down analysis on the North Star indicator

The idea of ​​drilling down needs to follow the logic of subdividing from macro to micro, layer by layer, but it does not mean that the subdivided data of all dimensions needs to be displayed at each layer. There are many dimensions for drilling down. You need to determine which dimensions to use for drilling down based on your understanding of the characteristics of the business itself and the strategy of this activity. You only need to display the most important segmented data.

The drilling process is not limited to one or several fixed dimensions, but often involves multi-dimensional combination of nodes forking.

When forking, we often choose the dimension with the largest difference for further splitting. If the difference is not large enough, the branch will not be subdivided. Nodes that can produce significant differences will be retained and further subdivided until there is no difference. Through this process, we can find out the factors that affect the changes in the North Star indicator.

Here are some common drill-down analysis dimensions in e-commerce promotion activities:

The most common formula decomposition is GMV=traffic*conversion rate*average order value=UV*UV value. By comparing the indicators after decomposition, we can find the key indicators that bring about the rise and fall of the North Star indicator.

The following data case shows that the main factor leading to the decline in GMV is the sharp drop in traffic UV. The next step is to drill down further from the traffic UV to see which channels have experienced a significant drop in traffic sources.

Breakdown by channel/traffic source

Channels refer to the sources of traffic. Common ones include natural on-site visit traffic, off-site Weibo/WeChat/Toutiao/NetEase and other distribution channels, fission sharing traffic, etc. By comparing the data of each channel with the overall market, you can find the key channels that affect a certain indicator.

If you find that certain channels significantly increase/decrease key indicators, you can further drill down and analyze this channel. In the example below, the key indicators were improved on the APP side, while the key indicators were lowered on the Weibo channel.

Disassembly by terminal

Common terminal segmentation includes IOS/Android, APP/M/PC/Mini Programs, device models used, etc. By comparing the contribution of different terminals to the overall market and horizontally comparing the data indicators of different terminals, the key terminals that affect a certain indicator can be found. In the case below, the APP side-IOS improved the key indicators, while the M side lowered the key indicators.

Breakdown by time/period

E-commerce activities often use different promotional strategies based on time rhythms, such as warm-up/climax/return. If there is such a period-based strategy, the data can be broken down and compared by period to find the periods with significantly high/low data indicators.

In addition to the time period, analysis is performed by time series dimension (month, week, day or hour) to find the special time points that affect the key indicators. Then, review whether there are any differences in the quality of traffic delivered, time-sharing operation strategies, product selection, page design, etc. at that time point, and check for possible factors affecting the key indicators.

Breakdown by business/category

This dimension is often biased towards the operational/procurement and sales perspective, focusing on the output of key indicators of a certain business/category. It needs to be combined with specific product selection and inventory strategies to analyze whether it is effective.

When the number of categories is large, you can refer to the Pareto model in the figure below to observe the data. The main point of observation is to combine the volume and exposure of each category to determine whether the change in category has brought corresponding sales output. If it is too low or too high, it is worth paying special attention to, and the subsequent stocking and exposure strategy can be optimized accordingly.

Disassembly by functional module

This dimension is most closely related to page design, and involves the arrangement of the page content framework and the function and information presentation of each specific module. You can observe the contribution of each module (floor) to the North Star indicator or its broken down indicators. Pay special attention to those with high exposure at the front but low contribution (low input-output) and those at the back but relatively high contribution (high input-output) to further explore the reasons.

Breakdown by user group

Conventional population classifications include those based on demographic information such as gender, age, marital status, region, and new and old users. It is worth noting that some activities will carry out more refined user segmentation operations, such as making differentiated strategies according to groups with special characteristics such as the elderly, Generation Z, and small town youth.

When choosing a specific population dimension to drill down, the primary reference factor is whether the activity strategy has differentiated operations for this dimension. The strategies mentioned above include refined operations based on user groups, so you must drill down according to this dimension to see whether each population strategy is effective and whether it brings about growth in the North Star indicator.

The second is whether the difference in population data under this dimension is large enough, and whether we have the resources to develop this dimension in the future. For example, if the proportion of female users in a certain activity is significantly high, but the GMV contributed is significantly low, we can try to conduct further drill-down analysis on female users to determine whether we can use existing resources to improve the sales conversion of female population.

The method of drilling down into different groups of people can be more refined (sufficient data source support is required). In addition to directly comparing the core indicators between different groups of people, the behavioral path funnel, purchase preferences and other characteristics of the front-end and back-end links of a certain group of people can also be analyzed, and combined with qualitative user research, the problems that this group of people may encounter and the opportunities that can be further leveraged in the future can be explored.

3. You can also conduct cross-analysis around key dimensions and combine them with other dimensions

After completing the multi-dimensional drill-down analysis, you can also try to perform multi-dimensional cross-analysis on certain particularly important dimensions.

For example, by crossing user gender and channel, you may find differences between male and female users on WeChat and male and female users on Weibo. By crossing category and time period, you may find that different periods are more suitable for the outbreak of categories with different characteristics.

But we need to have an expectation that this step may be difficult to implement in practice.

On the one hand, it has high requirements for data extraction and processing, and requires professional data analysts to devote more energy to support;

On the other hand, the judgment of cross-objects requires sufficient business sensitivity and industry experience, in-depth thinking about the current situation, and insight/inference of some signs that there is a high probability that two factors are related.

Otherwise, it is possible that a lot of time and resources will be spent on cross-analysis but no conclusions of practical value will be drawn.

4. Process indicators should not be ignored

Why look at process indicators?

Data indicators can be divided into result indicators and process indicators. Process indicators are measures of the intermediate process links that produce a certain result. Result indicators are often derived from the business objectives of the activity and are used to measure whether the business objectives have been achieved. They are more often used for data review after a certain stage has ended.

For example, in a promotional activity, sales volume and order volume are result indicators, while the layers of visit traffic, clicks, add-to-cart volume, and successful payment volume that bring about these orders are process indicators.

But in daily data tracking, what is more valuable is to make timely adjustments based on the current data situation to ensure that the results indicators meet expectations.

At this time, process indicators become more important, because result indicators are just results, but process indicators can trace the detailed problem links, guide the optimization of the links, and bring about the improvement of result indicators.

How to find process indicators

Process indicators can be summarized from the response relationship of project execution and the user's touchpoint path. For example, in a big promotion event, many links will be involved, including event page planning, procurement and sales, market promotion, user understanding of the event, ordering and payment, logistics distribution, customer service, etc. By sorting out the stakeholders and actions of the entire event in chronological order, you can pick out the links that are strongly related to your role and their corresponding process indicators.

How to use process indicators

The most important use of process indicators is to track and monitor in real time to determine their health level. The judgment criteria can be what the data patterns of the business are during the same period, whether the current stage of progress is lower or higher than normal, and make strategic decisions based on this.

For example, the pattern of 618 in the past was that there was a burst of traffic in the early morning of June 1st, but this time the burst was not obvious, so it is necessary to check whether the delivery of each channel is proceeding normally and whether it is necessary to increase the budget investment in time.

5. Verify the detailed strategies

This step is an essential part of every data analysis. It is relatively simple and direct. The main idea can refer to the OSM (Objective-Strategy-Verification) model introduced in the previous data series articles. Select appropriate measurement indicators according to the phenomenon, and then conduct comparative analysis of the data before and after the strategy.

I won’t go into details.

3. Several points to avoid

1. All data comes from the database, ignoring other data (external data, surveys, etc.)

The data in the database often cannot reflect all quantitative needs. For example, a user places an order in this event, but questions such as whether his shopping experience is good or bad, whether he is willing to continue to participate in such activities in the future, and how he feels compared with competitors are more suitable to find typical users for in-depth interviews or conduct extensive questionnaires.

2. Indiscriminate data analysis

There are many angles of data analysis, and different analysis angles are suitable for different analysis goals. You should choose the appropriate analysis dimension around the target problem, rather than running all of them according to the template.

In addition, high-quality data cannot be obtained in some links, and sometimes decisions need to be made. It is better not to do this analysis to avoid wrong decisions due to inaccurate data.

3. Have a target result first, then analyze the data, report good news but not bad news (only for reporting)

In some reporting scenarios, in order to help superiors quickly reach conclusions, the detailed analysis process and details are often not presented, but key conclusions are picked out to form a report.

As reporters, we often unconsciously report good news but not bad news, prioritize good performance, and ignore or skip some complex and difficult to explain issues. If this continues for a long time, you may be persuaded and not pay enough attention to the existing problems.

4. Data cannot answer all questions. Understanding and thinking about the business is sometimes more important (studying the essence of the business is more important than studying KPI indicators)

In most data analysis scenarios, KPI data is often the top priority for everyone. Once a decline occurs, the problematic data link will be located by continuously drilling down the dimensions. At this time, from the perspective of KPI, we will think about how to solve the problem to improve KPI, and then the work ends here.

The disadvantage of this model is that it will make people too superstitious about data and lazy in thinking, and they will only do things to remedy the situation instead of preparing for a rainy day.

Sometimes the problems behind data issues cannot be simply explained by current numerical values, but require people to step out of the current situation, look at industry development, study market trends, and understand user psychology to gain insight.

Therefore, in addition to looking at the data, everyone needs to take time to think about and explore the essential issues of the business.

Author: Hu Chenchuan

Source: Hu Chenchuan