Case Analysis | How to build a correct and efficient data operation system?

As the concept of refinement continues to gain popularity, the concept of "data operation" has received more and more attention. But what are the correct data indicators, how to collect data correctly, and how to use data to drive business growth? These common data myths plague most product, operation, marketing and even management teams. In today’s article, I will share with you how to build a correct and efficient data operation system.

1. What is Data Operations?

"Data operation" has two meanings. In a narrow sense, it refers to the job position of "data operation". It is a branch of operation, just like content operation, product operation, event operation and user operation. Engage in data collection, cleaning, analysis, strategy and other work to support the entire operation system to develop in a refined direction. In recent years, more and more Internet companies have opened "data operations" jobs, mainly in front-line operations departments. Unlike the position of data analyst, data operations focuses more on supporting front-line business decisions.

In a broad sense, it refers to the way of thinking of "using data to guide operational decisions and drive business growth", that is, data-driven operations. An essential skill or way of thinking for operations, it generally refers to discovering problems, solving problems, improving work efficiency, using data to guide operational decisions, and driving business growth through data analysis.

To sum up, data analysis is an essential skill for data operations.

1. Why data analysis is an essential skill for operations

Baidu Index Trends for Data Analysis

From the external environment, we can see that from 2011 to now, the Baidu Index of data analysis has continued to rise. 2014-2015 is an important node because since the beginning of mobile Internet in 2009, the first and second waves of smartphone replacement have driven the development of mobile Internet. The entry of a large number of companies has made the cost of growth very high, so refined operations through data analysis have become increasingly important.

Data analysis requirements for operating JD

We can also see from recruitment that we have looked at the JD (job description) for operations of many companies on the market. Whether it is content operation, product operation, event operation or user operation, data analysis capabilities are required.

2. Data Operations Responsibilities

We found 100 data operation recruitment JD (job description) texts, used R to segment them, and drew a word cloud.

Data Operations Responsibilities

It is not difficult to see from this word cloud that "data analysis" is the core work of data operation. The table on the right shows the top-ranked keywords and their frequency of occurrence.

The word "data analysis" appeared 106 times in 100 JDs, far ahead. Next are words such as "analysis report", "providing data", and "data report", which also shows that providing data reports and analysis reports are important tasks of data operations. Building "data indicators", "discovering problems" in a timely manner, and providing "solutions" are also high-frequency words in JD. In this way, the specific responsibilities of the data operation position are clear at a glance.

In summary, we can summarize it into three levels: data planning, data collection, and data analysis:

1. Data planning : collect and organize data requirements of business departments and build a data indicator system;
2. Data collection : collect business data and provide data reports to business departments;
3. Data analysis : In-depth analysis of business data through data mining, data models, etc.; provide data analysis reports, locate problems, and propose solutions.

Next, we will expand on these three aspects and explain in detail how to build a correct and efficient data operation system.

2. Data Planning

Data planning is the foundation of the entire data operation system, and its purpose is to figure out "what you want". Only by clarifying your purpose and what kind of data you need can the subsequent data collection and analysis be more targeted.

There are two important concepts in data planning: indicators and dimensions.

1. Indicators

Index, also called measure. Indicators are used to measure specific operational results, such as UV, DAU, sales amount, conversion rate, etc. The selection of indicators comes from specific business needs, events are summarized from the needs, and indicators are mapped from the events.

Indicator system

Indicators are divided into quantitative indicators and qualitative indicators. Web's pv, uv, visits, App's DAU, NDAU, etc. are all quantitative indicators; average visit time, visit depth, bounce rate, etc. are qualitative indicators.

(1) How to select core indicators

Here we introduce a concept - OMTM, OMTM (One Metric That Matters), the only important indicator, also known as the North Star indicator.

Four major criteria for selecting OMTM:

1. Closely aligned with business objectives;
2. Reflect the value needs of customers;
3. The indicators are simple and easy to understand;
4. Ability to calculate aggregates.

Here we can cite a case, the famous video social sharing application Viddy, authorization can be created by logging into Facebook to create an account and share, just like the third-party accounts such as WeChat and QQ in various common domestic apps.

A Case Study of Choosing the Wrong Metric - Viddy

In the early stage, they used "the number of accounts created" as the core indicator. We can see that after authorizing Facebook login, the number of visits soared in the first half of 2012, but plummeted in the second half of 2012. That’s the problem. The people at Viddy thought account creation was the right metric, so they did everything they could to increase it. In fact, they have not discovered the focus of the business.

In contrast, Google+ has grown its user base to 170 million using its own method, which is to share interesting content with friends through email. Google+ focuses on the user quantity indicator of "going to Google+ and sharing at least 2 updates per week". They focus on delivering the value of the product to users rather than just increasing a certain number.

(2) How to plan core indicators

E-book landing page example

Take the e-book download landing page made by GrowingIO as an example.

The selection of indicators comes from specific business needs, events are summarized from the needs, and indicators are mapped from the events.

Steps to Select Dimensions

Starting from the business needs of the content landing page, the process of analyzing and selecting indicators is as follows:

1. Clear requirements : Analyze data on pages to increase e-book downloads;
2. Inductive events : A user downloading an e-book is the final result of a series of events, including clicking on the promotion link, visiting the download page, starting to fill in information, and completing the download;
3. Corresponding indicators : download volume = visit traffic CTA click rate registration conversion rate.

Through the above analysis, we can conclude that download volume is OMTM (One Metric That Metter). At the same time, the entire indicator system includes three actionable indicators: visit traffic, CTA click-through rate, and registration conversion rate. Only based on actionable indicators can the core indicators be better optimized.

2. Dimension

(1) What is a dimension?

Dimensions are attributes used to break down metrics, such as ad source, browser type, visiting region, and so on.

Dimension classification

(2) Multi-dimensional analysis

Landing page traffic surges

For example, when we were observing the data every day, there was a sudden increase in traffic at 5 pm one day, and we wanted to find out the reason.

Visit source analysis

First, we analyze the visit source dimension. We find that starting from 5:00, the traffic from WeChat suddenly increased.

Landing page analysis

Then, we break it down from the landing page dimension and find that the main landing pages for traffic are pages E and G. Finally, we can draw the conclusion: at 5 pm, a large amount of traffic suddenly poured into pages E and G from WeChat. After synchronizing with the content colleagues, we found that they had launched an activity on WeChat that landed on pages E and G.

As can be seen from the case, multi-dimensional analysis can enable us to more clearly uncover the real reasons behind the data appearances.

(3) How to choose the analysis dimension?

The principle of selecting dimensions is: record those dimensions that may affect the indicators, and try to record comprehensive and multi-dimensional data.

Sharpening the knife does not delay the chopping of wood. Data operations require constant communication with business departments (marketing, sales, operations, products, etc.). Only by doing a good job of data planning can subsequent data collection and data analysis be more efficient.

3. Data Collection

Data collection is the basis of data analysis. Traditional data collection is a very time-consuming, energy-intensive and labor-intensive task, and is a huge hurdle for many companies. In the past, the entire data analysis process often spent 80% of its time on data collection, and less than 20% of its time was spent on data analysis.

1. What data should be collected?

Internet Trends

From the first half of the Internet when traffic was king to the second half when traffic became increasingly expensive, the cost of acquiring users has become increasingly higher. Around 2013 and 2014, the cost of activating a user for a tool app was only a few cents. In less than two years, the cost of acquiring a download reached several dollars. The cost of activating a user of a financial app can reach hundreds of yuan. Therefore, enterprises began to shift from extensive operations to refined operations, and the data they focused on also expanded from simple channel traffic data to more analysis of user behavior data.

So at present, the collection of user behavior data has become a more important part of data operations.

User behavior is composed of events.

event

These events include time, place, task, person, content, and interaction.

2. How to collect data

(1) Data collection plan

Data collection plan

There are currently three common data collection solutions, namely, point-burying, visualized point-burying, and no point-burying.

Buried point collection

Tracking, also known as tracking, is the process of manually adding statistical codes to products (web pages, apps, etc.) to collect the required data. Management can be further divided into front-end management and server management. If you want to collect the number of user registrations, you need to load the corresponding statistical code at the registration button. This is the method used by tools such as Google Analytics and Baidu Statistics.

However, due to the large workload and long cycle of point tracking, and the ease of missing or wrong tracking, point tracking has become a major pain point for data practitioners.

Visualization of buried points

Visual tracking is an extension of tracking, which replaces manual tracking with visual interaction. This approach lowers the threshold for users and improves efficiency. But whether it is point tracking or visual point tracking, data operations needs to play a connecting role: collecting data requirements of the business department, writing requirement documents, and submitting point tracking requirements to the engineering department. In essence, it is still a point tracking solution.

No embedding point subverts the traditional "define first and then collect" process. You only need to load an SDK to collect the full amount of user behavior data, and then you can flexibly customize the analysis of all behavior data. Some time ago, the foreign Mixpanel also launched a no-embedded solution.

Compared with the solution of burying points, the solution without burying points has low cost and high speed, and will not cause wrong burying or missing burying. No-embedded point is becoming the new darling of the market, and more and more companies are adopting the no-embedded point solution. In a scenario without tracking points, data operations can break free from the constraints of tracking point requirements and spend more time on business analysis.

(2) Data Visualization

After the data is collected and processed, the next step is to visualize it. The main forms of data visualization in operational applications include: charts, graphs, and data dashboards. Building a data dashboard is another task after data reports. It refers to displaying key business indicators (KPIs) and related data indicators in a panel and presenting them in the form of visual graphics. Data dashboards are often connected to the company's BI system and are part of data visualization.

Dashboard

The picture above is a dashboard made with a data analysis product. Everyone, or every business, needs to present the data in the most intuitive way and drill down layer by layer according to the chart to find problems, so a customizable dashboard is very important.

4. Data Analysis

Data analysis is the key task of data operation. The previous data planning and data collection are all for data analysis. Our ultimate goal is to identify problems, propose solutions and promote business growth through data analysis methods.

Therefore, this is why we recommend no embedding, because we hope to change the previous situation of "80% of the time is spent on collecting and cleaning data, and less than 20% of the time is spent on data analysis" to "80% of the time is spent on data analysis" and spend time on more valuable things.

Analysis methods and application scenarios

The data analysis method we choose should be combined with the business scenario. The above table summarizes the common operational data analysis methods and application scenarios, such as the UTM we use for advertising and channel tracking, the conversion funnel analysis, etc.

Unlike data analysts, data operations positions weaken the requirements for programming and statistics, and place more emphasis on the flexible use of analytical methods based on existing tools.

Below we list some common data analysis methods.

1. Data analysis methods

(1) Dimensional segmentation

Dimensional breakdown

It is difficult to find problems with a single data indicator. We need to start from multiple dimensions, such as region, platform, browser, access source, etc., break down indicators and locate problems.

(2) Funnel analysis

When users use a product, there is naturally a series of conversion paths, such as registration, ordering, downloading, etc. Operations require the conversion rate of each path, including the total conversion rate and the conversion rate of each step.

The conversion funnel tool displays each step of the conversion path in a visual way. Operations staff can focus on the areas with the greatest churn, as this is often where the ROI of optimization efforts is the highest.

In addition to breaking down the conversion rate of each step horizontally, we can also observe the changing trend of the conversion rate of each step from the time dimension.

Funnel analysis

For example, it is not difficult to find from the above figure that the conversion rate of the first step of the registration process on a certain day dropped significantly, thus affecting the overall conversion rate.

(3) Heat map

A heat map is a very common data analysis chart, also known as a heat map, which is a graphic that displays the user's page click location or the user's page location in a special highlighted form. With the help of heat maps, you can visually observe users' overall access and click preferences.

There are currently three common types of heat maps: heat maps based on mouse click locations, heat maps based on mouse movement trajectories, and heat maps based on content clicks. The principles, appearances, and applicable scenarios of the three types of heat maps are different.

Heatmap based on content clicks

The above picture is a heat map based on content clicks, such as the data analysis product GrwoingIO heat map, which records user clicks on web page content and automatically filters out clicks on blank areas of the page (without content and links). The heat map based on content clicks tracks changes in content and records users' click preferences for content within a relative period of time.
From the heat map, we can easily see which locations have high traffic, high user attention, etc.

2. Data-driven analytical process

Many star companies have invented great techniques in data-driven development, but any technique has its own life cycle. Often when you discover a technique, it has become an industry standard and may not necessarily match your business. So instead of relying on skills, rely on processes so that the team can operate as efficiently as a machine.

The most important thing in data analysis is to establish a data-driven process. A complete process can help you quickly locate and solve problems. Start by setting growth indicators, find small focus areas, analyze data, propose hypotheses, prioritize, conduct experiments, analyze and optimize, and continue the cycle until you find the problem and drive some improvement in your indicators.

Driving Process

1. Clarify your goals;
2. Analyze the current situation and existing problems according to the goals;
3. Propose ideas that may solve current problems or achieve goals;
4. Prioritize testing an idea;
5. Start testing and verify or disprove our ideas through experiments.

Then start a new round of analysis, hypothesis, optimization, and testing to achieve growth through continuous optimization.

6. Case Analysis

The following uses a real case to analyze "how to build a correct and efficient data-driven operation system" in actual business.

In our content operations, we created a content-themed landing page, hoping to track the effectiveness of the landing page and optimize the page.

Landing Page Example

On the left is the full picture of the landing page, which includes the three essential parts of a landing page: Hero Shot, Benfits and Call to Action; on the right is the home screen of the landing page.

1. Data planning

The ultimate goal of the entire content landing page is to get more users to complete the download behavior, so "e-book downloads" is our OMTM. By breaking down this indicator, we get the following formula:

Downloads = Number of visitors CTR Registration conversion rate

There are two types of landing pages: click-through landing pages and lead-generating landing pages. This landing page is a click landing page. It serves the purpose of traffic distribution and provides traffic for the lead generating landing page.

In combination with the purpose of our content topic, the click-through rate of [download e-book] is the CTR in the formula.

2. Data Collection

Collect data through circle selection without embedding points, and build a dashboard for the entire landing page based on the indicators.

Landing page dashboard

3. Data Analysis

Two levels of data analysis

Data analysis is divided into quantitative analysis and qualitative analysis.

(1) Quantitative analysis

Quantitative analysis is important in growth and serves as a guide. It will tell you where there are opportunities for growth and where you can do testing. The second is to measure the results to help you adjust your direction.

Landing page quantitative analysis

For example, if we use the heat map tool to view the clicks on the entire landing page, we can get the following data:

• Landing page bounce rate 0.36
• 【Download e-book】Click rate 0.48

Many times when we pursue growth, we want to influence and change user behavior, but one thing to remember is that a user is always a person, not just a piece of data. Sometimes it is necessary to observe the results through some data, but sometimes qualitative analysis is also very important.

(2) Qualitative analysis

Based on qualitative analysis, interviews, and user research, we can draw several conclusions:

• The main color of the page is too light, not bright enough, and the contrast between the text and the background is not prominent;
• The text information is too sparsely arranged, and you need to scroll down multiple times to read a page.

(3) Propose a hypothesis

Based on the above qualitative and quantitative analysis, we propose the following hypotheses:

• Changing the background color of the landing page can help reduce the bounce rate;
• Adding more [Download eBook] buttons will help increase click-through rates;
• Adding links to e-book images can help increase conversion rates;
• Reducing page white space and increasing information density can help improve conversion rates.

(4) Prioritization

How do we evaluate whether our ideas are feasible? And which idea should be tested first?

Sean Ellis, the father of growth hacking, summarized a set of evaluation methods – ICE, which scores from three perspectives: Impact, Confidence, and Ease:

1. Impact : How much impact does this idea have on our business growth? If the impact is very large, I will give it 10 points; if the impact is weak, I will give it 2-3 points.
2. Confidence : Are you sure that this idea will work? Again, rate it from 1-10, with 10 indicating that you are confident enough that the idea is valid.
3. Ease of implementation.

Combining the above three perspectives, we can arrange more reasonable priorities. So we made "changing the background color of the landing page to the color of the homepage" and "adding two new [Download e-book] banners" high priority and started the experiment.

(1) Start the experiment

1) Experiment 1- Change the background color of the landing page

Experiment 1 Revision

The data after the revision verified our idea. The bounce rate dropped from 0.36 to 0.12, and the "download e-book" conversion rate increased from 0.48 to 0.61.

2) Experiment 2 – Add 2 more [Download eBook] banners

Experiment 2 Revision

Before the revision, we had 2 CTA Banners, and we increased it to 4 to ensure that there was a CTA after each screen was browsed. After the experiment, the bounce rate increased from 0.12 to 0.13, which is a normal data fluctuation. The conversion rate increased from 0.61 to 0.83.

The data from the two experiments were analyzed and optimized, and then other hypotheses were verified experimentally. After repeated experiments on other hypotheses one by one, the overall conversion rate increased by 124%.

From this, we conclude that there is a positive correlation between experimentation and growth. Here are two foreign examples to illustrate this.

Growth Examples

Twitter once achieved very rapid growth, but then stagnated. In 2010, Twitter formed a new team. A new product VP came and he felt that Twitter was not testing enough. “We’ve only done a few tests in months, which is too little. We must do at least ten tests a week! 'Once they increased the frequency of testing, Twitter's growth resumed.

Mobile application product promotion service: APP promotion service Qinggua Media advertising

The author of this article @郭淑明 is compiled and published by (Qinggua Media). Please indicate the author information and source when reprinting! Site Map