Three principles for promotion data analysis!

The idea of ​​promotion and delivery analysis is relatively fixed, focusing mainly on user quality and cost-effectiveness of customer acquisition. After analyzing the problem, finding the cause requires a better understanding of the promotion and delivery scenarios and various influencing factors.

The difficulty in analysis lies in the fact that the promotion and delivery scenario is very deep and complicated, so most people will only be exposed to a small part of it. When problems arise, it is often difficult to guess what went wrong.

• On the one hand, in terms of analysis dimensions and methods, we generally make some cross-evaluation plans based on user quality and return on investment.
• On the other hand, when abnormalities are found in the data, how should we investigate and analyze them?

First of all, today’s promotion channels are extremely diverse. In addition to general advertising space in content platforms or app stores, in recent years, for example, mobile video advertising and programmatic advertising have grown rapidly. However, from an analytical perspective, these emerging methods are just different specific promotion methods and do not affect the goals of our analysis. Therefore, we can remain unchanged in the face of change.

From the perspective of analysis methods, we can look at it from four dimensions, using a visual rendering. This visual rendering represents information in four dimensions.

First, let’s look at the channels. Each colored bubble represents a traffic channel, and the size of the bubble represents the amount of traffic. The location of each bubble is the intersection of two dimensions of information. The vertical axis represents the overall user quality, from low to high, and the horizontal axis represents the overall return on investment of this channel, from low to high.

User quality is a measure of effective users. We can determine the key behaviors of users based on the characteristics of our specific product business, set scores for the triggering frequency of different behaviors, model the entire thing, and finally score the overall quality of channel users. The investment amount indicator hopes to obtain higher returns by improving the utilization efficiency of investment assets.

Channel acquisition analysis generally has the following three basic principles:

• The first is to clarify the goal. Our analysis and measurement process is not only to make a judgment. In fact, the core is to improve the cost-effectiveness of the promotion. After the evaluation, we need to analyze the reasons behind it. If it is a problem with the characteristics of the channel users, on the one hand, we can adjust our operation strategy for this channel user, on the other hand, we can cooperate with the business department to negotiate with the promotion channel to reduce the price of the promotion.
• The second is not to throw away any traffic entrance lightly. On the one hand, users are truly investing, and as long as the rate of return is not extremely low (as low as a negative number), the accumulation of users itself is still meaningful. On the other hand, during the promotion process, there may be connections between multiple channels. For example, a user clicks on an ad on Zhihu, enters the h5 and takes a look but finds it boring, so he may not download it. However, two weeks later, he sees a colleague from the company playing it on WeChat Moments, and downloads it again at the invitation of his colleague, becoming a high-quality user. In this process, Zhihu itself plays a warm-up role and gives customers a good impression. Without the warm-up of this channel, the subsequent user acceptance may not be so high. Therefore, it is recommended that you do not completely reject a certain channel.
• The third is to make more strategic adjustments and continuously optimize. Even if you want to remove a certain channel, it is recommended that you gradually reduce the investment by 25% first to see if it has little impact on other channels, and then gradually reduce it and finally remove it. If it is found during the process that its reduction has caused a certain impact on other channels, timely adjustments should be made and an overall evaluation should be conducted.

Then let’s talk about the analysis method of anomalies in data related to new users. When we find that the data is fluctuating or abnormal, we can proceed in three major steps:

2.1. Preliminary screening

At this time, we should first check the turning point of the data trend fluctuation, at what time the data is abnormal (for example, it increases or decreases), and then refine it to the minute-level time dimension. At the same time, we should check some actions related to our product operation and marketing promotion, and make a business association from the time point to find the cause.

There is another dimension, which is that anomalies are prone to occur in the user's regional city. In terms of proportion, it is recommended that you confirm the places where anomalies are likely to occur in the data initially screened in the first step. At this stage, we can find some clues of data anomalies, which can help us narrow the scope of analysis.

2.2. Segmented business screening

Then you can proceed to the second step of segmenting the business for screening. Simply put, it is to look at the data related to different channels, sources of promotion activities, sources of product versions, and user quality based on the dimensions analyzed in the first step. For example, the user's stay time, the active ratio of key business behaviors, and the data of next-day/7-day retention are all key points to be compared. By analyzing these dimensions and comparing them with past data, we can generally find the reasons for data fluctuations. There are actually two reasons for this:

• On the one hand, there is a problem with the product itself, which leads to abnormal traffic data, which is usually reflected in the data differences between app versions.
• On the other hand, there is a fluctuation in the quality of users in a certain channel, which involves another question: can the data fluctuation be attributed to these two reasons? For example, if we assume that there is a problem with product design due to data differences between versions, is this really acceptable? Could there be a problem with the packaged version, such as a problem with the quality of the traffic? Or could it be that the channel itself is like this, but the channel party, due to some reasons or periodic reasons such as activities, the traffic it imports to us is more accurate?

2.3. Comparison with natural traffic

At this time, it is still recommended that you compare the data of the segmented channels with the natural traffic in the same period, which will lead to our third step. When compared with natural traffic, if the difference is small and no product iteration has been done during the abnormal period, then it is basically a problem of traffic quality. If the difference is relatively large and there is product iteration, then most of the impact will be caused by product revisions.

At this time, comparing it with the natural traffic can actually help us make a relatively objective assessment no matter what differences we find in the business segmentation stage.

2.4. Is the traffic fraudulent?

Finally, in the customer acquisition stage, there is a relatively unique scenario, which is how to identify situations where some channels are inflating volumes or cheating.

If we have doubts about the data in this regard, we can first analyze the behavior trajectory of new users. Basically, we can find a cheating channel in 80% of cases. Generally, the new users do not trigger any behavior or only trigger a few fixed behaviors. The behavior trajectory pattern is relatively normal, and it is very simple for users. A brief summary analysis may be able to show that after all, most of the brushing is done through technology, which generally simulates the startup behavior and a few fixed browsing behaviors. It is difficult to achieve the same random effect as real user behavior. This is our screening based on business. On the other hand, in terms of technical screening, you can query one by one based on the IP device model and screen resolution. Basically, it can cover most of the scenarios, especially the screen resolution. After all, the IP and device model can be modified through simulation means, and although the screen resolution can also be modified in essence, it is often ignored by people who flash the device. It is also a very obvious feature. It is recommended that everyone pay attention to this aspect.

This customer acquisition scenario, from design to delivery, involves a budget of real money, and the industry is quite complicated, so this part can only be regarded as a starting point for everyone.

Author: Zhuge Jun

Source: Zhugeio