How to evaluate the pros and cons of multiple promotion channels?

I have always wanted to write an article like this to talk to you about how to evaluate the pros and cons of multiple promotion channels .

Today, I would like to take this opportunity to take the promotion of a certain financial APP as an example to stimulate discussion for your reference.

1
Input Data

Duration: 2 weeks

Platform: App store , SEM , information flow , total 8

Daily budget: 150,000

2
Select the dimensions of the data model

I selected four dimensions, namely: traffic , cost, stability and potential, which I will explain one by one below.

flow

Value: represents the channel 's ability to acquire customers, is directly related to the core KPI of operators , and is also an important indicator for distinguishing large channels from small channels.

Data: Arithmetic mean of customer acquisition volume.

cost

Value: represents the investment cost of the channel. Under the condition that user quality is relatively stable, the customer acquisition cost is directly related to ROI.

Data: Weighted average of customer acquisition costs.

Stablize

Value: represents the possibility of long-term stable delivery of the channel, and indirectly affects the operator's control over the budget consumption and KPI completion progress.

Data: coefficient of variation of cash consumption, coefficient of variation of customer acquisition volume, coefficient of variation of customer acquisition cost.

Note: Coefficient of variation = (standard deviation/mean value) × 100%

potential

Value: represents the future growth space of the channel, such as the gap with comparable competing products. Indirectly affect the operations staff's planning for future KPI growth.

Data: The budget or traffic of benchmark competitors in the same promotion channel.

3
Data standardization, calculation of dimension scores

The data normalization here adopts Min-max normalization.

Let minA and maxA be the minimum and maximum values ​​of attribute A respectively. Map an original value x of A to a value x' in the interval [0,1] through min-max normalization. The formula is: new data = (original data - minimum value) / (maximum value - minimum value).

flow

cost

Note: Cost is different from traffic. The greater the traffic, the higher the score. The lower the traffic, the higher the score. Therefore, based on Min-max, further processing is performed to subtract the result of Min-max from 1 to get a new score.

Stablize

Note: As with cost, the smaller the coefficient of variation, the better, so subtract the result of Min-max from 1 to get a new score.

potential

1) Be sure to select competitive products so that the data is referenceable;

2) Compare the daily budget data of competitors. You can consider asking agents, people in the industry, and the media. You don’t have to ask for particularly precise numbers.

3) The average daily number of new installations is very valuable to app stores because there is no significant difference in the conversion rate from installation to activation and registration. This data can be obtained from third-party data service providers, such as QuestMobile.

Due to insufficient data on the potential dimension, we will only use the first three dimensions to build the data model for now.

Now it comes to the issue of additional weights. Here are three solutions for reference only:

Taking ROI orientation as an example, the following scoring model is derived:

Note: Comprehensive score = traffic score + cost score + stability score

The conclusion is obvious. Based on ROI guidance, the channel with the best overall effect is: App Store B, followed by App Store A, Information Flow A, and App Store D.

Q: To evaluate the effectiveness of a channel, aren’t the two indicators of quantity and cost enough? Is it necessary to be so complicated?

A: Yes. In actual delivery work, it is easy to forget the fundamental goal because of one temporary goal or another. Establishing a scoring model can evaluate the effect more rationally and objectively, and when your thinking goes astray, it will decisively pull you back. Different weights can be set according to different KPIs. I hope everyone can establish a scoring model that suits them.

Q: When attaching weights, just using the simple 1, 2, 3, is that too arbitrary and unreliable?

A: I also struggled with this problem for a long time. Later, after referring to a lot of information, I decided to use the simplest 123. It is important to do it first, 1 is much bigger than 0.

On April 19, 2017, the Harvard Business Review website published an article titled “Creating Simple Rules for Complex Decisions.” Daniel Kahneman also specifically discussed this idea in his book Thinking, Fast and Slow. In simple terms:

1. Select several factors (no more than six) that you think have a significant impact on the results.
2. Score each factor according to a uniform standard, such as a full score of 10.
3. Simply add up the scores to get the total score.

One of the most interesting facts in the book is that if you use more complex scoring methods, or calculate weights, or use some big data statistical algorithms, the results you get will not be much better than simple methods.