How to estimate the expected effect of a product solution before designing it?

Before starting to work on new product plans or iteratively optimized product plans, you need to estimate the expected results to measure whether the plan is worth starting. So, how do you estimate the expected effect of a product solution? Below, I will explain in detail.

1. Define the effect

After a product solution is launched, the first thing to worry about is the online effect, and the success or failure of the product solution is evaluated by the quality of the effect.

Generally speaking, the effectiveness of the evaluation can be measured from the following three dimensions.

• Basic traffic indicators: PV, UV, page dwell time, bounce rate, conversion rate, etc.
• Business indicators: order volume, average order value, GMV (gross merchandise volume), GP (gross profit), etc.
• Operational indicators: Operational indicators can be divided into two categories. One is product operation indicators, including exposure, sharing rate, number of new users, activation, and conversion rate; the other is internal operation indicators, which mainly measure the improvement of internal staff work efficiency.

The above indicators can measure various types of products, such as tool products, e-commerce products, etc.; they can also measure various end products, such as B-end or C-end products. The specific indicator to be chosen to define the effect requires selecting one or more indicators for comprehensive measurement based on the purpose of the product solution - that is, specific analysis of specific issues.

2. Define expectations

Expectations are estimates of future conditions. The purpose of defining the expected purpose here is more to limit the scope of the effect evaluation from a time dimension.

The expectation can be: one day after launch, one week, half a month, one month, etc.

Generally, the expected range is relatively small. In theory, the best effect is often achieved in a short period of time after going online. However, there are exceptions. For example, for products with a long transaction cycle, it is not easy to measure GMV, GP and other business indicators in a short period of time. In this case, you can consider changing the effect indicators or expanding the expected range.

3. Selecting the Estimation Method

1. Estimation Method

Let’s use the eggs laid by an old hen to introduce the following estimation schemes.

• Project background: Zhang San's 5-jin free-range chicken lays eggs that are basically 50g in summer. He wants the chicken to lay bigger eggs in winter.
• Project purpose: To increase the egg weight of free range chickens.
• Project plan: Buy super feed to feed old hens.
• Expected results of the project:? ? ?

Method 1: Expert judgment

It refers to the valuable insights provided by experts on the project environment and information on previous similar projects based on historical information. Expert judgment can also inform decisions on whether to use multiple estimation methods in combination and how to reconcile differences between methods.

Expected results of the project: The eggs laid by chickens fed with super feed will weigh 70g.

Basis: Super feed experts concluded based on historical experimental data that the nutritional content of super feed is 1.4 times that of ordinary feed.

Method 2: Analogy estimation

It refers to estimating similar parameters or indicators of the current project based on the parameter values ​​or scale indicators of similar projects in the past. When estimating, this technique uses the actual results of similar projects in the past as a basis for estimating the results of the current project.

Expected results of the project: The eggs laid by chickens fed with super feed weigh 65g.

Basis: Ducks that eat the same super feed lay eggs that are 1.3 times larger than those laid by normal ducks.

Method 3: Parameter Estimation

Refers to the use of statistical relationships between historical data and other variables to estimate the effectiveness of project plans.

The accuracy of parameter estimation depends on the maturity of the parameter model and the reliability of the underlying data. Parametric estimating can be used for the entire project or a portion of a project and can be used in conjunction with other estimating techniques.

Expected results of the project: The eggs laid by chickens fed with super feed will weigh 60g.

Basis: Statistics on the egg laying of 1,000 free-range chickens in summer and winter found that eggs laid by summer chickens are generally 1.2 times larger than those laid by winter chickens. In addition, the nutrition of super feed is 1.4 times that of ordinary feed, so the eggs laid by 5 kilograms of free-range chickens eating super feed will weigh 60 grams.

Method 4: Bottom-up estimation

Bottom-up estimating is a method of estimating the components of a project.

First, make the most specific and detailed estimates of the effects of individual influencing factors; then aggregate or "roll" these detailed effects up to a higher level for subsequent reporting and tracking.

The accuracy of a bottom-up estimate, and the cost of it, often depends on the size and complexity of the project.

Expected results of the project: The eggs laid by chickens fed with super feed weigh 62g.

Basis: Statistics on the egg-laying conditions of 1,000 free-range chickens of different kilograms in winter found that for every kilogram increase in weight of the chicken, the egg size increased by 5 grams. Super feed can make old hens gain 1.5 catties in one winter. If a 5-catties free-range chicken eats super feed until winter, the eggs it produces will weigh 57.5g.

Method 5: Three-point estimation

The accuracy of project estimates can be improved by considering the uncertainty and risk in the estimates and using three types of estimates to define an approximate range of project expectations:

• Most Likely Expectation: A realistic estimate of the project that needs to be undertaken.
• Most optimistic expectations: Project expectations based on the best possible scenario for the project.
• Most pessimistic expectation: The project expectation obtained based on the worst case scenario of the project.

Expected results of the project: The eggs laid by chickens fed with super feed weigh 64g.

Basis: The most likely expectation is method four, 62g; the most optimistic expectation is method one, 70g; the most pessimistic expectation is method three, 60g; the expectation of the three-point estimate is 64g.

The above mainly uses the example of free-range chickens laying eggs to explain the contents of five estimation methods. The expected results obtained by using different estimation methods are different.

PS: Factors that affect egg size are related to the breed of old hens, the weight of old hens, the egg-laying season, feed nutrition, etc.

1. Advantages and disadvantages of five estimation methods

Considering only the two dimensions of estimation efficiency and estimation accuracy, the advantages and disadvantages of the following five estimation methods are as follows:

1. Key to the five estimation methods

• The key to expert judgment: Find a person who plays an expert role in the project or organization. This person needs to be familiar with the project’s past and future development and be able to provide relevant advice and guidance.
• The key to analogical estimation is to find a historical project that is similar to this project and collect the results data of this historical project.
• The key to parameter estimation: think about the calculation formula for the expected effect, then collect the known values ​​in the calculation formula, and finally get the result. If there are unknown values ​​in the calculation formula, other estimation schemes can be used to determine and finally obtain the result.
• The key to bottom-up estimating is to break down the project, analyze the factors that affect the expected results of the project, and summarize them from the bottom up.
• The key to three-point estimation is to find the risk points of the project, which may be positive risks or negative risks, and consider them comprehensively to get the most likely result.

1. Choosing the best estimation method

From the example of hen laying eggs, we can see that the choice of estimation method or methods is closely related to the expected results.

Therefore, at the beginning of the project, it is necessary to select an estimation method based on the current situation and available resources. For example, if the project is in a hurry to go online and the initial evaluation time is very short, then the priority should be an efficient estimation method or the resources available for the initial estimation plan.

If the resources for data analysis are limited, then efficient estimation methods can also be given priority.

IV. Conclusion

In summary, there are three elements to estimating the expected effects of a product solution: defining the effects , defining expectations , and selecting an estimation method .

Defining the effect determines the direction of the estimation; defining the expectation determines the estimation period; and selecting the estimation method makes the entire expected effect estimation more concrete and executable. But in fact, these three elements also determine the focus of the product solution, the attention period after launch, and the dimension of post-launch effect evaluation.

Therefore, product developers should spend more time on estimating the expected effects, rather than just opening Axure to draw prototypes and write logic.

The above are based on my own thoughts and reflections on estimating the expected effects of product solutions (the estimation method refers to the estimated cost in PMP). There may be certain limitations and are for reference only. Everyone is welcome to communicate and learn more!

Source: Dong Xiaobai