From 0 to 1, how to build a user portrait system?

Starting from the data product level, the article analyzes how to build a user portrait system from 0 to 1. It mainly consists of four steps: business needs analysis, label system construction, portrait system construction and portrait application, for everyone's reference and learning.

From PC to mobile Internet, a group of Internet products that took advantage of the traffic dividend have risen rapidly. But now, the traffic dividend has disappeared, and a crazy, legendary traffic era has ended. The lack of incremental growth and more intense competition in the existing market have given rise to refined operations. By combining big data, users are grouped and different marketing strategies are adopted for different groups of users.

The Art of War says: "Know yourself and know your enemy, and you can fight a hundred battles with no danger of defeat." In the entire process of refined operations, the establishment of a user portrait system plays an indispensable role. Earlier, I introduced the construction of the data infrastructure layer, including point data collection, indicator system construction, data warehouse and OLAP analysis. Next, let’s study how to build a user portrait system from 0-1, as well as the application scenarios of user portraits.

Getting to know user portraits

The concept of user portrait was first proposed by Alan Cooper, the father of interactive design. It is a characteristic portrayal of the target group of a product or service.

In the early days, when there were fewer sources of user data and the amount of data was relatively small, the research on user portraits was mainly based on statistical analysis, and user portrait labels were constructed through user surveys.

Later, Syskill and Webert from the University of California manually collected website users' satisfaction with pages, and then gradually built a user interest model through statistical analysis.

With the development of the Internet and information collection technology, California University of Management developed Web Watcher, which can record various browsing behaviors and interest preferences of users on the Internet through data collectors, build user interest models, and expand and update system models as data continues to accumulate, and user portrait labels will become richer.

In recent years, with the explosive growth of massive data on the Internet, many companies have had new opportunities for user portrait research. Based on data labels such as user attributes, behaviors, interests and hobbies, using algorithms to analyze and model features, thereby abstracting the overall picture of the user, has become the focus of product personnel.

For example, for Luffy, his user portrait can be simply described as a middle school boy between 18 and 25 years old, worth 1.5 billion, who loves meat, cool robots, and is an impulsive consumer. If an e-commerce website knows the user's information in advance, it can push meat and high-tech products to him based on his preference characteristics, thereby promoting Luffy to complete the purchase on the platform.

In this process, the key factor used to describe the user portrait is the label. Usually, the label classification is different in different application scenarios.

For example, Tencent Advertising categorizes tags into:

• Demographic labels: gender, age, place of residence, education, marital status, assets and work status, etc.
• Interest tags: business interests, general entertainment interests, semantic interests, etc.
• Device tags: device brand, operator, networking method, model, operating system, etc.
• Behavioral tags: people’s workplace, travel frequency, e-commerce shopping, O2O, gaming and other application-specific behaviors, interactive behaviors, payment behaviors, etc.

According to Alibaba e-commerce’s classification of labels, they can be divided into:

• User attribute tags: gender, age, region, registration date, mobile phone brand, mobile phone system, contact information, historical purchase status, user activity, and RFM value.
• User behavior tags: number of visits in the past 30 days, average order value in the past 30 days, number of active days in the past 30 days, visit duration in the past 30 days, average visit depth, recharge users, etc.
• Consumption tags: income status, purchasing power level, purchased items, purchase channel preference, last purchase time, and purchase frequency.
• Product category tags: high heels, boots, shirts, French dresses, meat jerky, beef jerky, sweeping robots, smart speakers, etc.
• Social tags: frequently active time periods, active locations, single, number of reviews, degree of positive reviews, etc.

The entire portrait system includes label modeling, portrait system, and portrait application. So from the perspective of data products, how to establish a user portrait system from 0 to 1? Next we expand according to the following structure:

1. Business needs analysis
2. Building a labeling system
3. Building a portrait system
4. Portrait Application

Step 1: Business needs analysis

The construction of a user portrait system cannot be created out of thin air. It needs to be centered on economic development and consider the value that the portrait system can bring to the business based on actual business needs. Therefore, the first step we need to take is to analyze business needs.

Clarify the objects that user portraits serve in the enterprise, such as product, user operation, event operation, marketing, risk control and other departments; then, based on the needs of the business side, clarify future product construction goals and the expected results after user portrait analysis.

As for the company as a whole, his goal is to increase the overall revenue of the platform. In the process, he will drive colleagues in product, operation, data analysis, marketing, customer service, etc. to work together. The portrait will pay more attention to how to conduct refined operations and increase the company's revenue;

As for the operator Nami, her goal is to improve the conversion rate. In the process, she will adopt the strategy of personalized content push and accurate user reach, and the portrait will pay more attention to the user's personal behavior preferences;

For data analyst Robin, her goal is to warn users of churn and conduct targeted precision marketing, which requires analyzing user behavior characteristics and consumer preferences.

During the demand analysis phase, we need to analyze the business process, the core concerns of each department, department KPIs, organizational structure, user behavior paths, and functional flow charts. The analysis process here is similar to the tracking business demand analysis. For details, please refer to the tracking business demand analysis. It will not be expanded here.

Step 2: Build a labeling system

From the perspective of the label system for data product construction, labels can be divided into three categories according to the statistical method of labels: statistical labels, rule labels, and prediction labels;

1. Statistical tags

User portrait is the most basic and common tag. Statistical tag values ​​are tags that count user-related values ​​and objectively describe user status. This type of data can usually be obtained from user registration data, user access, and consumption statistics.

For example, for a user, fields such as gender, age, city, zodiac sign, number of active days in the past 7 days, number of active times in the past 7 days, cumulative purchase amount, cumulative number of purchases, average monthly consumption amount, etc.

2. Rule-based tags

It is generated based on user behavior and determined rules. In the actual process of developing portraits, it is jointly formulated by operations personnel and data personnel according to business needs, including activity tags, RFM tags, etc. For example, the definition of "active trading" on the platform is "number of transactions in the past 90 days>3".

Next, I will introduce the commonly used methods of dividing user activity tags and RFM tags.

(1) User activity tags

In actual business scenarios, users may be labeled as highly active, moderately active, low active, or lost based on their activity.

So how are the high, medium and low time ranges divided in this process?

Here, Luffy's impulsive decision will not work. The construction of labels requires that the definition must have a basis and a method .

First, we divide the user's churn cycle. There are usually two methods:

The first is the inflection point theory: an increase in the value on the X-axis will bring about a significant gain (decrease) in the value on the Y-axis, until it exceeds a certain point, when X increases, the data gain (decrease) of Y drops significantly, that is, the marginal benefit in economics decreases significantly, and that point is the "inflection point" in the chart.

For example, when the churn cycle in the figure increases to 5 weeks, the rate of reduction of user return rate drops significantly, so the 5 weeks here is the turning point. We can use 5 weeks as the period to define user churn. That is, if a user who has visited/logged in before does not visit/log in for 5 consecutive weeks, then the user is defined as churn.

The second is to count the time interval between the user's last visit and the penultimate visit. It can be assumed that users whose time interval is greater than this will basically not visit again, that is, the user has been lost. Looking at historical data, we can see that less than 10% of users have a gap of more than 30 days between their last visit and their second-to-last visit. Users with a larger gap than this can be considered "lost users."

After dividing the churn cycle, users are further divided into high, medium and low activity according to their activity level. Historical data is divided according to the 80/20 principle.

For example, if analysis shows that users who are active more than 10 times account for 20% of the total number of users visiting in the past 30 days, then this group is "highly active users"; users who are active 5-10 times are further classified as "medium active users"; and users who are active 1-5 times are classified as "lowly active users".

(2) RFM tag

The RFM model is mainly composed of three basic indicators: the time of the most recent consumption, the frequency of consumption, and the amount of consumption. Based on historical data, we can check the percentage of users and divide them according to the 80/20 principle to obtain segmentation labels.

• R: If 80% of the users in the historical data have visited the site less than 60 days ago, the data is considered “recent”; if the users have visited the site more than or equal to 60 days ago, the data is considered “distant”.
• F: If the order volume of a user that accounts for 80% of the historical transaction order volume is less than 5 orders, it is considered "low frequency", and if the order volume is greater than or equal to 5 orders, it is considered "high frequency".
• M: For example, if the transaction amount of 80% of the historical transaction order amount is less than 500 yuan, it is considered "low amount", and if the transaction amount is greater than or equal to 500 yuan, it is considered "high amount".

3. Predicting class labels

Based on the user's attributes, behaviors, locations and features, we use decision tree algorithms, regression algorithms, etc. to mine the user's relevant features and potential needs. Based on these potential needs, we label the users and push them with different marketing strategies.

For example, we can judge a user’s preference for a product based on his consumption habits; and we can predict his risk level based on his behavior such as returning negative reviews.

Generally, statistical and rule tags can meet application requirements and account for a large proportion in the development process. Machine learning mining labels are mostly used in prediction scenarios, such as judging user risks, user purchase preferences, user churn intentions, etc., and their development cycle is long and the development cost is high.

For example, Toutiao has accumulated a large amount of text data such as articles and posts related to the topics of data products. Due to historical reasons, these articles have not been classified and labeled accordingly, making it difficult to manage the content. Now you need to tag your posts with the appropriate hashtags.

First, according to the already defined article types, the classified articles will be automatically divided into the corresponding types.

Second, it supports the centralized management of articles, automatically rewarding each article with tags related to its topic based on the content of the article.

(1) Feature selection and development process

Data classification: manually annotate a batch of documents accurately as training set samples, and an unannotated batch of documents as test set

Data preprocessing: perform word segmentation on the test set and training set text, build a vocabulary library, remove stop words, modal particles, etc.

Naive Bayes classification: classify articles based on precision, recall, and F-measure

(2) Calculate label weight

Different behaviors of users on the platform have different weights at the user tag level. For example, the weight of a user's purchase of a product is higher than the weight of a user's adding a product to a shopping cart, collecting a product, or browsing a product.

During the label formulation process, user portrait modelers and business personnel need to communicate closely and formulate different behavior types and weights based on business scenarios. Commonly used methods for determining weights include TF-IDF word space vector and time decay coefficient.

1) TF-IDF word space vector

TF-IDF is a statistical method used to assess the importance of a word or term relative to other words in a document set or a corpus. The importance of a word is directly proportional to the number of times it appears in the document set and inversely proportional to the number of times it appears in the corpus.

2) Time decay coefficient

When user data reaches a sufficiently dense level, the attributes corresponding to the labels on users will show a high degree of stability, which matches the personal characteristics formed by users' long-term behavior.

User label weight = behavior type weight * time decay * number of user behaviors * TF-IDF calculated label weight

Step 3: Build a user portrait system

As a supporting system, the portrait system's main target users are marketing, operations, product, data analysts and other personnel, meeting their needs for user analysis, tag query, and marketing activity docking. Therefore, the design of the portrait system needs to consider the functional user analysis requirements and the non-functional interface development requirements.

1. Functional requirements

Functionally, it can be divided into: home page portrait data, tag management, user query, user grouping, etc.

• The home page portrait data displays the overall situation of user data, including basic user characteristics, such as gender, age, region, occupation distribution and other basic information; user value characteristics, such as user activity, membership level, churn warning, user value RFM distribution and other information.
• Label management allows data personnel to add, delete, modify, and query labels, including label classification, new label creation, label review, label offline, abnormal labels, etc.
• User query, the main capabilities include viewing user portraits and other detailed data by entering user ID, such as user basic information, user attribute information, user behavior and other data.
• User grouping, when the application scenario is mainly for business use of tags, often not only one tag is used for push. In most cases, multiple tags need to be combined to meet the business definition of the crowd. User grouping is equivalent to making a crowd template and pushing crowds in different scenarios.

When adding a group, you usually configure the group name, the conditions that must be met, calculate the number of people covered, and push it to message notifications, emails, and text messages.

2. Non-functional requirements

Non-functional requirements mainly include interface requirements, which ensure that the portrait system data is connected with various systems, such as push systems, marketing systems, advertising systems, recommendation systems, BI platforms, etc., and ensure the real-time update of data in each system to avoid the problem of different numbers from the same source.

The label system and user portrait system have been built, so where and how should user portraits be used, and what value can they bring to the business?

Step 4: Portrait application

In the fields of advertising, e-commerce, etc., user portraits are often used as the basis for precision marketing and recommendation systems. The main application scenarios include three categories: precision marketing, user analysis, and personalized recommendations.

1. Precision Marketing

Based on historical user characteristics, operators can analyze the product's potential users and related needs, and provide personalized marketing services for specific groups. Commonly used services include precise push notifications via SMS, email, in-site messages, and push messages, customer service’s different scripts for users, and VIP services such as rapid refunds and returns for high-value users.

SMS/email/push marketing

In daily life, we receive marketing information from multiple channels. A text message push about the arrival of a red envelope may prompt users to open an app they have not visited for a long time. A message about a price reduction on their wish list may stimulate users to open the push link and make a purchase directly.

Things to note when using the portrait system for marketing:

SMS sensitivity: Some users are less sensitive to marketing SMS. For example, according to historical data, out of 10 SMS messages sent to them, they may only open it once or never. Considering that the SMS channel requires marketing costs, this group of users can be excluded and the interference to users can be reduced.

Invalid mobile phone number: For users who randomly fill in a non-own mobile phone number on the platform, whose mobile phone number has been invalidated/changed, and who reply "TD" after receiving a text message, the text message cannot be received and is on the text message blacklist. Such users also need to be excluded

Users interested in marketing products: Users who have browsed, collected, added to cart, or placed orders multiple times recently are potential users of certain products and can be marketed to through discount coupons or red envelopes.

Customer Service Talk

When we complain, consult or give feedback to the customer service department of a platform, the customer service staff can accurately tell us our purchase situation on the platform, the results of the last consultation issue and other information, propose targeted solutions, and provide special services such as VIP customer service channels for high-value users.

2. Recommendation System

The operator of the application can recommend different content to users based on labels such as gender, age group, interests and hobbies, browsing and purchasing behavior in the user portrait. Such as the personalized article content recommendations on Toutiao, the personalized video content recommendations based on user portraits on Douyin, and the personalized product recommendations based on user browsing behavior and other portrait data on Taobao.

3. Data Analysis

User portrait labels can be applied to various types of analysis, including user analysis, order analysis, funnel analysis, population characteristic analysis, etc.

Summarize

This article mainly discusses how to build a user portrait system from 0 to 1 from the perspective of data products. Friends who have read the articles on tracking points and indicator systems written by Straw Hat Boy may have discovered that the construction of a portrait system is the same as the construction of tracking points and indicators. It also follows the ordinary product design process, from demand analysis to label/indicator design, background design, and finally application to business.

As the saying goes, no matter how things change, they still remain essentially the same. Just like Luffy's skills seem to be ever-changing, the core point is that he uses the ability of rubber to make various changes. The core of data products in various forms lies in business.

Author: Straw Hat Boy

Source: Straw Hat Boy (luckily304)