Exclusive interview with Tencent's Liu Yafei: Challenges of mobile game operation and maintenance in complex environments

From 2013 to 2014, the global mobile game market output value will soar from 13.2 billion US dollars to 22 billion US dollars. Behind the rapid growth of output value is the continuous increase in the number of players. The continuous increase in the number of players is a severe test for operation and maintenance.

Different from the operation and maintenance of games in the PC era, the operation and maintenance of mobile games faces a more complex network environment, more fragmented game time and a rapid change in the number of players, all of which bring many challenges to the work of game operation and maintenance personnel. An exclusive interview with Liu Yafei, the head of the mobile game operation technology review expert group, explains the operation and maintenance challenges from PC games to mobile games.

Reporter: Could you please introduce the situation of your technical review expert group and the tasks they are responsible for?

Liu Yafei: The review team consists of two teams, internal review and external review, with a total of six people. They are mainly responsible for evaluating the architecture, operating environment, and operating specifications of the game from the signing to the launch. In addition, the review cycle for mobile games is relatively short, and it will be conducted three to four times a month. Each review will be divided into two parts, online and offline. The review will be conducted from the aspects of architecture, basic operating environment design, logic processing, DB, and storage design logic.

Reporter: You have provided operation and maintenance support for many PC games, including some very popular games such as CrossFire, League of Legends, and DNF. What was your team's estimate of these games before they were launched? When the actual situation far exceeded the estimate, how did you quickly expand the capacity?

Liu Yafei : Take CrossFire for example. Although no one anticipated how popular it would be before it went online, we considered the convenience of expansion as a major consideration when reviewing the game. When there was a big difference between the estimate and reality, we were able to expand capacity quickly. When the expansion capacity reached the standard, it didn't make much difference to the operation and maintenance personnel whether a game was popular or not, because our resources were quickly in place to support the game.

Unlike PC games, which have a relatively long inspection and expansion period, mobile games explode in popularity within a week after they are launched. We can only refer to previous similar games to make an evaluation, but precisely because of this speed, we can complete the front-end expansion in ten minutes, and the back-end expansion will be completed within the next few hours.

Reporter: When talking about Tencent games, we have to mention the most popular game on the market - League of Legends. This is a game that almost always has people online. How does Tencent operate in terms of fault repair and processing?

Liu Yafei : For general basic faults, we will quickly repair them through monitoring and self-healing. Self-healing means that when a process crashes or a basic environment has a problem, we will automatically switch, automatically pull up, or perform self-healing in a similar way. In addition, during the review stage, we will recommend developers to write back important data in real time. The read-write method basically determines the rollback time. If this is not considered at the beginning of the game design, it will take ten minutes or more to write back to the DB, which will be very difficult once a problem occurs.

Reporter: What are the differences between the operation and maintenance of mobile games and PC games? Will you feel uncomfortable in the beginning?

Liu Yafei: There is a sharp contrast between PC games and mobile games. PC games are slow, while mobile games are fast. PC games take a long time, while mobile games take a short time. There are few PC games, while there are many mobile games. In fact, the biggest change from PC games to mobile games is getting faster, and this speed requires us to adjust and adapt quickly.

Reporter: China's network environment is very complex, and the time spent playing mobile games is more fragmented. For example, what impact does a weak network environment like the subway have on operation and maintenance?

Liu Yafei: For weak network environments, we deal with two aspects: first, we optimize the entry experience, work with operators to do some network switching in weak network environments, and optimize the logic processing of interactions between different network environments. Then feedback to each game. The second aspect is that during the review stage, we will require developers to have a certain weak network processing capability.

Reporter: We know that Tencent is the agent of many Korean games. The game logic of Korea and even the development logic of some domestic studios are different. What difficulties will this bring to the operation and maintenance work?

Liu Yafei : Because the Internet in South Korea is very good, the home broadband is above 8M, and the mobile network is much better than that in China. These games must first be localized and adapted to the local environment before entering China, because China's network environment and models are much more complicated than those in South Korea. Therefore, we will continue to discuss and connect with developers in the early stage of the release to make some special localization suggestions.

For domestic developers, first of all, the architecture must be able to be adjusted dynamically. In addition, the operation and maintenance personnel should pay more attention to the rapid repair of faults in the later stage and high stability. These have a great impact on the player experience. The operation and maintenance personnel should have in-depth communication with the developers. The specifics should also be based on individual games. After all, the user groups are different and the pressures are also different. Tencent has relatively high requirements for cache, disaster tolerance, and rapid fault recovery capabilities, while other companies pay more attention to other aspects.

summary:

When the reporter asked which pitfall experience left the deepest impression on him, Liu Yafei recalled that in fact, those pitfalls were valuable experience and wealth. People who do operation and maintenance will more or less fall into pitfalls. Some of these pitfalls come from developers and some from surrounding teams. In fact, the biggest pitfall is to have a blank mind.

During the interview, Liu Yafei lamented that, in fact, the current gaming industry is quite tough, but in order for players to have a better gaming experience, having tens of millions of people play the game you made gives you a very strong sense of accomplishment.