How to operate and maintain her community with millions of monthly active users in 8 months

This article is the on-site dry goods of WOT2016 Internet Operation and Developer Conference. The new session with the theme of WOT2016 Enterprise Security Technology Summit will be held at JW Marriott Hotel in Beijing Pearl River Delta from June 24th to 25th, 2016!

Guest Profile

Wen Min joined Langtaojin in 2014 as the head of operation and maintenance. In the process of rapid growth of mobile users and data, he relied on his strong learning ability to effectively ensure the stability of the system and accumulated rich experience in mobile application operation and maintenance. Previously, he worked in system operation and maintenance for many companies for a long time and has rich experience in operation and maintenance business and management.

In a speech at the Operation and Maintenance Security Session of the WOT2016 Internet Operation and Maintenance and Developer Conference, Wen Min shared the challenges posed to operation and maintenance during the rapid growth of "Her Community" from 0 to tens of millions of monthly active users, and the responses made in various aspects such as operation and maintenance system architecture and network.

Background

The Android version of her community was launched in April last year, and the iOS version was launched two months later. After about eight months, that is, at the end of 2015, the monthly active users reached 10 million, the number of user posts was about 2 million, and the daily increase of pictures and audio reached 200GB. At present, her community has more than 6,000 circles and more than 1,300 circle owners. These circle owners are also composed of users and are responsible for some circle management work.

Initial business structure

In the early days, in order to get the business online as soon as possible, there was no large and comprehensive overall architecture. We made some basic components, such as using Nginx as a proxy, using Redis and Memcache for the backend application services, and using Mysql and Mongodb for the underlying data storage. We also used some third-party components for image storage and log collection.

Problems and solutions

As the number of users increased, some problems gradually surfaced, including network problems, component problems, storage problems, and subsequent processing problems.

***, network problem.

Hijacking problems often occur when the number of users increases. DNS hijacking, we use a third-party httpdns to solve this problem. There is also content hijacking, and we have taken many detours to solve this problem. For example, changing the domain name, changing the IP, and doing some code obfuscation. There will be some effect at the beginning, but it will return to the hijacking state later. *** can't go online https, and the domain name and IP information are all changed. So far, no large-scale hijacking has been found.

There is also the problem of slow network performance. Because there was little consideration in this regard at the beginning of the launch, the line quality was mostly not optimized. The solution is to find some high-quality lines as reverse proxies, based on which we can analyze the user's network performance. The client automatically reports to the server, compresses the data packet on the server, formats the log, stores the log in the backend, and finally generates a visual chart through calculation.

Second, the database.

The first problem we faced was the large amount of data. We performed a table splitting operation, which was transparent to the application. However, after the table was split, there would be problems with the ID, so we used a third-party ID generator. There was also a performance issue, which could be solved by upgrading hardware, such as hard disks and CPUs. SQL also needed to be specially optimized, supplemented by data change service notifications to reduce the read and write pressure on the database.

Third, storage.

Currently, we use the fastdfs file system, which has a capacity of about 34T. These are all pictures uploaded by users. We will crop or compress the pictures to optimize the speed of opening pictures. There are also some audio processing, mainly some format conversion.

We have also researched third-party cloud storage, which are relatively stable and mature, with rich APIs and a wide range of data processing types. This work has been completed. Audio, images, and videos will also use third-party cloud storage and their native processing functions.

After talking about the above points, the whole architecture is in a relatively stable stage. Of course, there are still some details and small problems, such as frequent data reading and writing. In addition to these basic functions, there are also automation and operation and maintenance related functions. Including DNS management, CDN management, Git management and uploading files or regular tasks. In general, these are the main tasks in the eight months.

Lecture video: http://edu..com/lesson/id-100755.html