Contribution | The technology carrier behind Alibaba's mobile e-commerce

The 2015 Tmall Double 11 Global Carnival has just ended, and the full-day transaction volume of 91.2 billion has once again broken the record, of which wireless accounted for 68.67%, and the wireless transaction volume reached 62.6 billion.

On November 16th, US time, InfoQ English site invited Zhuang Zhuoran (nickname: Nantian), senior technical director of Alibaba Wireless Business Unit, and Xu Zhao (nickname: Changgong), senior technical expert of Alibaba Wireless, to give a speech entitled "ALIBABA MOBILE INFRASTRUCTURE AT "CHINA SCALE"" at the QCon Global Software Development Conference in San Francisco, revealing the evolution of the technical architecture of the super app Taobao Mobile and sharing the technological development of China's Internet in the wireless field with technical peers around the world.

[[156640]]

Zhuang Zhuoran has more than 10 years of experience in Internet R&D and management. He is one of the parties who witnessed and led the evolution of Alibaba's business architecture. He joined Alibaba in 2009 and was responsible for the architecture and R&D of marketing, membership, transaction, shopping guide and other systems in Taobao and Tmall. He experienced the technical evolution of Taobao from a simple application to a distributed service-oriented business platform that carries hundreds of millions of users. At the same time, he was also the technical leader of Tmall during the Double Eleven event from 2011 to 2013, and continued to promote the R&D work of the Double Eleven product architecture evolution, stability transformation and performance optimization. In 2013, he switched to the mobile R&D battlefield and led the team to focus on the R&D and construction of mobile Taobao and mobile basic platforms.

Xu Zhao joined Tmall in 2012 as a technical core. He has led multiple technical teams including Tmall sellers and shopping guides, and personally created and witnessed Double 11 becoming an important festival in the e-commerce industry. He is currently the technical leader of the Taobao mobile seller ecosystem team, focusing on the overall wireless architecture, large-scale complex mobile application construction framework, and wireless technology openness. He also pays attention to the evolution and implementation of new technologies and development models in mobile Internet products.

The editor brought you the transcript of Zhuang Zhuoran’s speech on the spot:

Alibaba in my eyes

In the past decade, Alibaba has gradually influenced and changed the lifestyle of Chinese people. Before this, everyone could only shop in offline stores, but now everyone has developed the habit of online shopping and enjoys it. On the one hand, so far, more than 10 million sellers have chosen to open online stores on Alibaba; on the other hand, Alibaba has built China's largest public cloud computing platform and ranked among the top three in the world. In China, Alibaba Cloud currently serves more than 30% of websites and more than 50% of small and medium-sized enterprises.

Alibaba is an international company, as can be seen from this year's Double 11. In addition to bringing all kinds of overseas top products to China, we also bring domestic high-quality products to the world. AliExpress, founded in 2010, has developed into the world's largest cross-regional trading platform. In addition to e-commerce, Alibaba's global business also includes finance, logistics, health, entertainment and cloud computing. This is why I think it is inaccurate to regard Alibaba as China's Amazon and eBay. As Jack Ma said, "If Amazon or JD.com is compared to an apple, then Alibaba should be an apple tree." We see ourselves as an ecosystem, the water, electricity and coal of future life. Our goal is to create an open, transparent and cooperative business platform to make it easy to do business in the world.

In the past two years, the proportion of mobile transactions on Alibaba's e-commerce platform has doubled to more than 60%, thanks to the major changes we have made in mobile and system architecture. Therefore, today we will mainly introduce the evolution of our e-commerce operating system in the mobile environment and how this architecture copes with the challenges of the PC era. In general, it will be divided into four parts: client containerization architecture, mobile network pipeline architecture, cloud service architecture and data architecture. Due to time constraints, we will focus on the first three.

Seven years ago, Singles’ Day was jokingly called Singles’ Day in China, but in 2009, Alibaba decided to hold shopping promotions on this day for the first time, turning this day into a shopping carnival that attracted worldwide attention.

Five days ago, my team and I just experienced the 7th Double 11, and Alibaba's transaction volume on that day was about 14.3 billion US dollars (91.217 billion RMB). The total transaction volume in the United States during Black Friday last year was 11.4 billion US dollars (including the total sales of all offline and online e-commerce companies), and the 14.3 billion US dollars generated by Double 11 was just the transaction volume of Alibaba's e-commerce platform in one day. The entire carnival covered more than 200 countries and regions. The whole society participated together, turning an ordinary day into a global carnival festival. This is the power of Alibaba's ecosystem and the power of the Internet.

Taobao Mobile

Taobao Mobile, as the "aircraft carrier" of Alibaba Group's mobile terminal, has developed very rapidly in the past two years, with the number of users increasing nearly 10 times. On the 2015 Double Eleven Day, our wireless terminal accounted for 68% of transactions. During the peak period, more than 45 million people were online at the same time, and 140,000 orders were placed per second. Today, we have more than 300 million monthly active users and are the world's largest consumer mobile app. All of these have posed huge challenges to our system capabilities.

Below I will share with you a typical day in the life of a user on Taobao Mobile.

Rethinking Mobile Architecture

What Alibaba has built is not a software system platform, but an e-commerce operating system. This e-commerce operating system has supported the rapid development in the past 10 years. Similarly, we are also rethinking how to continue to upgrade on the mobile side and how to complete the ecological construction in the mobile app.

Mobile Internet is not just an extension of traditional Internet. This consensus comes from the "destructive" impact of the powerful characteristics of mobile terminals on products and business models, and from the changing trend of technology.

In the era of the web, we say BS architecture, which can usually be understood as a variant of CS architecture. However, on the client side, we run in a container environment such as the Browser, and communicate with the server through a stable broadband network access and a simple and transparent protocol such as HTML. Since the browser only provides a very simple capability interface, there is only a very thin layer on the client side, which only does rendering work. The main job of engineers here is to focus on solving compatibility issues. All logical calculations and all the core issues we need to solve, such as scalability, scalability, stability, performance, etc., are actually concentrated on the backend.
With the development of personal computer computing power, front-end technologies such as Ajax and Angular have sprung up like mushrooms after rain, and we have entered the era of web2.0. However, the boundary between cloud and end has not changed fundamentally at this time.

In the mobile era, the computing power of mobile devices is developing more and more rapidly. More importantly, mobile operating systems expose the capabilities of hundreds of sensors on mobile devices to upper-level applications, allowing us to obtain stronger capabilities than in the browser. In this way, the app side can do more things that were previously impossible in the browser. Therefore, as the Internet has developed to this day and entered the era of mobile Internet, we have truly entered an era of rich clients, and the previously simple and clear boundaries between the cloud and the internet have been completely blurred.

At this stage, many of the architectural issues we discussed before, such as scalability, isolation, stability, and performance, are no longer just server-side issues. Clients also need to start thinking about these issues from an architectural perspective. At the same time, the simple protocols and stable channels for communication between clouds are also undergoing a lot of changes. The characteristics of mobile networks that require access anytime and anywhere also put higher demands on our infrastructure. Next, let's take a look at the differences here.

#p#

Rethinking Mobile Architecture-The Embarrassment of Sandwich Bread

But the solution to this problem is not like the server side. In this process, we reflected comprehensively. You can see in this picture that we used a vivid metaphor to reflect the traditional server software. The bottom layer is the operating system, usually Linux, and the top layer is our business functions and services. In the server architecture, we are accustomed to solving problems by adding an architectural layer. For example, the cache layer, data access layer, etc. In the architecture, we build different levels of connection links according to our own wishes, so that the architecture has sufficient flexibility and scalability. Even if it is stacked like this, it is still very reasonable.

Mobile app architecture is a completely different scenario. We have to go back to the ancient CS architecture era, an era that lost the flexibility of the web because all functions must be integrated into one APP.

I call it "the sandwich that looks simple but is actually very awkward."

Therefore, we need an elegant client architecture design to enable us to regain the capabilities of the previous web Internet era. In this context, we built a client architecture such as Taobao Mobile that can support the e-commerce ecosystem. In summary, the design concept can be summarized as follows:

***, isolation during design. This architecture must have isolation during design so that we can maximize concurrency during development and packaging;

Second, isolation at runtime. This isolation is applied to the client and is intended to improve the reliability of the client. We must ensure that the impact of a single module failure is contained within the module itself and does not cause an overall crash.

Third, dynamic rendering and deployment capabilities. This architecture must have a certain degree of runtime adjustment and dynamic deployment capabilities. In this process, we built a faster HTML5-like rendering engine; at the same time, we also implemented a private container that supports dynamic loading and updating of modules. With these two powerful tools, the repair of online problems and the automatic update of new modules are no longer dependent on version releases. Unfortunately, due to the relatively closed and strict review system of AppStore and GooglePlay, especially the former, our dynamic deployment capabilities are often unable to be used, although they have been fully used in the Android application market in China.

Fourth, the strongest compatibility guarantee to ensure that we do not experience overall pain during the system version upgrade process.

During the overall architectural reflection process, we redefined the boundaries between the client and the server (cloud). Once such a reasonable architecture is established, it can ensure that our mobile app can continue to efficiently support business development for a considerable period of time.

Challenge in mobile architecture

Another challenge comes from the unstable mobile network environment. In the PC era, the access conditions for visiting websites were relatively constant, so the impact of the network on user experience was rarely considered during development.

However, this is not the case with mobile apps, especially in China, where the basic mobile network environment is not good. Moreover, many of our users access the site in environments such as subways and buses, and the frequent switching of mobile base stations further increases network instability. Data from Taobao Mobile shows that 15% of our daily active users come from weak network environments such as 2G. If the connection from the end to the cloud is unstable and has high latency, then there is no way to talk about any user experience.

The efficiency of the basic network is like a train, latency is the speed of the train (start-up time), and bandwidth is like the load capacity of the train carriages. The entire transmission physical link is like the rails of the train. The mobile network conditions we face are very complex. There are advanced transmission channels such as high-speed rail, and there are also many old and slow green-skinned trains that still serve many users. Our goal is simple, that is, we want all users to have a smooth experience on Taobao Mobile, regardless of whether you are sitting on a "high-speed rail" or a "green-skinned train."

The following picture can give you a more intuitive understanding of China's mobile network environment. It describes the end-to-end routing from users to IDCs. Not only does data transmission take a long time, but the packet loss rate is also high, and the security is also quite poor. DNS hijacking and content hijacking are commonplace in China.

Therefore, we have done a lot to improve network channels, explore ways to break through the limitations of operators' basic networks, and strive to create the best shopping experience for users:

• Reduce network interactions: Use more efficient communication transmission technology to improve connection multiplexing and reduce roundtrip times;

• Reduced traffic: Network traffic is reduced through more efficient compression and synchronization technologies;

• More secure: Improve data security by optimizing TLS1.3 without reducing user experience;

• More stable: A more flexible scheduling system enables multi-site active-active and shortens the time for failover.

#p#

Alibaba's wireless technology practice

The above is an overview of Alibaba's rethinking of architecture in the mobile era. Based on these reflections, we have put forward the following core requirements for the direction of architecture evolution:

Isolation: Traditional mobile app development and integration methods are not suitable for enterprise-level development. A modular architecture with good isolation is the basis for large-scale parallel development.

Dynamicity: The Internet pursues speed, and from an architectural point of view, we hope to retain the lightness and flexibility of the web in the era of rich clients. Dynamicity is an important prerequisite.

***Network experience: The full-duplex access platform ensures the scalability and stability of the access layer, so that users can have a smooth experience regardless of the network environment.

Compared to the PC era, mobile development is very complex. Users want an amazing experience, but also want the installation package to be as small as possible. So developers need to have lightweight development and deployment capabilities like in the web era, while also being able to obtain native capabilities, performance, and scalability.

Parallel development and independent deployment architecture for platform-level products

A good architecture needs to support parallel development and independent deployment, so as to meet the flexible and ever-changing business growth. At the same time, we should give full play to the advantages of the aircraft carrier-class platform, support more diversified client applications, generate economies of scale, and expand the business boundaries. Therefore, our solution is to build an isolated architecture on the native platform (Android to be precise), on which the functions are cut as small and simple as possible; at the same time, without losing native performance and capabilities, we have also introduced more web features into mobile development.

The principle of our client container architecture design is "everything is a component". On the Android platform, based on our isolated container architecture design, we can not only improve the efficiency of multi-team collaboration during the integration period, but also gain flexible deployment at runtime. By splitting the huge project into a series of independent services and our container self-upgrade capabilities, there is no need to compile and integrate all functional modules at the same time. Once a module has completed testing, we can upgrade it as needed. In addition, our APP also has the ability to hot-fix online.

Cross-platform thinking leads to agility (WEEX)

Another cruel fact is that all our functions need to be implemented repeatedly on Android, iOS and H5. When we face the limitations of client package size and number of methods, we need to support 200+ function upgrades, multi-platform continuous delivery, coverage, fault tolerance, hot fixes and other complicated things. Therefore, to scale within the limitations of devices and OS, we need an ultra-lightweight and flexible solution that can perform low-cost parallel development and rapid release based on something highly reusable.

Weex is the killer feature we want. Developers can build iOS and Android apps with a set of cross-platform codes. When building the application UI, developers can use our basic library, which abstracts the differences between different platforms. The DSL that developers face is based on a lightweight js library called Vue.js. Compared with React's JSX syntax, this is a more component-oriented, script-style separated language, which is more friendly to developers and more conducive to collaboration between developers and designers. JSBundle can be updated in a timely manner with WYSIWYG. The V-DOM abstraction layer provides cross-platform rendering capabilities, and also has the ability to gracefully degrade native components.

How does this solution compare to similar solutions in the industry? First, we chose a standard scenario: rendering a customized image list and displaying 200+ pages.

The following figure shows the actual data we measured, including rendering time, memory consumption, CPU consumption and frame rate. We don’t want to emphasize that we are better than all other solutions through this figure, because all the effects will be different in different scenes. What we want to say is that we can achieve both performance and dynamics through such a solution.

#p#

Flexible, efficient, safe and stable network dispatching service (ACCS)

Next, we need to solve the problem of end-to-end scalability. As the mobile Taobao business expands, our active users are also growing. In order to provide better end-to-end capabilities, we must improve the quality of connection services. This is achieved through efforts in the following four aspects:

***, As service consumers, we cannot determine the latency and bandwidth of the network, but we can work hard to reduce round-trips.

Second, we established an httpdns service and cached it locally to reduce the time loss of client DSN retrieval; and we modified it based on TLS1.3, which greatly improved the performance while ensuring the security of the link.

Third, Alibaba's backend system adopts a multi-data center multi-active architecture deployment method, so the wireless gateway needs to have correct, flexible and variable user scheduling capabilities.

Fourth, we have encapsulated the entire network capability into a standard SDK. On the one hand, this ensures that our APP group can enjoy the benefits of the entire optimization together, and on the other hand, it ensures that the H5 page can use the capabilities of the Native channel.

After explaining the client architecture and network architecture, we once again focused on the backend services. Around 2010, we began to consider multiple data centers. On the one hand, we achieved horizontal expansion of services and data. As the scale of our business continued to grow, the scale of our applications became larger and larger, and the risks brought by single-point instability were multiplied. On the other hand, more and more applications were deployed, covering e-commerce, logistics, big data, etc., and the space that a city can provide us is limited after all. What's more fatal is that if a single computer room fails, it is basically unacceptable for e-commerce, so we started to enter the dual-active or even multi-active transformation in 2011.

Alibaba's open source plan for wireless technology

Architecture is very worth sharing and discussing. Good technical architecture can continue to support great business dreams. Therefore, many of Alibaba's wireless capabilities will be shared with the outside world through open source or cloud services, hoping to give back to the entire community. At the same time, we also hope that our technical thinking can inspire everyone. If you are interested, please continue to follow our github address.

The mobile era is just a transition to the DT era

Technological development will move from the IT era oriented towards productivity to the DT era oriented towards intelligence, and data will eventually become an infrastructure like water, electricity and coal. In such a context, I believe that the mobile era is just a transitional stage, and all our work in this stage is actually focused on three things:

First of all, all kinds of devices will eventually have the ability to connect to the Internet, and our infrastructure must be prepared. This is why we will build a billion-level wireless cloud platform like ACCS to create a powerful wireless infrastructure to welcome the arrival of the era of the Internet of Everything. Secondly, the boundary between cloud and end is bound to go through a process of becoming increasingly blurred and then becoming clear again. In the past, W3C and browser manufacturers have made a lot of efforts. The traditional Internet is an open, shared and transparent ecosystem. The existing mobile terminal platform has not actually made any preparations for super apps. On the contrary, it often restricts the development of super apps. We are also willing to participate in this process to make the mobile Internet more open, shared and transparent.

***, terminals should not only be passive data consumers, but more importantly, they should become data producers in multiple dimensions.

Connect everything, integrate the cloud, share everything, let us work together to welcome a smarter future.