The road to decoupling mobile Tmall

This article is a compilation of the author's speech at the GMTC Global Mobile Technology Conference. The speech PPT can be found at: http://ppt.geekbang.org/slide/show/194

The title of this article is decoupling. There are many ways to talk about decoupling. This article uses the evolution of architecture as a clue to share with you the decoupling path of Tmall Mobile. I think that in the growth process of Tmall Mobile, some metaphysical thinking and accumulation are certainly of reference value to everyone, but tools and solutions are more valuable for reference. Therefore, this article will focus less on the process and reasons, and more on the tools and solutions.

What drives evolution?

As a technical team, we have various reasons to upgrade our technical architecture. What factors are the most critical and what can be the reason for evolution?

• Business upgrade

The most important factor must be business upgrade. A technology that cannot generate business value is not worth paying attention to, let alone implementing.

Good technology always takes a step ahead of business, waiting for business to catch up not far ahead - technology drives business. Therefore, the continuous upgrading of business requires our technical architecture to run faster.

• Team size & cooperation model

Another important factor driving the evolution of architecture is team size. One of the most important functions of technical architecture is to ensure production efficiency and quality. The human factor is very critical. With more people and more complex cooperation, the technical architecture must be upgraded to ensure that when so many people work together, efficiency is high and there are no problems.

• Code size & Project size

The scale of code and engineering is a natural result of development. As the business becomes more complex, the team becomes larger, and the number of people increases, the scale of code and engineering will inevitably increase. Although the difference in scale between an app with hundreds of thousands of DAU and an app with tens of millions of DAU is not inevitable, it is also highly likely.

The technical architecture of a platform and product is bound to be very different when one person uses dozens of files to complete a product with simple functions, and when more than a dozen people use thousands of files to complete a product with complex functions, and then further to more than a dozen teams and hundreds of modules.

• New Technology

New technologies are the engineers' own business. There will always be new technologies in the industry, and these technologies may be developed by other engineers to adapt to their business development and architecture evolution. However, technology has no boundaries. If new technologies are good and can bring us value and improve our efficiency, then we will use them.

For example, when Facebook's RN came out, we all thought it was good, and many companies should have used it on a large scale. Large-scale use of RN for development will also require an upgrade of your original architecture.

Of course, what is more important is how to balance the curiosity of quickly upgrading the technical architecture with just meeting the business and team requirements. Excessive pursuit of technical architecture innovation and new technologies will not only fail to promote the business and team, but will cause disasters. Therefore, as an excellent technical team, you must always weigh whether to do or not, and whether to do more or less.

How the architecture evolves

In what aspects does architecture evolution manifest itself? As a technical team, how do we implement architecture evolution? This question varies from project to project, from team to team, and from direction to direction. This article only introduces the evolutionary process of Tmall Mobile related to decoupling during its development.

• Upgrading development mode

The concept of development mode is a bit broad, so this article will only discuss the aspects related to decoupling: teamwork mode and engineering organization form. This will be discussed in a separate section below, so I will not go into details here.

• Decoupling of all dimensions

When the project becomes larger, it needs to be split up, whether it is componentization, plug-inization, or something else. Decoupling is the first step, and it should be decoupling in all dimensions.

• Complete toolset

In the process of pattern evolution and decoupling, many tools will be derived. In the process of evolution, we will also think about which tasks need to be tooled and take the initiative to develop tools. A complete toolset will greatly improve the productivity of the team and can be said to be the most valuable part.

Development mode upgrade

The Tmall mobile team has grown from a small team of less than ten people to a large team of nearly 200 people. The following article describes in detail what changes the development model has undergone.

• A project

Three years ago, the mobile Tmall team was just formed, with about ten engineers developing the first version of the Tmall App with only basic functions. The entire team consisted of just a few people, including iOS, Android and Server, with only three or four people on each platform; the functions of the App were also very simple, able to complete basic shopping guides and transaction processes.

Tmall App uses the simplest architecture, independent project, MVC architecture. And we think that such an architecture is completely sufficient in this case, and it is true.

• Modularity

As the wireless business developed, the Tmall Mobile team began to expand explosively. Soon one team became two, and two became four. As the number of teams increased, division of labor emerged, and the project became larger and larger, we began to find that the original architecture was no longer sufficient, and it was imperative to split the modules.

At this stage, the module splitting of Tmall Mobile was also done very simply. The project was first divided into horizontal layers according to function, and then vertically sorted out at the business layer. The codes of different modules were simply placed in one folder, and the organizational form of the project did not change.

By splitting like this, we can achieve code independence, and there will be basically no conflicts on the same module code across teams.

• Plugin

As the business grew, the team became more and more complex, the team work became more and more segmented, the number of people increased, and the amount of code became larger and larger. The simple way of using folders to organize modules was not enough. Multiple businesses and teams, different development rhythms, and complex dependencies caused us to spend a lot of time solving compilation problems. Waiting for other modules to be integrated became the biggest bottleneck of our development efficiency.

To solve this problem, our solution is plug-in. So what is the difference between plug-ins and modules? I think the biggest difference between the two is independence. Plug-ins can be developed, released, and run independently, while modules must rely on the environment of the main project. Independent plug-ins can well isolate dependencies between teams, develop independently, and release versions at their own pace.

Based on this thinking, we introduced dependency management facilities (iOS introduced Cocoa Pods, Android used Maven); the previous modules were further separated into independent projects and versioned separately; each independent plug-in is responsible for the released version number, and both other plug-ins and the main project rely on the stable version released by the plug-in.

However, plug-inization is not as good as we imagined. The code is out, but it cannot be compiled independently. The dependency management facilities are in place, but they are not well managed. Since we have never sorted out the dependencies before, whether it is a module or a plug-in, it is just a way of working for code management and release processes, and it cannot solve the problem of independent development and independent operation. At this stage, we chose to tolerate this problem, because independent development and independent operation do not seem to be so valuable to us, and the inability to achieve these two things does not become our bottleneck. So everyone is still in the same project, but the code is submitted to different repositories, and then assembled into the main project through the dependency management facility and version number, and the source code is finally run together.

• Independent Publishing

What problems will the inability to publish independently bring? Very obviously, slow! After a period of plug-inization, we found that the slowness seriously affected our efficiency. At this stage, we already had more than ten teams and more than 10,000 source code files for iOS projects. Since the main project is combined with the source code of each plug-in, each re-indexing and compilation takes more than half an hour.

To solve this problem, we need to carry out plug-inization to the end and realize the other two independences of plug-ins: independent development and independent operation. The most important work is the decoupling of our theme today, sorting out the dependencies between various plug-ins. Let each independent plug-in rely on other plug-ins as little as possible, and compile and execute normally within the minimum scope. Each release is no longer a stable version number, but a stable binary package.

With such dependencies, we split the compilation process of more than half an hour into dozens of modules, and the main project depends on dozens of binary packages, so the compilation is fast.

The entire model upgrade basically went through the following stages:

• Code independence, decoupling from the form first
• Independent code engineering lays the foundation for independent operation
• Sort out dependencies, independent projects can be compiled
• Abandon source code dependency and speed up integrated compilation

Along the way, we took one step at a time and finally achieved complete decoupling. In this process, we have accumulated a lot of methodologies and best practices. I think there are two tools worth introducing, which are detailed below.

Decoupling Toolbox

If you want to do your work well, you must first sharpen your tools. Everyone says this, but not everyone can do it. A team with a tool culture will have great advantages in terms of quality and efficiency.

For a project that has rapidly expanded from its original state to Tmall's current size, the complexity of its dependencies is beyond imagination.

In this expansion process, I divide coupling into three categories:

1. Interface coupling means that in the user operation process, from home page to search to details to store, the jumps of these interfaces are hard-coded
2. Dependency coupling, as the name suggests, the dependency between two modules is coupling
3. Project coupling: each module has its own life cycle and runtime, and each module needs to rely on the runtime of the main project in the production environment

Beehive (Beehive is open source, you can see the source code on Github: https://github.com/alibaba/BeeHive)

Beehive is a runtime framework that mainly solves dependency coupling and project coupling.

Speaking of coupling, for an App as large as Tmall Mobile, the dependencies are bound to be very complex, and the coupling between modules is bound to be intricate. What we need to do is not to deal with these dependencies and couplings one by one, but to sort them out, find out the unreasonable ones, and solve them, so that the entire project is within a healthy and reasonable range of dependencies and coupling. There are basically the following types of problematic dependencies:

1. Circular module dependencies
2. Reverse dependencies between layers
3. No strong functional dependency

The following figure is a schematic diagram of dependencies.

The dependencies with dashed lines are the dependencies that I think are problematic, and they abstract out the problematic modules.

After introducing Beehive, the dependency relationship will lead all the red lines to the Beehive module, while the Beehive module is independent of each layer.

The principle of Beehive is that each module that provides external services needs to register an abstract interface to the Interfaces (interface pool) provided by Beehive. Note that there are only abstract interfaces in this pool.

During the development phase, the caller relies on the response interface in the interface pool and uses the interface as a parameter to obtain a service instance through the factory method provided by Beehive. This instance can provide services normally.

At runtime, the Beehive factory method constructs a service instance based on the service registration configuration. If the current runtime environment does not depend on the module that provides the service, it returns empty; if the current runtime environment has complete dependencies, it starts to construct the service and returns.

Through such a solution, decoupling between modules can be achieved.

Unified jump protocol & Rewrite engine

The unified dispatching protocol is a URL-based redirection solution that works with the Rewrite engine to decouple all App calls. Apple Core previously introduced this in an article, so I won’t go into detail here:

http://pingguohe.net/2015/11/24/Navigator-and-Rewrite.html

Differences between Beehive and Unified Jump & Rewrite

The purpose of Beehive and unified jump protocol is decoupling, but they focus on different points. Unified jump mainly serves interface decoupling, and the business requires strong dynamics of interface links; Beehive is module decoupling, solving the pain of development phase caused by strong module dependency.

The above is the decoupling process that we have experienced in the entire mobile Tmall in the past few years. In this process, we have had a lot of thoughts, stepped on many pitfalls, and of course accumulated a lot of useful tools. I hope that there will be more opportunities to share with you in the future, and you are also welcome to communicate with us and learn from each other.

Other recommended articles on Tmall Mobile:

Don’t write it rigidly! Practice of dynamic configuration center of Tmall App

Tmall App A/B Testing Practice

Safe Mode: Tmall App launches protection practice