A "mouth cannon" about architecture

Author: Duan Hechen

The title of the article is very casual, and has a slight "sly" intention to deceive clicks; but the content is full of sincerity, and I'm sure you have already felt it.

This is a discussion on the software architecture of the Android client of Toutiao. It is named "mouth cannon" because it is somewhat unrealistic. At the same time, it is mixed with some methodologies, which is not only applicable to client software architecture, but also to other work scenarios. I hope it will be helpful to everyone.

In order to maximize the reader's sense of involvement, and with "we" as the subject, we will look at the challenges, judgments and strategies of the architecture.

Our challenges are high expectations

Excellent companies have high expectations for architecture. They all hope to have a good top-level design, a unified understanding from top to bottom, follow common specifications, write comfortable code, and even be a little lazy. Is there a "once and for all" architectural design that can ensure the longevity of the foundation?

However, high expectations mean high disappointment. When faced with disappointment, we tend to become anxious:

When can code be written to look like it should be written? Why is it that now it feels like climbing a mountain of shit?

When will the document be written in both concise and detailed terms? Why is it that the concise ones are incomprehensible now, and the detailed ones are redundant?

When will the tool be better and more powerful? Why does it keep failing now? I have to wait for years for the functions I want.

When will I be able to find a sense of accomplishment in architecture work, instead of just trying to run away?

Big responsibility

The ultimate cause of a large number of problems is code problems: unreasonable design, irregular use, obscure logic, and too many coding pitfalls.

There is no single team that can take responsibility for these issues, and we have received many complaints:

Who the hell wrote this? It's disgusting. I'm going to start over and show you my true strength.

XX planted a landmine here, but XX doesn't care anymore. Now that things have come to this, I can only take the last resort.

This shouldn't be used like this at all. It wasn't originally designed for this scenario. Is it my fault for messing around?

Shit, this is a hidden skill. It secretly changed my code during compilation. I couldn't find where it was infiltrated.

On the one hand, it feels good to talk big and we get some temporary relief from "complaining" about historical codes; on the other hand, it also means that the responsibility is passed on to us: if we handle it well, our output may still be regarded as dregs, but if we handle it poorly, we will ruin the future of business development.

Things are difficult

The architecture never faces a single business problem, but rather cross-cutting issues involving collaboration among multiple businesses and multiple people. Moving forward with this heavy burden is the norm.

Business is ever-lasting, and historical baggage is superimposed on new scenarios. Just a simple move of the knife will pull out the carrot and bring out the mud. For example: Toutiao's October 2021 version has XXXX components, which has doubled compared to a year ago; the number of classes is XXXXX; the number of plug-ins is XX; the number of warehouses is XX; the number of ttmain warehouse permissions is XXX people. (XX represents the order of magnitude, and the specific number is hidden, ^_^)

Technology stacks emerge in an endless stream. On the one hand, they must remain mature and stable, and on the other hand, they must actively explore implementation. Architecture students must be familiar with multiple technology stacks. For example, cross-end technologies usually coexist in client businesses (H5/Hybrid/Mini Programs/Lynx/Flutter). Which technology stack should a business choose to carry, and how much does it cost? What are the limitations after selecting a technology stack, and are there insurmountable obstacles?

Slow efficacy

We often say that the code complexity is high, and we regard reducing complexity as one of the key tasks in the direction of architecture; however, there are many factors that affect complexity. From an external perspective, there are three angles: subjective feelings, objective indicators, and industry benchmarks; from an internal perspective, there are three angles: engineering organization, code implementation, and technology stack. Even if we optimize the engineering structure factor well, it is difficult to feel a significant reduction in complexity in a short period of time.

We often talk about governance, but in fact it means designing a mechanism that operates under this mechanism until the disease is cured.

Just like when an old Chinese doctor prescribes a prescription, what he prescribes is not a specific medicine, but a method to deal with the symptoms. Whether it is a useful prescription still needs to be tested by practice and time. I hope we will not become quack doctors who randomly pick up a few herbs and stew them, and then boast that the medicine can cure the disease.

Our Verdict

Architecture issues are commonplace

Whoever reviews the architecture problems will inevitably rehash some old ideas; whoever plans the architecture direction will inevitably use the keywords "burden reduction", "refactoring", "reuse" and "standardization". The difficulty lies in rehashing old ideas and implementing the direction.

Architectural direction always exists

Architecture is not limited to the initial stage of a product, but accompanies the entire life cycle of the product. Architecture is not static, it is only suitable for specific scenarios. The past architecture may not be suitable for the present, and the current architecture may not be able to predict the future. Architecture is constantly evolving with the business. There will never be a situation where the architecture direction has reached its limit and there is nothing to do. Architecture is always vibrant.

Enforce compliance with standards: Business-common components are usually required to gradually sink to the basic component layer, but this standard can easily be broken over time.

A mature team is needed: domain experts (roles who are very familiar with business details) and the development team need to work closely together to build the core domain model. However, blindly trying DDD often underestimates the practical cost of the domain-driven design methodology, such as complicating simple problems and falling into the trap of over-emphasizing technical models.

By far the most popular software architecture design pattern for commercial applications is the Big Ball of Mud (BBoM), which is "...a haphazardly constructed, disorganized, messy, haphazardly pasted jungle of code without a clue."

The ball of mud pattern will stifle development, and even though refactoring is worrying, it is taken for granted. However, if there is still a lack of attention and consideration for domain knowledge, new projects will eventually become a ball of mud. No developer wants to deal with a big ball of mud, and for the enterprise, being stuck in a big ball of mud means losing the ability to quickly realize business value.

——"Domain-Driven Design Patterns, Principles and Practices" Scott Millett & Nick Tune

Complex systems continue to increase entropy

As long as the business continues to develop, increasing complexity is an inevitable trend, which is in line with the law of increasing entropy in thermodynamics.

The process of increasing complexity entropy can be viewed from two dimensions: the cost of understanding becomes higher and the difficulty of prediction becomes greater.

Understanding cost: scale and structure are two factors that affect the understanding cost

The grand scale is hard to comprehend. For example, it is easy to get lost in a city, but in a village there are only a few roads.

Complex structures are difficult to understand. For example, a clock is more difficult to understand than a pair of underwear.

When the demand increases, the scale of the software system will also increase, and this growth trend is not linear, but steeper. If the demand also changes in ways that were not anticipated in advance, and we do not have adequate risk response measures, we will inevitably compromise on the design under time pressure, treat the symptoms without addressing the root cause, and patch up various parts of the system, thereby accumulating technical debt. When the technical debt accumulates to a certain critical point, the quantitative change will lead to a qualitative change, and the complexity of the entire software system will reach its peak, entering the old age of decline and becoming a "terrible" legacy system.

Just like the "cow rule" of the feedlot: cows gradually age and eventually have no milk to produce; however, at the same time, the cost of raising them is rising.

——《Implementing Domain Driven Design- Deeply Understanding the Complexity of Software》Zhang Yi

Prediction difficulty: current chips are not enough to cope with future changes

Business changes are unpredictable. For example, Toutiao was originally just a single-end consulting product. Five years ago, no one would have designed the Lite version, Douyin, and Dongchedi. The changes brought about by multiple terminals and new business scenarios are unpredictable. Many times, we only need to achieve "appropriate" architecture design at the moment, but we need to keep it as "orderly" as possible. Once we are out of "order", it will inevitably lead to chaos and become more unpredictable.

Technological changes are unpredictable. For example, as a Java developer, the simplicity of Lambda expressions, the pleasure of functional programming, and the experience of declarative/responsive UI are all "really good" technological changes. However, old Java versions and supporting dependencies need to be upgraded. Once upgraded, they will be accompanied by daunting problems such as coexistence of multiple versions and dependency hell (transitive upgrades). Many times, we don't need and can't make architectural designs for future technologies, but we need to keep the architecture "clear" so that we can embrace technological changes faster.

Since it is destined to be a disadvantageous situation, then whoever runs to the end will win.

Too many process specifications will make everyone feel like a puppet, and puppets are destined to be blown away by the wind.

We should "emphasize" some principles more, such as: divide and conquer, control scale, keep the structure clear and consistent, instead of requiring everyone to design the architecture according to a certain guide, which will neither reduce complexity nor keep up with changes. "Emphasis" does not directly solve the problem, but highlights the important problems, so that everyone can find solutions to the problems themselves under certain principles.

Our approach is: define the problem → determine the architecture → implement the solution → review the results. The earlier the steps, the more important and abstract they are, the more difficult they are, and the more they reflect the architect's skills. Therefore, the first step of our approach is to recognize the problem.

Identify the problem

Problem Classification

The problems of architecture are complicated and intertwined. Putting all the problems together, we can prioritize them and classify them into different categories. By classifying the problems, we can match the corresponding people to solve the problems within a certain boundary.

Engineering architecture:

Business structure:

Basic abilities:

standardization:

Problem classification

Challenges, problems, and means are often confused. Which are challenges? Which are problems? Which are means? In fact, these are all the same thing, namely contradictions. It’s just that the contradictions are at different levels in different scenarios. Here is an example:

We believe that the current R&D experience cannot meet the needs of the growing business. This is a contradiction, a current challenge, and a primary problem. To handle this contradiction, we have to break it down, so there are secondary problems: Is our code logic optimized enough? Is the R&D process convenient enough? Are the documentation tools complete enough? Secondary problems are also contradictions, and solving secondary problems is our means of dealing with primary contradictions. By going step by step, we can grasp some basic capabilities that we need to focus on optimizing and building: plug-in, hot update, and cross-end capabilities.

In the specific practice process, the basic technical capabilities need to be further disassembled. For example, there are many hot update capabilities (Robust/Qzone Super Patch/Tinker at the Java layer, Sophix/ByteFix at the Native layer, etc.), and different hot update solutions have their own advantages and disadvantages, and their applicable scenarios are also different. We must make judgments based on the current situation, learn from the strengths of many solutions, and either make greater technological breakthroughs or integrate existing technical solutions for combined innovation.

Be diligent in thinking about the issues behind the problems

Henry Ford said, "If I ask customers what they need, they will tell me they need a faster horse." From this sentence of Henry Ford, we can extract the most direct problem: customers need a faster horse. Looking for a solution based on this problem itself may never give a satisfactory answer: find a better breed, a more scientific way to train horses.

Think about the question behind the problem: why does the customer need a faster horse? Maybe the customer wants a faster means of daily transportation. After going up a level, we immediately found a better solution: building cars.

We cannot just focus on the problem itself, we also need to see the problem behind it, then it will be easier to find more solutions.

Cognitive Pyramid

I would like to encourage everyone by citing the cognitive pyramid model, which will enable us to extract the wisdom to solve problems from the most original data.

• DATA: The bottom layer of the pyramid is data. Data represents various events and phenomena. Data itself has no organization, structure, or meaning. Data can only tell you what happened, but not why it happened.
• INFORMATION: The next level up from data is information. Information is structured data. Information is very useful and can be used for analysis and interpretation.
• KNOWLEDGE: The next level above information is knowledge. Knowledge can organize information and tell us the logical connection between events. Clouds cause rain, and the weather becomes cooler because of rain. These are all knowledge. Idioms and thinking routines are all knowledge. Models can be said to be a kind of advanced knowledge that can explain some things and make predictions.
• WISDOM: The top layer of the cognitive pyramid is wisdom. Wisdom is the ability to identify and select relevant knowledge. You may have mastered many models, but when it comes to the specific problem, which model to use and whether you dare to use this model is wisdom.

This is the “DIKW model”.

Step by step

We can’t wait or rush to solve the problem of architecture. A large application software does not require all parts to be perfectly designed. For some low-complexity areas or areas that are unlikely to be invested, it is sufficient to meet the conditions for availability without investing in high-quality construction costs.

Take the complexity of managing headlines as an example:

• Long-term architectural goals: wider (multi-terminal reuse), faster (single-terminal development speed), better (problem clearing and pre-interception)
• Current prominent problems: Too much coupling between businesses, lack of standard specifications, redundant and obscure code

The details have been censored, please do not mind them. The key is to clarify the problem and then spiral upward to have a long-term direction and short-term feedback.

at last

After a lot of output, we must not forget the humanistic care, after all, people plan. Architecture lions should be worshipped, they have a far-sighted vision and are good at making plans; but architects need to be enlightened more, they may be "shoveling shit" all year round, they expect to be recognized, and some of them don't have a partner... As they work, the story of architecture is still there, but people seem to have already turned the page.