A brief discussion on the core of the program - complexity

[[151780]]

By An Bolin

In "The Art of Unix Programming", complexity control is taken very seriously. There is a sentence in it that mentions that the core of a programming project is the control of complexity, and the simplicity principle is actually also talking about this.

I also wrote about this topic in 2008: Complexity and Habits. Seven years have passed, and I have experienced the trials of heavy projects such as "Tian Ya Ming Yue Dao" and have gained more insights.

The key to complexity

The key point of complexity is the burden that the program brings to the brain. It is equivalent to the difficulty for programmers to improve and develop programs. This burden grows approximately quadratically with the complexity of the module.

If the burden is low, then a program is easy to control and programmers can easily improve the quality of the program (including development efficiency, operational stability and operational efficiency).

Therefore, we do not need to minimize complexity at any time or under any circumstances. If a module is small in size, then we do not need to spend a lot of effort to simplify it further (of course, it is a good idea to do this if time permits, out of the instinct of self-improvement and excellence).

At the same time, low complexity does not mean the code with the least number of lines, but the code that puts the least burden on the brain. For example, in the code example given later, although the other writing method has more lines of code, it conforms to a more stable pattern, so the burden on the brain and psychology is lighter, so it can be considered as lower complexity code.

The practical significance of complexity control

Actual Value

Let’s start from a practical point of view: it is related to operational efficiency and development efficiency (of course other scalability and so on are also included, but the actual feeling in the project is that these two are particularly obvious).

In fact, I believed this without a doubt 7 years ago, but it was not easy to put it into practice. It was only a few years ago that the development principles were truly formed.

Development efficiency

This most profound understanding of principle came when I was developing the terrain system, including the underlying part of the editor (the UI part was done by another colleague) and the runtime part, from materials to height maps. The system was large and complex.

During the development process, it is inevitable to encounter changes in requirements (including very disruptive changes such as the capabilities of the material system and the size of the map).

The time is tight and the task is heavy. I always want to get things done as quickly as possible. During the development process, I didn’t do much code organization and overall system control. Then other groups can proceed simultaneously and then organize the code.

But for a large system, this strategy is not a good one.

When writing a program, the best way to ensure quality and efficiency is to always have a very clear understanding of the entire system at the code level. You know what you want to write, and during the writing process, the overall code is clear and reasonable, consistent with what you have in mind. Then you can write very quickly and calmly, and the whole process is very enjoyable.

However, if during the implementation process, you lack a good understanding and organization of the system, and hope to "just do it and then organize it", this is ok in a small situation, but in a large system, even if the thinking remains clear, the huge system lacks organization and becomes very complicated. Many things are in an unreasonably complex situation due to inconsistencies in the previous and subsequent designs - you need to memorize them.

The result of this is that even if you have a very clear idea of ​​the overall system design, during the programming process, due to a certain degree of confusion in the system, you will not be able to proceed with the entire process clearly and calmly. The whole process will be bumpy and exhausting.

So in the second half, I stopped and changed my strategy. I first did a thorough cleanup, removed the unnecessary parts, and then organized the code until it was completely ready to implement the new code. Only then did I start the new implementation. This is actually the fastest way and it is also pleasant and quick to write.

Operational efficiency

Regarding processing efficiency, the general basic approach is to profile hotspots and disable features based on the game situation.

But this is very limited. If you want to further improve performance and get close to the performance limit, you must do the following:

• Have a full understanding of each module
• It can be quickly iterated

Dealing with performance hotspots is a very efficient approach in the early stages of optimization. To be more precise, hotspot processing is a method of "efficiently improving performance in the presence of water."

However, in the pursuit of extreme performance, hotspot optimization is still not enough. Whether the performance consumption of a certain module exceeds what it should be, or whether a module ranked outside the top 10 does not actually need to run at high frequency, etc., these are all problems that hotspot processing cannot solve.

With a full understanding of the program, you can make more thorough adjustments, running a large number of operations in parallel, executing them at a low frequency, or directly optimizing them.

In practice, such processing will bring the performance of the program to a new level.

This principle is easier said than done. The difficulty lies in how to fully understand a very large system (for example, for "Tian Ya Ming Yue Dao", it is the entire client, covering hundreds of thousands of lines of code) and how to easily and thoroughly modify and optimize it.

So the key point comes back to complexity. Only when the complexity of the program is under perfect control can we do this job well.

Later in practice, during the optimization process, about half of the time was spent on adjusting and refactoring the code. Reasonable code will make the optimization more feasible and efficient.

Complexity Control Methods and Practices

In practice, I think the ability to control complexity can be broken down into three aspects: desire, goals and time accumulation.

eager:

First of all, the most effective way is to undertake actual development tasks that cover a very large range. In this case, you will feel the pain of complexity, and you will understand very clearly what complexity is, what is important, what drives you crazy, and what is just bluffing and insignificant. With sufficient desire, the subsequent accumulation and practice will be much easier.

Target:

There are many methods and practices, but the goal is much simpler, which is to always keep the entire system very clear at the code level. When developing, designing, and making decisions, you can be as smooth as still water. So to a certain extent, it can be said that complexity control is still quite subjective and depends on the timing. For example, sometimes the project is relatively small, even if the complexity control is not very good, it is still very clear. Hold on, then you can focus more on other aspects.

method:

In personal practice, you can pay attention to the following aspects:

- Task division + code organization: When smaller tasks are completed, start small-scale code organization to keep the code clean at all times

- Pattern + Natural: Accumulate more patterns. For example, a large piece of code is actually doing the work of pool. Then the complexity of this large piece is one word: pool. Make everything more natural and conform to the best practices of programming. In this way, there are fewer things you need to remember and pay attention to, and it is a very low complexity.

For example, the following code:

 int a[ 5 ];
 for ( int i = 0 ; i < 5 ; i++)
 {
 printf( "%d" ,a[ 5 ]);
 }

This is not a good practice in actual programs. When you see this code, you should instinctively notice that if the size of a[5] changes, there is a possibility of out-of-bounds access by for.

 #define ARRAY_NUM(a) (sizeof(a)/sizeof(a[ 0 ]))
 int a[ 5 ];
 for ( int i= 0 ; i<array_num(a);i++) {= "" printf( "%d" ,a[i]);= "" }<= "" pre= "" ><p>

Then when you see such code again, you will feel more at ease and just go through it. Then this can be considered to be of relatively low complexity (there are fewer things to pay attention to or remember deliberately).

Therefore, it is very important to maintain a summary and accumulation. The more you accumulate programming patterns or algorithms, the higher the dimension can be used when developing and thinking. This is very important for compressing complexity and improving the speed and quality of thinking.

And, at this level, a programming style that tries to return to basics is a more powerful programming style.

The enemy of complexity control

Not realizing the importance of complexity

I have come across many programmers (even most of them) who are insensitive to complexity, and place the importance of algorithms and efficiency far above complexity, and even take pride in writing very complex programs. This is not easy to communicate with, and only when you actually undertake a large number of program implementations and experience the pain of complexity can you have a true understanding.

Another problem is that they do not communicate with the project team in time and strive for enough time to deal with complexity and clean up the code. Many programmers do not have a full understanding of complexity, so it is unreasonable to ask the project manager to have a sufficient understanding. Basically, the more feasible method is for programmers to give sufficient communication and leave sufficient margin in the implementation time. If there is no awareness, no sufficient communication, or even ignoring complexity in order to please the manager and frantically pursuing the implementation time, it is too bad.

Progress issues

The situation of tight time and heavy tasks has been mentioned before, but it still occurs repeatedly in actual projects. This can actually be a big topic.

First of all, every programmer needs to establish a profiling mechanism for code implementation - I personally always use worklog, and then keep track of my own development efficiency, so that I can know which method is correct and faster. You can only know when sharpening the knife will not delay the work of cutting wood after profiling.

Adopt specific strategies according to the specific situation. In my personal experience, in most cases it is faster to implement and organize at the same time.

The basic skills of programming can be achieved quickly and stably, which requires long-term conscious accumulation.

good for the programmer's soul

Low-level programming is good for the programmer's soul." - John Carmack

I totally agree with what Card God said. To do the low-level code implementation, it is crucial for programmers to have a thorough understanding of the hardware and system. It is crucial for programmers to clearly understand how the entire program runs. Then you will be able to think better with a low-level mindset.

The same principle can also be applied to high-level complexity control. More excellent programming practices, better understanding of what to do, understanding of the system itself, and ultimately achieving the simplest implementation. The entire design and implementation process can make people feel calm and at peace, which is also "good for the programmer's soul".