Are you a seventh-stage programmer?

[[130300]]

Preface

A programmer's programming skills will gradually improve as he or she accumulates experience. I think programming skills can be divided into several levels.

The following discussion of the programming ability hierarchy model will be carried out through two dimensions.

One dimension is the level of programming skills, and the other dimension is the level of domain knowledge.

Programming skill level

Programming skill level refers to the ability of programmers to design and write programs. This is the foundation of programmers.

Section 0 - Non-Programmers:

When a beginner programmer encounters a problem, he is completely confused and has no idea how to program to solve the problem. In other words, he is still a layman and cannot be called a "programmer". The computer is still a mysterious black box in front of him.

Section 1 - Basic Programmer:

After learning programming for a period of time, when you receive a task, you can write a program to complete the task.

The written code can work under normal circumstances, but in actual operation, various bugs will appear when encountering some special conditions. In other words, you have the ability to develop demo software, but when the developed software is actually delivered to customers for use, you may be scolded by customers.

The programmer has written the program, but he himself doesn't know why it sometimes works properly and sometimes doesn't.

When a bug is encountered during operation or the requirements change, the code needs to be modified or added, and soon the program becomes chaotic, the code bloated, and full of bugs. Soon, even the original developer himself is unwilling to take over the maintenance of the program.

Section 2 - Data Structure:

After a period of programming practice, programmers will realize the meaning of the old saying "data structure + algorithm = program". They will use algorithms to solve problems. Furthermore, they will realize that algorithms are essentially dependent on data structures. Once a good data structure is designed, a good algorithm will also come into being.

It is impossible to grow a good algorithm if the data structure is designed incorrectly.

I remember a foreign sage once said: "Show me your data structure!"

3 paragraphs - object-oriented:

After that, programmers will appreciate the power of object-oriented programming. Most modern programming languages ​​support object-oriented programming. But it does not mean that if you use object-oriented programming language to program, use classes, or even inherit classes, you are writing object-oriented code.

I've seen a lot of procedural code written in Java, Python, and Ruby.

Only when you have mastered interfaces, polymorphism, and the relationship between classes and classes, and between objects, can you truly master object-oriented programming techniques.

Even if you are using a traditional programming language that does not support object-oriented programming, as long as you have "objects" in your mind, you can still develop object-oriented programs.

For example, when I program in C, I consciously use object-oriented techniques to write and design programs. I use struct to simulate classes, and put functions of the same concept together to simulate classes. If you doubt whether you can write object-oriented code in C, you can take a look at the Linux kernel, which is written in C, but you can also see the strong "object" flavor between the lines of its source code.

It is not easy to truly master object-oriented programming techniques.

In my technical career, there are two obstacles that gave me the most headaches.

One hurdle was the transition from Dos to Windows development. For a long time, I couldn't understand the concept of framework. In the Dos era, it was all about calling function libraries, and your program actively called functions. In the Windows era, it was replaced by frameworks. Even your main program is actually called by the framework. The UI thread will get messages from the operating system and send them to your program for processing. The Spring framework, which Java programmers are familiar with, is also such a reverse calling framework.

Nowadays, because the term "framework" seems very high-sounding, many "class libraries"/"function libraries" call themselves "frameworks". In my opinion, this is an abuse of the name.

"Class library"/"function library" means the code I wrote calls them.

"Framework" means that I register callback functions to the framework, and the framework calls the functions I write.

Another hurdle was object-orientation. For a long time I didn't know how to design the relationship between classes, and I couldn't design a class hierarchy well.

I remember reading a book written by a foreign expert. He talked about a very simple and practical object-oriented design technique: "Describe the problem. Then find the nouns in it and use them to build classes. Find the verbs in it and use them to build class methods." Although this technique is quite useful, it is too grassroots. It has no theoretical basis and is not rigorous. If the problem is not described well, the resulting class system will be problematic.

There should be many ways to master object-oriented thinking. I got inspiration from relational databases to understand and master object-oriented design ideas.

In my opinion, a table in a relational database is actually a class, and each row of records is an instance of a class, that is, an object. The relationship between tables is the relationship between classes. O-R mapping technology (such as Hibernate) is used to map from object-oriented code to database tables, which also shows that classes and tables are indeed logically equivalent.

Since database design and class design are equivalent, to design an object-oriented system, you only need to use the design techniques of relational databases.

The relational database table structure design is very simple:

1. Identify the relationship between tables, that is, the relationship between classes. Is it one-to-one, one-to-many, many-to-one, or many-to-many? This is the relationship between classes.

2. Identify the fields of the table. An object certainly has countless attributes (e.g., a person: height, weight, gender, age, name, ID number, driver's license number, bank card number, passport number, Hong Kong and Macau Pass number, work number, medical history, marital history, etc.). When we write a program, we only need to record the attributes we care about. These attributes of concern are the fields of the table, that is, the attributes of the class. "From the vast expanse of water, I take a sip"!

4th paragraph - Design mode:

I once saw this sentence on the Internet: "If you don't have 100,000 lines of code, don't talk to me about design patterns." I agree with it.

I remember the first time I read Gof's book Design Patterns, I found that although I didn't know about design patterns before, I still consciously used some design patterns in the actual programming process. Design patterns are objective laws of programming. They were not invented by anyone, but were first discovered by some early senior programmers.

Without design patterns, you can also write programs that meet the requirements. However, once the subsequent requirements change, your program will not be flexible enough and will be difficult to continue. After the real program is delivered to the customer, there will definitely be further demand feedback. And the development of subsequent versions will definitely increase the requirements. This is a reality that programmers cannot avoid.

When writing UI programs, whether for Web, Desktop, Mobile, or Game, you must use the MVC design pattern. Otherwise, your program will not be able to survive the subsequent changes in UI requirements.

The most important idea of ​​design pattern is decoupling, which can be achieved through interfaces. In this way, if the requirements change in the future, you only need to provide a new implementation class.

The main design patterns are actually object-oriented. Therefore, design patterns can be considered as an advanced stage of object-oriented. Only by mastering design patterns can we be considered to have truly and thoroughly mastered object-oriented design skills.

When I learn a new language (including non-object-oriented languages, such as functional programming languages), I always look at how various design patterns are implemented in this language after understanding its syntax. This is also a trick for learning programming languages.

5th paragraph – Language Expert:

After a period of programming practice, programmers have become quite proficient in a common programming language. Some have even become "language lawyers", good at explaining the usage and pitfalls of the language to other programmers.

Programmers at this stage are often loyal believers in the language they use, and often argue with users of other languages ​​in communities and forums about which language is the best programming language. They believe that the language they use is the best programming language in the world, bar none. They believe that the programming language they use is suitable for all scenarios. They only see hammers, so they will treat all tasks as nails.

6th segment – ​​Multilingual experts:

Programmers at this stage have learned and mastered several programming languages ​​due to work or simply out of interest in technology. They have learned the different design ideas of different programming languages ​​and have a better understanding of the strengths and weaknesses of each language.

They now believe that programming language is not the most important thing, programming language is just basic skills.

They will now choose different programming languages ​​to solve problems based on different task requirements or different resources, and will no longer complain about not using a favorite programming language for development.

There are many schools and philosophies of programming languages, and some programming languages ​​support multiple programming paradigms at the same time.

Statically typed programming paradigm

A programming language that uses a statically typed programming paradigm, where variables need to have their types explicitly specified. Representative languages: C, C++, Pascal, Objective-C, Java, C#, VB.NET, Swif, Golang.

The benefits of doing this are:

1. The compiler can find type errors at compile time.

2. The compiler can improve performance by knowing type information during compilation.

This paradigm assumes that the programmer must know the type of the variable. If you don't know the type of the variable, then you can't mess around! When compiling, the program will report an error.

Both Swift and Go are statically typed programming languages, but they do not require explicit type specification, but can be automatically determined by the compiler through inference.

Dynamically typed programming paradigm

A programming language that uses static type programming paradigm, where variables do not need to be explicitly specified. Any variable can point to an object of any type. Representative languages: Python, Ruby, JavaScript.

The philosophy of dynamic typing can be summarized by the concept of duck typing. The duck test proposed by James Whitcomb Riley can be expressed as follows: "When you see a bird walk like a duck, swim like a duck, and quack like a duck, then this bird can be called a duck."

This paradigm assumes that programmers must know the type of a variable and the methods and properties it supports. If you don't know the type of a variable, then you're out of luck! The program will crash when it runs! Who should you blame for the program crash? Blame yourself, you're not a qualified programmer!

The benefits of dynamic typing are:

There is no need to explicitly define interfaces and abstract types. As long as a type supports the required methods and properties, it is OK. The program will be quite flexible and simple. The interface/base class that C++, Java, and C# regard as the lifeblood is nothing in dynamic languages!

The disadvantages are:

1. If the type is incorrect, the compiler cannot find the error, and the program crashes at runtime.

2. Because the compiler does not know the type of the variable, it cannot optimize performance.

Object-oriented programming paradigm

Object-oriented programming paradigm, which has been popular since the late 1970s. It supports classes and class instances as modules for encapsulating code. Representative languages: Smalltalk, C++, Objective-C, Java, C#, VB.NET, Swift, Go, Python, Ruby, ActionScript, OCaml.

Early programming languages ​​were all process-oriented. Sequences, conditions, loops, and functions were all built into them. As the size of the code grew, people found it necessary to modularize the code. The code corresponding to a concept was placed in one file, which facilitated concurrent development and code management.

People also discovered the law of "Program = Data Structure + Algorithm". Therefore, the data structure and function corresponding to a concept should be placed in one file. This is the concept of class.

The object-oriented programming paradigm has indeed greatly improved production efficiency, and has therefore been widely used. Therefore, there are many languages ​​that support the object-oriented programming paradigm at the language level.

Although C language does not support object-oriented programming paradigm at the language level, modern C language development will apply object-oriented modular thinking, placing data structures and functions of the same type in one file and using similar naming methods.

After all, C language does not support object-oriented programming at the language level, so many programmers want to add object-oriented support to C language. The representatives are C++ and Objective-C.

C++ is a new language, but most language elements are compatible with C.

Objective-C is fully compatible with C. Objective-C adds a thin layer of syntax sugar to C to support interfaces (that is, classes in other languages) and protocols (that is, interfaces in other languages). Even the initial implementation of Objective-C was a precompiler for the C language. To be honest, except for the added syntax that does not quite conform to the C flow, the object-oriented system design of Objective-C is actually quite sophisticated. Jobs had a keen eye for it in his early years and took Objective-C into his pocket. Because it was closed in the Apple/NextStep system, few people knew about it. With the popularity of the iOs system, Objective-C has become famous all over the world in recent years.

Functional programming paradigm

Functional programming is a programming language invented by mathematicians who believe that programs are mathematical functions. Representative languages: Lisp, Erlang, JavaScript, OCaml, Prog.

Many experts have advocated functional programming languages, believing them to be revolutionary. But I think they overestimate the power of the functional programming paradigm. I don't think the functional programming paradigm is superior to the object-oriented programming paradigm.

Functional programming languages ​​are based on functions, and they do not have the concept of classes. However, their functions are not like those in traditional procedural languages, and they support the concept of "closures".

In my opinion, the functions of functional programming languages, that is, "closures", are actually "classes". As programming languages ​​have developed to this day, they need modularization, that is, they need to combine "data structures" and "algorithms". No matter what language, there is no way out if the programming method does not combine them.

Object-oriented programming languages ​​use classes to combine "data structure" and "algorithm". The core of a class is the "data structure", that is, its "attribute", rather than the "algorithm", its "function". In a class, functions are attached to attributes.

Functional programming languages ​​use closures to combine "data structures" and "algorithms". Functions can access external fields. "Attributes" are attached to "functions".

"Class" is essentially equivalent to "closure". Many object-oriented programming languages ​​now support closures. By observing their codes, we can find that they actually use "class" to implement "closure".

Which is easier to use, "class" or "closure"? Obviously "class".

Closures are more concise, so they are often used to replace anonymous classes in object-oriented programming languages. It is too troublesome to write a class with only one function as a class, so it is better to write it as a closure, which is more concise.

Let me complain about the OCaml language. Its predecessor, Caml, is a pretty good functional language itself, but it added a complete set of object-oriented mechanisms to it, supporting both object-oriented and functional programming paradigms. It can easily cause a brain split like C++.

There are also many object-oriented language fans who find JavaScript annoying and always want to add object-oriented support to JavaScript. ActionScript is one of the attempts. I have used it and it is really not much different from Java.

Let me complain about ExtJS again. When I was choosing a web front-end development framework, I compared ExtJS and JQuery.

ExtJS was obviously developed by Java experts. It used JavaScript to simulate the design concept of Swing and created a UI library.

The developers of JQuery obviously understood the functional programming paradigm of JavaScript, and created a UI library based on the characteristics of JavaScript's dynamic functional programming language, which immediately killed ExtJS.

From the story of ExtJS and JQuery, we can see how important multi-language programming skills are. The author of ExtJS is proficient in and loves Java, so he used JavaScript as a hammer, hitting Java randomly, which was a thankless task.

Functional programming languages ​​also have some tricks such as tail recursion. Tail recursion can avoid stack overflow during recursive calls.

Template Programming Paradigm

Template programming is to use types as parameters, and a set of functions can support any number of types. Representative language: C++.

The need for template programming was invented when C++ was developing container libraries. Because containers need to store objects of any type, there is a need for generics.

C++ template programming is to create corresponding types of code according to the usage in the source code during compilation. In addition to C++, Java and C# also have similar mechanisms called "generics", but their implementation methods are very different from C++ templates. Their compilers do not generate new code, but use forced type conversion to implement it.

In programming languages ​​without templates/generics, how do you store objects in containers? You can store and access objects of common base class types (Java, C#) or void* pointers (C), and cast them to the actual type when you retrieve them. In dynamic type languages, you don't care about types, and it doesn't matter. You can just throw any object into the container and use it directly after you retrieve it.

Some C++ experts have come up with "template metaprogramming" based on templates. Because template programming is done by the C++ compiler, template metaprogramming is to let the compiler calculate, and the result will be calculated after compilation. I don't know what this thing is used for except for research and showing off skills?

summary

I think there are several criteria for whether a language is worth learning:

1. Whether you want to use it or not, you have to learn it. There is no doubt about it. After all, we all have to eat.

2. Does its language feature give you a refreshing feeling? If so, then it is worth the money. For example, Go language abolished exceptions and used multiple return values ​​instead. I agree with this. In fact, I have actively stopped using exceptions for many years. Because, I think since C can survive well without supporting exceptions, why do we need exceptions? If an error occurs, return an error code. If the error cannot be recovered, just Abort the program! Moreover, exceptions actually violate the principle of process-oriented programming. A function should have only one entry and one exit. Throwing an exception means there is one more exit.

3. Are you good at a certain field? If you only have a hammer, you can only hammer all tasks as if they were nails. But if you have multiple tools in your toolbox, you will be much more comfortable with different tasks.

7th section - Architecture design

You also need to master the ability of architectural design to design excellent software. There are some techniques for architectural design:

1. Layering

A software is usually divided into:

Presentation layer – UI part

Interface layer – the communication interface part of the background service

Service layer – the actual service part

Storage layer - persistent storage part, stored in files or databases.

Layered software can decouple modules, support parallel development, and is easy to modify and improve performance.

2. SOA

Modules are interconnected through network communication and loosely coupled. Each module can be deployed independently, and deployment instances can be increased to improve performance. Each module can be developed using different languages ​​and platforms, and previously developed services can be reused. SOA, commonly used protocols include WebService, REST, JSON-RPC, etc.

3. Performance bottleneck

1) Change synchronization to asynchronous.

Use memory queues (Redis), workflow engines (JBpm), etc. Memory queues are prone to data loss, but they are fast. The workflow engine will save the request to the database.

By converting synchronous requests to asynchronous requests, basically 99.99% of performance problems can be solved.

2) Use single-machine parallel hardware for processing.

For example, use GPU, FPGA and other hardware to process and improve performance.

3) Use cluster computers for processing.

For example, a Hadoop cluster uses multiple computers to process data in parallel.

In your own software stack, you can also deploy multiple copies of a module and process them in parallel.

4) Use cache to satisfy requests. After the commonly used content is cached, a large number of user requests are just memory reads, and performance will be greatly improved.

Cache is God's algorithm. I remember that its performance is only slightly lower than the best performance, just like you are God and can foresee the future. Now X86 CPU has encountered the main frequency limit, and the main way to improve CPU performance is to increase the cache.

4. Big system, small implementation

Don't panic when you encounter a large system. Divide it into multiple modules, use multiple small programs, and solve it through SOA collaboration. This adheres to the design concept of Unix. A large number of small programs with single purposes have been developed on Unix. It advocates that users use pipelines to allow multiple small programs to collaborate and solve user needs. Of course, pipeline communication has too many restrictions and is not flexible enough. Therefore, now we can use URI and SOA to allow multiple programs to collaborate. Applications on Andorid and iOS now collaborate through URI. Is this also a modern development of Unix design thinking? !

5. Sharding

There is a trend now, which is to get rid of IOE. I-IBM mainframe, O-Oracle database, E-EMC storage. In the past, large systems often used IOE to de-architecture, deploying an Oracle database on the mainframe, and the Oracle database used EMC storage to save data. IOE is the most powerful computer, database and storage today. But they will also be unable to withstand the massive system one day.

Oracle database shares everything and can be run on a computer cluster (server nodes cannot exceed 16). All computer clusters share a single storage.

The movement to eliminate IOE marks the bankruptcy of the ShareEverything model. ShareNothing must be used so that the system can be infinitely expanded.

You can use MySQL database to handle data of any size. The prerequisite is that you know how to use Sharding. Divide a large system into several small systems, and divide them into several cheap servers and storage. A more modern method is to divide them into a large number of virtual machines.

For example, the 12306 website of the Ministry of Railways. We know that train tickets belong to a certain train. So if we divide each train as a unit, we can divide the 12306 website into thousands of modules. A virtual machine can carry several modules. When some trains become performance bottlenecks, they can be migrated to independent virtual machines. Even if some listed services are unavailable in the end, the system will not be completely unavailable.

The 12306 website has only one global part, which is user login. This can be handled by a third party, such as allowing users to log in using WeChat, Weibo, QQ, etc.

You can also implement the user login service yourself. Or use multiple Redis servers in a slicing manner to provide services. The Redis server stores the sessionId and userId, role, permissions and other information of each logged-in user. The sessionId is randomly generated, and some of its bits can be selected to identify which Redis server it is on. After the user logs in, the sessionId is sent to the client. The user sends the sessionId back to the server each time he requests. The server sends the sessionId to the Redis server to query its user information and process the user request. If the sessionId cannot be found on the redis server, the user is asked to log in. Even if all registered users log in at the same time, not much memory is needed. Moreover, when there is too much session memory, the session of the user who logged in first can be deleted, forcing him to log in again. The number of active users at the same time will not be too many.

Domain Knowledge Level

All the previous levels focus on programming skills, which are the basics, and they don’t generate much value. But too many programmers waste too much time on those basics.

Some programmers like to delve into programming languages. Whenever a new programming language comes out or an old language becomes popular, they will invest their energy in researching it. I am one of them, and I have wasted a lot of energy on programming languages ​​and tricks.

I think C++ is a very big pitfall. It was originally developed as an object-oriented C language. Later, template programming was discovered, and template programming and further template metaprogramming were strongly advocated. Recently, new standards such as C++11 and C++14 have been introduced, adding many new things, such as functional programming and type inference. C++ is too complicated, and too many pitfalls consume a lot of programmers' energy. When I use C++, I only use the object-oriented part and the template part, and do not use other overly sophisticated features.

Computer science is a very broad subject. There are many areas of knowledge that need to be studied in depth before we can write valuable programs. Software must be integrated with the industry and put into practice to be valuable. You cannot write valuable programs just by studying programming skills without understanding domain knowledge.