How to choose an open source project that suits you to read

[[148226]]

People say that reading source code is a great way to improve your programming skills, but it is very difficult to find a source code that is suitable for you to read. There are so many excellent open source projects that you can never finish reading them all. And if you don't have a clear purpose and just read it because it's hot, you will get half the result with twice the effort.

I am more of a backend developer, so the following opinions are based on the perspective of a backend developer.

Starting from Node.js and Tornado

A few months ago, I learned the Tornado framework and used it to build a project; and I just started learning Node.js in the last few days. So there may be some inaccuracies in what I say.

Tornado is a combination of an asynchronous non-blocking server application and a lightweight web framework. Node.js is an open source cross-platform runtime environment. In my opinion, Node.js is basically a server application, because this part is almost exactly the same as the server part of Tornado.

Our main question is what kind of open source code should we choose to read. So far, the projects we can choose are Tornado and Node.js. Don't rush to make a decision, and continue to explore in depth. First, simplify the problem, because the advantages of Node.js are not only the high performance provided by asynchronous non-blocking, but also many other things. We will not ignore those other parts for the time being.

• Tornado = AIO Server + Web Framework
• Node.js = AIO Server

If you have studied operating systems, you will know that the reason why these two programs achieve such high performance under Linux is due to the epoll provided by the Linux 2.6 Kernel. My idea is that if Node.js runs under Linux, it will definitely call the epoll provided by the operating system; if it runs under Windows, it will definitely call the IOCP provided by the operating system.

Following this line of thought, I discovered two open source projects, libuv and pyuv.

• libuv: It is generally believed that libuv was born because of nodejs. The role of libuv is to hide the differences between operating systems from users, encapsulate Linux's libev and Windows' IOCP, etc., and provide a cross-platform asynchronous operation library.
• pyuv: Provides an interface for Python to call libuv

Clear relationship

The concepts (projects) mentioned above are nodejs, tornado, libuv, pyuv, libev, and IOCP. Most of them can be found in open source code. If you want to choose a suitable project for your study, the best way is to first clarify their relationship.

First of all, Tornado and node.js can both be used as website backend server applications. Tornado provides a web framework.

Tornado's web framework provides view and routing functions, which can be used to easily write a web application in MVC mode. For the web framework, it directly gets the user's request from the server application, processes it, and gets a response, which is directly returned to the server. There is no need to understand the details of how the request is transmitted and how the response is returned to the user.

Since many clients will send requests to the server at the same time, the server needs to process these requests as quickly as possible. To improve performance, the CPU is usually allowed to process as many I/O requests as possible per unit time. The method is usually to use blocking multithreading or non-blocking single threading (of course, multithreading or multi-process is also possible).

No matter which I/O model is used, it must be supported by the operating system. If the CPU does not have an interrupt function, the operating system will not have a lock function, and there will be no semaphore, no Monitor and other synchronization mechanisms.

For Tornado, the Python interpreter provides the above series of synchronization mechanisms. Therefore, what kind of I/O model Tornado can use depends directly on the Python interpreter.

The Python interpreter runs on the operating system. If the operating system does not provide a lock, the interpreter cannot create a lock. Therefore, what kind of I/O model Tornado can use indirectly depends on the operating system.

I think the same is true for Node.js.

Draw Conclusion

• If you want to know how a website framework puts together routing, views (html, css, js, etc.), and data processing, you can read the web framework part in the Tornado source code.
• If you want to know how the server application uses the interfaces provided by the operating system to complete high concurrency processing when there are many requests coming at the same time, you can read the server part of Tornado or the source code of Node.js.
• If you want to know how to build a cross-platform unified interface based on different operating systems, you can read the source code of libuv
• If you want to know how these I/O interfaces are implemented under the Linux operating system, you can read the source code of libev
• If you want to know how to use libuv, you can also call the cross-platform I/O interface in Python. You can read the source code of pyuv

Not only that, we can learn more from the conclusion: If you want to see the source code of libuv and know how libuv is implemented, you must first know how to use it. Only when you know its function can you read the source code with a purpose. To know how to use libuv, you have to refer to a small part of the Tornado source code above it, or consult the libuv documentation.

General method for selecting source code

1. List some keywords of interest
2. Understand the upper and lower levels of keywords
3. Sort out the entire relationship diagram
4. Draw conclusions, make choices

After the steps just now, we have gradually sorted out a clearer route from user-initiated requests to the operating system and even the hardware level from some vague understanding of Tornado and Node.js. Although we haven't read a line of source code yet, we already know what knowledge we can learn from reading different source codes. On this basis, choosing the source code that suits your current needs to read can achieve twice the result with half the effort.