Ongoing front-end engineering and performance optimization

As a developer, not just a front-end developer, you can't avoid optimization. Our goal is to make the product faster. The object of optimization is not only the product itself, but also our daily development process. From joining the team to now, about half of my time has been spent on optimization. During this period, I encountered many problems. Looking back, optimization is not that simple. It is also a very trivial thing. It is really not easy to create a painless development process and to be able to continuously optimize the product relatively easily. There is no silver bullet. Any project will take a lot of time and effort. But any improvement that is worth improving makes us feel that our efforts are not in vain. Sometimes I just want to be lazy and free myself from boring repetition.

When I first came in, there were three people in my team who were dedicated to front-end development. The project was relatively simple, and predecessors had already done modularization, but they felt it was not thorough enough. Later, the project gradually became larger, and with the reconstruction of the classmates, the number of people reached 13 at the peak. Many problems were encountered in this process. The front-end framework evolved three times, all because of some difficult problems, and adjustments had to be made. There is no absolute good or bad, only whether it is suitable or not.

11.0 Era

The early modularization has been done well, at least there is no need to spend a lot of time to refactor the code. The module division is as shown in the figure below, and the logical level is relatively clear.

[[153121]]

The mainstream libraries that front-end modularization relies on are Seajs in China and requirejs abroad, which will not be described here. Seajs is used as the module manager, zepto is used as the basic library file, and lib mainly contains the mainstream third-party library files used in the project.

We know that the biggest disadvantage of modularization is the increase in the number of HTTP requests , so files must be merged when going online. The package module in the figure below is a large collection of files, which packages many JS modules. Except for the basic library files and business module layers in the figure above, most files are packaged in package.js when going online.

[[153122]]

The JS requests for most pages are as follows:

Careful students may notice two problems: file size and loading time. The screenshots just now were taken on a PC, and the performance on mobile phones and different network environments will be even worse.

Now let's look at the directory

[[153123]]

Problems:

• The directory looks standard, but it actually mixes public and business items together.

• Most files are merged into one file, and the merge strategy is unreasonable.

• The third problem caused by the second point is that when the release is online, as long as two people release package files, conflicts are bound to occur.

• When publishing, you need to download the previous file and compare it with the merged new file to avoid making mistakes.

• Note that the fourth point is manual. I accidentally sent the wrong file, or overwrote the file just released by others. This happened more than 10 times.

• With only one test machine, it is common for the test environment to be frequently covered.

• Version control issues: the version is not based on SVN, but on the code on the pre-release machine, which makes management confusing

I can’t imagine what it would be like if a team of 10+ people developed together in this mode.

22.0 Era

The problems caused by the first version were really annoying. Every time we developed a version, it was a pain, especially in the testing and release stages. So we started to slowly solve them. As the business expanded and the number of people increased, the following diagram was born.

[[153124]]

Optimization measures:

• Adjust the modules to make the shared modules more shared and the business modules follow the business itself.

• Change the module merging strategy. Since it is too big, I will divide it into small ones, which alleviates the conflict to a certain extent.

• Replace the original file synchronization tool, including the test and formal environments, connect to ARS, and the test and release process is much smoother.

• ARS has a conflict detection function, which means you no longer need to manually check and merge, and overwriting is no longer likely to occur.

• Public JS files are cached in localstorage, simulating manifest with version control.

• Use SVN as the section control tool and no longer compare with external network codes.

Some statistics

• localstorage local cache

[[153125]]

localstorage cache hit rate

[[153126]]

First screen time

[[153127]]

window.onload time

[[153128]]

Everything seemed fine, but the good times didn't last long, because new problems arose.

1. Previously, the package.js file was split into multiple files, and the actual request was merged, including the JS file and the CSS file. The combo file actually had a delay problem, and there was a synchronization problem at the release time node, which directly caused the page to hang.

2. Aside from combo files, due to browser cache issues, you need to manually add a timestamp every time you update the version to avoid errors caused by the cache. This is also a very annoying point.

3. As the business grows, more and more pages are accessed in the APP. The touch screen page is no longer the focus, and how to better use the APP to accelerate the page is the focus. Many people think of the offline package of mobile QQ, but it is said that it is not particularly convenient in practice. We adopted the strategy of caching hash files on the client side to say goodbye to 304. So automation is involved here again.

4. Sprite images are basically handmade;

5. Code obfuscation is not compressed;

6. The CSS merged file needs to be manually addressed, similar to the following:

 http://at.qq.com/min/f=cssv4/common/reset.css,cssv4/common/base.css,cssv4/module/btns.css,cssv4/module/tab.css,cssv4/module/app-list.css,cssv4/module/talk-bar.css,cssv4/module/popup.css,cssv4/page/game-detail.css,cssv4/page/talk.css,cssv4/module/comments-bar.css

33.0 Era

In order to solve the above problems, the process needs to be further optimized, or simply put, the degree of automation needs to be increased.

3.1 Exploration Period

We also discussed the solution selection in the early stage. We did not have enough time to write it from the bottom up. We tried Grunt before and found it was not so easy to use. Later, we found that Baidu's front-end solution FIS could meet our needs.

1. Generate files named with hash values ​​(suffixes), change the code, generate new files, and automatically update the references in HTML (core demands), as shown below:

2. Merge Sprites

3. Compressed obfuscated files

4. File merging (including JS files and CSS files)

If we can do these things, our needs will basically be met. Everything is unknown in the early stage, and we don't know what big problems we will encounter. At first glance, it looks very easy to use. If it is a simple page, it will be very simple, and it can be done with just a few lines of configuration, but now the configuration file of FIS is 200+ lines. Some features are difficult to meet and require secondary development. It is easy to get started, but difficult to go deeper. You must read the source code, modify the source code, and write plug-ins. This is probably the experience of using FIS.

In the early stage, I thought about how to make the process of development-testing-pre-release-release run smoothly with tools. The general idea is as follows:

[[153129]]

Note:

1. Separate debug and release code from source code

2. For local debugging, use a proxy such as fiddler, or go to the development machine

3. deploy is a function of building tools to synchronize files

4. Ensure the source code version is the latest and release the code via ARS.

The engineering progress was not as smooth as expected. We encountered some problems in practice and had to grit our teeth and solve them.

3.2 Suffering period

Conflict issues have always existed, but they are not so obvious in the 2.0 era. The reason is that the JS in the test environment has been merged once.

Time issue: Since there were relatively few files at the beginning, the build speed was basically no problem. Later, as the business grew, more and more developers participated. The number of files increased sharply to 4000+, and the build time increased step by step. Development and debugging took 3987ms

[[153130]]

When publishing a build, it takes 181745ms because of md5 calculation, file compression, etc. It is unbearable.

ARS process Students who have used ARS must understand that the process is still quite painful. It requires completing a series of operations such as submitting to SVN, clicking synchronization, etc., which is cumbersome.

The build command is long, has many parameters, and is difficult to remember.

There are many output files, which makes publishing troublesome. Every time you change a letter, a new file will be generated when publishing. When publishing, you really have to look for it in the pile of files! !

This series of problems was overwhelming, and the developers in the group could no longer develop happily.

3.3 Deep Optimization

• The solution to further reduce the conflict problem is to subdivide the modules and not merge files in the test environment. In this way, the probability of conflict is almost small because the public library has been basically stable after the adjustment of 2.0.

• Shorten the time The build time is so long, it is not feasible to continue like this. I spent some time studying the FIS source code and found that FIS monitors the entire project file. Every build must scan all files, so the time will inevitably increase as the number of files increases. Then I made a deep transformation and became less like FIS. I optimized again and again and changed the build strategy. Dependency build : When a file depends on another file, the other file will be built. If a.html depends on b.js, only these two files will be built in the build output, and other files will not be built. The number of files is also reduced, and the time is greatly shortened.

Development build (in ms)

[[153131]]

Release build (in ms)

[[153132]]

[[153133]]

Build command optimization. It is troublesome to enter commands, so just use the GUI interface and click buttons. I would like to thank @koppthe@kolawang for writing a GUI with node in a short time.

[[153134]]

After the output files are multi -dependent, only relevant files will be output, the output files will be greatly reduced, and the difficulty of publishing will be greatly reduced.

When the ARS process is used to fix bugs, ARS synchronization is not required. Changes in monitored files can be synchronized directly to the test environment.

It only takes 8ms !!! If the synchronization button is turned on, the modified files will be uploaded to the test environment immediately, and there will be no problem of mutual overwriting. Everyone in the group is responsible for different modules, and the public modules are basically stable, so it is difficult for such problems to occur. In practice, such things rarely happen. On the contrary, my friends feel relieved. Compared with the cumbersome process of finding and transferring files, this is much easier.

The release optimization build tool will generate a file list. You can click it to open the folder and find the corresponding file. The list corresponds to the SVN path and can be directly pasted into ARS to submit the order.

When the sprite image is optimized and released, all referenced CSS files will be merged into one, and then the referenced icons will be merged into a sprite image, which is a bit rough. Because it is more reasonable to merge the images of the public CSS files into a sprite image separately, and the frequency of public images changing will not be so high. Develop a plug-in to create a sprite image once before CSS merging, and once after merging.

Statistics and Optimization

User network type (rough)

[[153135]]

Unknown is impossible to count, so theoretically, WiFi still accounts for the majority. From the other four types, WiFi accounts for the vast majority, and 2G users are very few.

PC VS Mobile

[[153136]]

In terms of JS download and execution efficiency, the mobile terminal is obviously lower than the PC terminal.

JS optimization

Previously, the JS files in the APP were downloaded through seajs. Later, I found that why not just write <script> to download directly? After optimization, the download execution time was significantly reduced:

Note: The time counted here includes the time to download JS and execute JS.

Comparison between JS embedded and external

When considering optimization, one of our solutions is to embed the external JS code (only business code) into the page through tools. Now let's compare the performance: http://gqq.gtimg.com/static/mobile/js/v3/page/gift/list /inappand.a9a524eb.lc.js file size is 8.7kb, 3.3kb after gzip compression. The inline loading time is almost 0 0.0015793, and the external download time is:

[[153137]]

[[153138]]

The download time is less than 200ms, but it is relatively long at only 8.7kb.

We know that the triggering of the DOMContentLoaded event basically means that the page has been rendered, JS has been executed (except asynchronously), and has reached an interactive state. For details, please refer to this article. Let's take a look at the impact of inline and external links on DOMContentLoaded:

[[153139]]

The blue line is the external JS, which takes significantly longer than the inline one, about 200ms. This shows that inlining a small amount of JS files into the page can increase the speed. However, there are no experiments to test whether large JS files are suitable for inlining. It is necessary to strike a balance between reducing HTTP requests and utilizing cache. In many cases, optimization criteria are not so easy to implement because they may contradict themselves or the project itself. Optimization is not done by following the rules, but requires flexibility.

The optimization measures were carried out according to Yahoo's 14 optimization guidelines, and all the unfinished tasks were addressed.

• Script domain switching, remove cookies.

• File Merger

• Use browser cache to infinitely increase the cache period max-age=15552000

• Sprite

• Reduce HTTP requests and build tools to embed JS into HTML

If the speed is still not fast enough, make full use of the local cache and the cache capabilities provided by the APP

• Browser cache scripts using localstorage

• APP cache hash file name script

• Caching HTML fragments

Debugging, testing, and experience process

Reverse proxy + whitelist control strategy, the domain name is 403 for external access, and can be accessed through the company's intranet. No proxy is required, and the mobile phone can directly connect to wifi to access. The environment is divided into development - testing - pre-release - official (each environment corresponds to an independent domain name), and any role (development, testing, design, product) can access it at any time.

The debug package of APP can switch the above four environments at will for debugging, testing and experience.

[[153140]]

Summarize

This is a summary of the work in the past year. I have never calmed down to sort it out. Looking back, I wonder if I have made this process more complicated. There may be a process from simple to complex and then to simple. I have been insisting on optimization without stopping. As long as it can be improved a little bit, I will try it. As the saying goes, life is endless, and struggle is endless.