Exploring the front-end and back-end separation architecture

About five years ago, I was still a little lecturer (Apple AATC training certification), a rookie who didn't know anything about programming. By chance, I joined the company's development department. What position did I take? User Experience Designer. The reason was very stupid - I had worked in advertising design and catering service industry before, so I had two advantages: I could listen to and understand the needs of customers, and then I could make some pictures.

At that time, many small and medium-sized IT companies like ours (with about 200 people, mainly composed of large and small project teams) had the desire to develop their own products. This was determined by the market: it was becoming increasingly difficult to find business. So many unconventional product development teams were born...

They are called product development teams, but in fact they are just a group of programmers who are used to following orders and implementing functions according to RFP. The only difference between them and other project teams is that they are just "dreamy salted fish". Therefore, you can easily guess the childishness and shittyness in the development process.

At the beginning, I organized the bosses' ideas into requirements that everyone could understand, then strung them together into sketches (mockups) and handed them over to various engineers to implement. This role is similar to the fashionable "product manager" nowadays. However, I soon found that everyone was always working overtime. Why? Adjusting CSS styles!

As someone who is used to doing graphic design, I didn’t know how much trouble it would be to turn something I drew into something in the browser. (Today, when I was interviewing some page cutters, I found it funny to hear them talk about pixel-level restoration, but thinking about myself five years ago, I still feel a little ashamed... But what makes me even more speechless is that even now, the fledgling front-end developers still regard pixel-level restoration of page cutters as the supreme skill. Isn’t this thing itself very "frustrating"?) When my colleagues who wrote the pages repeatedly told me that what I drew was not practical, I couldn’t hold it back - I just didn’t believe that what I drew couldn’t be realized!

With a "grudge", I resolutely embarked on the road of HTML + CSS. There is no need to talk about the process. The only good advice is: don't look at domestic tutorials, don't believe in patchwork resource sites like w3school. Anyway, the result of this matter is that half a year later, almost all the pages of the entire project team were written by me. (Today, I have become a front-end architect, and I still have to write the custom styles of all pages myself. I don't blame anyone because I know that many engineers still look down on the technology of writing HTML + CSS in their hearts. If you don't want to learn, I won't force you, I will do it. I also want to thank you. In order to feed you, I spent a lot of time learning the development of CSS frameworks, so I mastered the entire ecological chain, from pre-processing to post-processing).

The world is like this: once you master a skill, it's easy to see how well related skills are doing at the current level. As the old saying goes: rising water lifts all boats. It's true.

So I, who was exposed to HTML+CSS all day long, and listened to the videos and tutorials of various foreign masters, soon understood one thing: what I was doing was called front-end development, and HTML+CSS was just the beginning. There was a mountain called JavaScript waiting for me. And the front-end development we were doing was still very naive, so you deserved to work overtime to modify the style and fix bugs all day long, because you didn't take the right path from the beginning.

Fortunately, it’s not too late, but the theme of this article is separated into front and back, so I have to press the fast-forward button to get straight to the point.

JavaScript was too difficult for me, but jQuery was OK because it had a great API design and compatibility, which was very suitable for a beginner like me. At that time, a guy named Jeffery Way recorded a 30-day tutorial to learn jQuery, which benefited me a lot. I think his tutorial was good for two main reasons:

He didn't mainly talk about how to use various APIs. From the beginning, he taught me an important idea: learn to read the documentation;

He mainly talks about how to analyze the implementation of a function and how to organize the code logic with jQuery as the core;

Let me first insert a paragraph here. All of you who can use Rails at work are extremely lucky! Because Rails has sorted out the View-Template link simply enough, even if you don't know anything about Rails, you can quickly learn how to put static pages into Rails templates with a little guidance.

I was very unlucky, because I encountered the very annoying JSP at that time! Don't laugh, just ask those front-end developers who came from page writing background how they feel about JSP, they will definitely not be in a good mood. Yes, I admit that I am a novice. I can write static pages, but converting them into JSP templates really made me die at that time! What's worse is that if you give the written page to the back-end engineer to apply the template, the final result will be a mess! That's right, they will not take good care of every tag you carefully designed. They will do all kinds of weird things to destroy the original perfect page structure, forcing you to constantly modify the style and scripts to adapt to these "patches".

What's even more difficult is debugging! Originally, you could write HTML+CSS with a lightweight editor, but after they are converted to JSP, it's not that easy to debug. You need:

Running environment, such as Java + Tomcat, don't understand? No problem, learn!

Ecosystem, such as Maven or Gradel, don't understand? No problem, learn!

IDE, such as eclipse or IDEA Intellij, don't you understand? No problem, learn it! By the way, do you know how difficult it is for someone who has never touched Java to run an application? That's why I fell in love with Rails!

…

Just like that, in order to debug HTML+CSS, you end up becoming a Java development engineer who knows everything except writing Java code.

Can you guys who come from the backend understand how much courage and perseverance it takes for the front-end web developers to take this step?

Can you imagine that they have to learn to do this just because you can't take HTML+CSS+JavaScript seriously?

Why do you do front-end work in a back-end environment? Actually, it’s just for three words: clean up your ass!

You can say that we are all inexperienced, and I admit that, but you have to know that the environment does not always give you all kinds of choices. Sometimes the only choice you can make is to change yourself. So what can I do as someone who only knows HTML + CSS + a little JavaScript? I don't know when it started, "It would be great if we could do our part without relying on any environment." This naive thought began to linger in my mind...

Back to Jeffery's video tutorial, in one section he demonstrated an example of using Ajax to obtain remote data and then dynamically modify the DOM. The example at that time used the Twitter API, and then pulled a few new data at regular intervals to refresh the page instantly...

Don't laugh, you know how shocked I was? I thought we were just a bunch of fucking idiots, okay?

The next day, I showed this video to the backend architect and asked him if it was feasible to implement something like this. He paused for a long time: We directly render data to JSP, and have never done API...

Shit! Can’t I do it if I haven’t done it before? I quickly threw out the bait: If you can do it, you won’t have to use templates anymore!

This guy agreed immediately...

After that, it was a lot of hard work. I searched through all the materials I could find, translated them into Chinese or explained them directly to the backend in person. There were some things we couldn’t understand, so I wrote them down first. At night, I went to SO to ask questions and searched for conference materials on Youtube.

Then they told me that it might be easier to use Spring, because they could find examples of using Spring to develop APIs on Baidu. So we started the transformation.

The first step of the transformation is to not write JSP (or write a small amount), but the static resources are still placed in the Tomcat container, because we found that the cross-domain problem could not be solved after trying (yes, I was a rookie at that time and didn’t even understand reverse proxy), but it doesn’t matter, anyway, I have learned to run Tomcat locally, at least we don’t have to write JSP. Now think back to the original motivation of separating the front and back ends, it turned out to be to stop writing JSP, how funny! However, thinking about it the other way around, this also reflects a fact: how bad the ecological environment of front-end engineers is!

Then came the boring process of practicing jQuery to the maximum level and starting mindlessly brushing dungeons. Of course, I also experienced some new things in the process, such as front-end template engines (Jade/Handlebars/Art, etc.), module systems (SeaJS/RequireJS), etc. My JavaScript level and understanding have made great progress, and I finally started to look like an engineer.

Then, as we all know, node.js came out of nowhere. All of a sudden, HTTP Server, API Service, Shell Scripting, etc. can all be done with JavaScript. npm bower grunt gulp, etc. came into being. Suddenly, the front-end started to have its own ecosystem! Although it is still very weak and chaotic, it gave us, the guys who came from the wild and struggled, a ray of hope. It let us see:

We can not rely on the backend operating environment: node.js

We can have our own ecosystem: npm

We can use any convenient development tools as we like: so I became a vim user

…

We have many possibilities. We can do what we are good at even better without the backend brothers having to worry about it. We can save a lot of work that the backend needs to cover and let them concentrate on writing their own code. The separation we envision is feasible. It is not just separation for the sake of separation, but separation for better specialization, versatility, collaboration, and management!

Why is there such a narrow view of the separation between the past and the present? To this day, I still don’t understand why so many people hold such doubts and prejudices.

You don't have to worry about redundancy in the data layer logic, because distributing the Model logic to the front end can save some of the back end code and processing work, and the front end can also more easily combine the Model it needs according to the business
You don't have to worry about the cache of the view layer, because after the separation, only static resources exist in the front end. We can use CDN, load balancing, and many other technologies to share the work that the back end had to bear in the past. You don't have to worry about the page rendering speed. If the home page is too slow, we can let the server render it, or add a very simple node server in the middle to do static rendering of the home page, and leave the rest to the front end. It will only be fast, not slow. You don't have to worry about scheduling different resources for multiple clients. As long as the API is planned, a set of Services can support the business systems of multiple clients, and the front end has enough capacity to even write multiple versions for A/B testing. You don't have to worry...

There are so many advantages.

It’s not that these advantages are mature at present, nor that there is no cost to achieve them, but if you don’t do it, they will never mature. It’s not that there can be a cost, but the key is to look at the long-term benefits.

A very realistic example is that we had a system originally designed for ourselves, but later when a client found out about it, they were very interested and wanted to customize it for them. We analyzed it and found that the existing API could already meet the needs of users. We only needed to expand a few interfaces for a few specific business logics to make the data load more reasonable. So we only took three days to provide the client with a fully usable and quite stable demo. The client was satisfied and began to redesign a set of UI according to their VI. Then the rest was to find a few pages and write them, and then make a few minor changes to the existing Angular logic.

Do you think it's not worth it?

Of course, I am not advocating the belief of front-end separation, nor am I forcing everything to be done separately. The trajectory of a product needs to be controlled by the product development team itself. If you just follow the crowd and use whatever is popular, it will be no different from being brainless. Some scenarios do not really need to be separated. For example, the needs of portal websites, CMS, and Mini Site can continue to use mature development systems. However, as I mentioned before, there is another important benefit of exploring and practicing the separation system, which is that it allows your existing front-end development team to explore and organize a set of single-soldier combat environment systems. Even if I don’t use a separation architecture, I simply use node.js to write a portal website, CMS, Mini Site, etc. It won’t be slower than Rails! In this way, I can still use the back-end resources for more important underlying services or business logic, and let the front-end independently complete those businesses that are related to the page UI interaction but have a small connection with the data layer. What’s wrong with that?
Front-end and back-end separation = SPA? SPA = bloated framework?

I think it is necessary to analyze this point clearly. The two question marks in the title are the most common misunderstanding I have seen.

First of all, the separation of front-end and back-end is an architectural issue. It is definitely painful to do it for the first time, but there are still many benefits after doing it once. Here are some examples:

We have made a conference application. The application was originally designed without a web frontend, only an admin backend, and the frontend was all mobile. For this reason, we still separated them (because you have to provide APIs to mobile terminals, how can you do it without separation?), and the backend was quickly completed using mature Angular.

Unexpectedly, there was an additional requirement that users should generate an online conference manual when creating a meeting. This conference manual is a simple multi-page CMS system. When users create a new meeting, they fill in the manual-related content in the background. We need to generate a series of pages for it to display (similar to Mini Site). It has two specific requirements:

No login required, public access. However, the original design was that you could not attend the meeting without an account, and you needed to register, so our backend and mobile app directly require login or registration first, and the corresponding API requests are also the same, with authentication control.

It needs to be accessible from multiple terminals and can be embedded in the webview of the native application. However, there was only one day to add the new features because it was too late.

In the traditional architecture, you write pages and then use templates to debug. Although there are less than 10 pages, it is still very time-consuming. But we have separated them, and it is not worth going back for such an additional requirement. So how to do it?

Create a separate project for conference manual;

The template is quickly created using Sass + Handlebars;

The interface requests a token for each conference service provider (later, the administrator was allowed to regenerate and bind the token in the background), and the token is hard-coded in the <meta> tag when rendering the page (just like CSRF processing) to bypass authentication.

Write a simple node service that does one thing: render handlebars templates

When creating a meeting, the Java API passes the meeting ID to the node service, saves the rendered page separately in the static resource server (creates a separate directory with the ID), and then returns the calling address

The background receives the meeting address and embeds an iframe for manual preview

Finished in one day. It is worth mentioning that the whole manual uses many new features of HTML5, such as History API, SessionStorage, OfflineCache, GeoLocation, DesktopNotification, without Polyfill, because these are optional features, and they will not work if they are not supported. The key point is: jQuery is not used from beginning to end - it is not that I have a problem with jQuery, it is really unnecessary and too big. Not to mention the SPA framework, there is no SPA framework at all.

So you see, the separation architecture allows us to complete such small tasks quickly, and can be maintained and managed separately, or we can directly share existing API resources. It is not necessarily separated just for SPA, and there is no technical difficulty. We can complete it in a short time and ensure quality because we have mature construction and CI. If we change to the original JSP system, just configuring a local environment is enough, and I dare not imagine the rest.

Separation is an architectural choice that determines how you manage, allocate and coordinate existing resources. Whether you want to build a SPA or other application model after separation is completely your freedom, not a mandatory decision to bundle one plus one. Building a separation system will certainly have setbacks and costs, no one denies this, but 1) it is optional; 2) whether you can see and use its benefits.

As for whether SPA must be bloated? I can only say that you have a narrow vision if you hold this thought. Compared with web development ten years ago, can I say that Rails is bloated now? Not to mention ten years ago, even today, some people still say that Rails is too heavy! What do you think? So what? Shouldn't you use it when you need it? People with high skills naturally know how to split and reduce weight. Even Rails itself knows how to slim down a Rails API. Do you think the so-called "bloated" is the patent of SPA framework? The reason why SPA is bloated is determined by two main environmental factors at this stage:

Many new features emerge in an endless stream, providing us with weapons and ammunition to develop richer and more powerful applications. However, the fragmentation of browsers (and other operating environments) and devices means that these new features cannot provide consistent performance or performance, so various frameworks have to make compatibility supplements and improvements at the bottom level, and by the way, they have to repackage API interfaces for new features that have not yet formed standards. For example, why did Ember create an Object interface? Isn't it because the Observable interface doesn't exist? What's the big deal? ES2016 has it (very likely), or you can use a third-party Observable component instead of Ember's own.
How different is what jQuery does from this? Yes, jQuery is relatively light, but it also covers less areas. Who uses jQuery without having to use 10 or 8 plugins? If they are combined, it will be light?

In contrast, there is a huge gap in the overall level of front-end engineering in the industry. The best are very good, while the novices are very mediocre. However, the novices also want to use the best tools, but they don't have the foundation to solve the problems that the best can solve. So the best encapsulates all the features and connects them together to allow you to use them as quickly and easily as possible.
If most engineers have grown up, there is no need to have a big and comprehensive solution. Isn't React and its ecosystem a good example? Instead of a big and comprehensive solution, you can only have a small and specialized one. Do you think that if you connect all those things together, it is not a SPA? Naive!

Let me say it again, SPA is a technical form of a product, not a product under a specific framework (or frameworks). The tool chain that meets this technical form can be bloated or simple, which is determined by the environment and people.

Single Page Application, not Some Particular Application

There is another benefit of front-end separation. Front-end engineers all have a common characteristic: they can write a page easily, but if you ask them to be responsible for a complete business, they will most likely be at a loss. Why? Because they are too unconventional. In the past one or two years, I have started to interview and reserve a lot of new people, and nine out of ten of them are like this: HTTP? I don’t understand! Ajax? I understand! (Do you think it makes sense?) jQuery request API? Yes! Have you used Promise? ... No. In other words, deferred object? Oh, I have seen it! (Do you think it makes sense?)

Such problems are common, which makes me worried about the future of the front-end industry. I started as a rookie who knew nothing. If I didn't have the naive ideal of "getting rid of JSP" at that time, how could I have gained such a comprehensive knowledge and understanding today by exploring the separation of front-end and back-end? The road I have traveled has made me understand that exploring the separation of front-end and back-end is not "separation for the sake of separation" as many bystanders say, but "separation for a better understanding of web development".

Because when you start exploring this path, you have to face many fundamental problems. Take cross-domain resource sharing for example. In the past, front-end engineers rarely had to face this problem, but you will inevitably encounter it as soon as you separate. Then you have to learn technical details such as JSONP, CORS, HTTP protocol, browser security mechanism, PreFlight Request, reverse proxy, etc. It seems to increase the learning cost (I think these should be the responsibility of the school!), but as a colleague, do you want someone who only "pursues pixel-level restoration" to work with you? Or do you want an engineer who has a solid understanding and practical experience of the above knowledge points?

Speaking of this, someone asked a question on SF yesterday, roughly asking: How can you be considered to have learned JavaScript well? I always feel that you need to be able to write a library or framework to be considered to have learned it well. What do you think?

I just had an argument with someone, and after calming down and thinking about it, I gave the following answer:

The boy practiced martial arts for 10 years and wanted to go down the mountain to make a name for himself. On the way, he met a man with a knife. After a few moves, he was defeated and returned home... After returning to the mountain, he went to ask his master

"Master, why do I lose after ten years of hard training?"

"Because you don't know that there's more to fighting than just fists."

"But you didn't tell me!"

"You only said you wanted to learn boxing, but you didn't say you wanted to learn how to fight!"

"Then I don't want to learn boxing, I want to learn fighting!"

"Then you have to learn more than just boxing. If you want to win a fight, you have to learn all eighteen martial arts. You don't have to be proficient in all of them, but you have to at least have the knowledge. In addition, you have to learn how to take a beating, how to heal, how to escape, how to track, how to use hidden weapons, how to use poison... Want to win? How can it be that easy!"

"Then can I still become a boxing master?"

"Haha, if you never lose a fight, who dares to say you are not a master?"

The meaning of this fable is self-evident. I agree with what a friend said here: we should not have a distinction between the front-end and the back-end. We can have our specialties, but we should know everything we should know about web development, otherwise how can we win? Similarly, if the back-end engineers need to understand the front-end by writing pages, then the front-end should also have a similar way to understand some of the things done by the back-end. Exploring the separation of front and back here is a good way to combine teaching with practice. No page boy is willing to cut pictures and write pages forever. They also envy the god-like coquettishness of the back-end brothers, but the environment they are in has caused them to only know how to cut pages for decades. If they can be given more support and help, who dares to say that they will not become masters in the future?

Many people use excuses such as being busy at work, lack of staff, and startups seeking efficiency to avoid exploring and advancing in the technology field. To be honest, I can understand this very well. What I did at the beginning was not much different from starting a business. We were also short of staff. Today, we only have three people maintaining four medium-sized and large-scale products with front-end and back-end separated architectures. These products have SaaS versions, and there are more than a dozen large and small products that are independently deployed at customers' locations. You read that right, only three people! A Java engineer, a front-end engineer who knows Java, and me, a jack of all trades who knows a little bit about everything but specializes in nothing.

We haven't done well enough, but we have tried our best to do the best we can. Compared with the ups and downs we have encountered in the past five years, exploring the separation of before and after is really not a big deal, okay?

As a front-end engineer (and someone who understands and respects back-end development), I am glad to be active in this era. As someone said: This is the best era for front-end developers, but also the worst era for front-end developers. However, history has proven countless times that true gold is not afraid of fire, and heroes emerge at the right time. Haven't those back-end language environments and framework systems experienced the same innovation and changes? Just because we used to be page developers who only knew how to write jQuery, should we stay stagnant forever?

This is my process and experience of exploring the separation of front and back, mainly to make a summary of the past five years before leaving. It is a bit messy and has no technical content. The fundamental meaning is to encourage many front-end colleagues: the school does not have our professional courses, and the society does not have an accurate understanding and evaluation of our work, which is not a big deal! The important thing is that we ourselves cannot underestimate our abilities and cannot give up our own value. Be brave to explore when you still have spare energy in study and work. Don't care what others say. Learning skills is the most important thing. Remember: you are an engineer, not a page boy!