Decoding the story behind Google's online empire

June 18th news: Three and a half years ago, a package was delivered to an office building in a small agricultural town in Iowa, and a magical story about Google began.

Inside the package were several strange computing devices, which were very wide and flat, like pizza boxes. One side of the box had dense openings, which were obviously used to connect a large number of network cables, and the other side was labeled "Pluto Switch". Their appearance was very different from regular network equipment, and there were many Finnish characters printed on the back of the box. The staff of this company obviously did not know the specific purpose of these boxes, so someone forwarded these boxes to their headquarters in Wisconsin, hoping that the IT experts there could answer the secrets of these boxes. The IT staff at the headquarters were also confused at first, but soon they got some answers on a very niche online forum: these boxes are network switches, they work under the complex and colorful surface of the Internet, responsible for transmitting and transferring a large amount of data between data centers, and their owner is Google.

Google does have a data center near the town in Iowa, so the mailman obviously sent the device to the wrong address. The company's staff packed the boxes in pairs and sent them to Google, and quickly disappeared from the forum, but the photos and technical discussions they had posted on the forum still opened a window for people to peek into the mysterious network world inside the Google empire. It points to a mysterious project that may change the entire Internet world, but Google has never talked about it in public.

[[137261]]

For many years, Google did not use the equipment of traditional network equipment vendors when building its own server network. They did not buy switches and routers from Cisco, HP or Ericsson on a large scale. In order to adapt to the rapidly expanding server network of Google, they specially designed their own network equipment and wrote their own software for them to configure and manage their own network. It was not until this accidental postman incident that the outside world saw the true appearance of these equipment for the first time. Due to the great significance of this project to Google, Google has been continuously increasing its investment in this project while keeping it secret from the outside world. Until this morning, at a technical conference in Silicon Valley, Amin Vahdat from Google, as the person in charge of this project, demonstrated the first five generations of hardware and software solutions that supported Google's network empire for 10 years for the first time, and introduced and discussed some basic technologies in this field. These discussions are forward-looking for the entire Internet world. After all, in the future, more Internet companies will take the path that Google has already taken.

[[137262]]

Amin Vahdat was originally a professor at UCSD. His research field was the expansion of the network. When he realized that it was no longer possible to build a network large enough in the academic world, he came to Google. According to Vahdat, the name of Google's secret project was "Firehose", which began in 2004. By 2005 and 2006, the products of this project had been deployed in several Google data centers and were running day and night. In addition to the products similar to the "high-performance top-of-rack" switches that the couriers delivered by mistake, they also developed another "cluster switch" (Cluster swtiches) for connecting large-scale networks to each other. Not only that, Google even designed and developed its own routing protocol and network management software. Vahdat explained: "The equipment at that time could not cope with the bandwidth and data throughput that our network needed to handle. Such equipment simply did not exist on this planet."

Talking about the initial motivation of the fire hose project, Vahdat said there were two main reasons. On the one hand, Google's server network expanded too fast, and the products of traditional network equipment vendors could no longer meet their needs for fast data exchange between the ever-expanding data centers. On the other hand, Google also needed to find a low-cost solution for this long-term challenge. Using traditional means to build a data center and network of the scale required by Google would cost an astronomical figure. The low-cost solution was to purchase a large number of low-priced civilian-grade chips and let them work together to complete the task of exchanging and processing massive amounts of data. It sounds very much like the prototype of the concept of cloud computing, which is currently being hyped.

When Google just started to build its own server network more than a decade ago, their choice was exactly the same as that of all other companies. They tried to buy the largest and most powerful cluster switches from Cisco. These cluster switches formed the backbone of the network inside each data center. Each such cluster switch cost Google hundreds of thousands to millions of dollars, and each such switch only had a fixed number of network interfaces that could be connected to other switches, which meant that building a data network that could meet Google's needs would cost endless dollars. "At that time, our procurement would only tell them, hey, buddy, give us the best switches you have, only the best."

So starting in 2004, Google set up a team to study its own network equipment solutions. In short, this solution is to use ordinary civilian-grade chips to build general hardware, and then run any software that needs to run on it. Cisco provides different hardware for different levels of needs, and each set of hardware runs specific matching software. Google's solution is much simpler and cruder. All the hardware is the same. When facing different needs, the difference in the solution lies only in the number of hardware that works together. A "high-performance top-of-rack" switch is equivalent to a board on a "cluster switch". The performance improvement is almost equivalent to the superposition of quantity, and these identical hardware can run any software that needs to run. According to Vahdat, the processing capacity of Google's "Jupiter" series cluster switches has now reached 40Tb per second, which is equivalent to the bandwidth and throughput of 40 million home users.

[[137263]]

The picture shows Google's "Jupiter" series cluster switches, which can process 40Tb of data per second, equivalent to the throughput of 40 million home users.

Compared to the hardware, Vahdat said little about the software part of their solution. But he revealed that their network equipment runs on Linux, and they have developed their own routing protocol "Firepath" to determine the best path for data exchange. In the network world, the industry standard routing protocols are BGP and OSPF, and developing your own routing protocol is very rare. Vahdat explained that when they started building the network, these standard protocols were not as simple and efficient as they are today, and they are going to build the world's largest and most complex network, so they need a simpler, faster and scalable solution. The details of the "Firepath" protocol are extremely complex. If you want to introduce its advantages, in a nutshell, "Firepath" users can use a central control software to uniformly configure the behavior of the entire network. In the traditional network world, if you want to achieve the same effect, network management experts must configure each switch in the network one by one.

Google's server network is probably the largest in the world. An amazing fact is that the amount of data exchanged between data centers within Google's network now exceeds the amount of data exchanged between Google and the entire Internet. Today, Microsoft, Amazon and Facebook are facing similar challenges as Google, and they are also carrying out similar projects. Facebook has made their SDN (software-defined networking) plans public from the beginning, and AT&T, the largest network service provider in the United States, has also been actively migrating to this new network technology.

The development of SDN technology has brought unprecedented challenges to traditional network equipment vendors with its huge advantages of low cost, high flexibility and almost unlimited scalability. This challenge is likely to be subversive, which means that the cake of traditional network equipment vendors has become smaller and smaller. When giants such as Google and Microsoft have abandoned Cisco, small and medium-sized companies will also want to turn to SDN because of its advantages. The reason why they cannot do so is that they do not have enough resources to design and develop their own hardware, software and solutions like those giant companies. So a group of new startups came into being, specializing in providing a new generation of network solutions for other companies. "When you only need to spend a little money to buy a bunch of cheap chips and use a central control software to configure the entire network, the expensive and complex traditional network solutions seem so clumsy and vulnerable." Now the booming development of companies such as Nicira (acquired by VMware), Big Switch Networks, and JR Rivers' Cumulus Networks has proved that the trend of SDN is unstoppable, and the network world is destined to usher in a revolution.