Google's fourth-generation TPU performance test is here! This year, it will provide services to Google Cloud users

This article is reprinted with permission from AI new media Quantum Bit (public account ID: QbitAI). Please contact the source for reprinting.

A TPU v4 pod can achieve 1 exaflop of computing power and perform 10 to the 18th power floating-point operations per second.

The Google I/O conference, which was absent for a year, really lived up to expectations.

In addition to Starline's 3D video call technology, which even Google AI chief Jeff Dean called a "magic mirror," the fourth-generation TPU is also attracting much attention.

Google introduced that TPU v4 will be mainly used in the form of pods. A pod consists of 4096 TPU v4 single chips and can achieve a computing power of 1 exaflop, which is equivalent to the sum of 10 million laptops.

Compared with the previous generation TPU v3, the performance of TPU v4 is improved by an average of 2.7 times at the scale of 64 chips.

In addition, the performance of TPU v4 pod is 10 times higher than that of TPU v3 pod. It will run mainly on carbon-free energy, which is not only faster but also more energy-efficient.

Google CEO Sundar Pichai revealed that TPU v4 pod will be used in Google's data centers and will provide services to Google Cloud users this year.

BERT training in two minutes

Although it has just been officially released, Google revealed the performance of TPU v4 as early as a year ago.

In the MLPerf training v0.7 list, an authoritative benchmark for artificial intelligence released in July last year, we can see the performance comparison between TPU v4 and various chips.

In the MLPerf training test, its benchmark includes 8 machine learning tasks such as image classification, translation, recommendation system and Go. The final result is the training time of these 8 tasks. The faster the speed, the better the performance.

The specific contents of the 8 tasks are as follows:

The specific training models are: ResNet-50, SSD, Mask R-CNN, BERT, NMT, Transformer, DLRM and Mini Go.

The performance of TPU v4 is as follows. Each system is distinguished by the number of TPU v4 accelerators: 8, 64, and 256.

From the comparison we can see:

In ResNet training, 256 TPU v4s shortened the training time to 1.82 minutes.

However, if Nvidia A100A100-SXM4-40GB wants to reach this level, it needs at least 768 acceleration blocks .

In BERT training, 256 TPU v4s also shortened the training time to 1.82 minutes;

The same 256 Nvidia A100-SXM4-40GB can only shorten the training time to 3.36 minutes.

And according to the published data, a TPU v3 pod consisting of 4096 third-generation TPUs can compress BERT training to only 23 seconds !

About TPU

Simply put, TPU is a chip developed by Google that can accelerate machine learning.

Unlike GPU, TPU is an ASIC chip, which stands for Application-Specific Integrated Circuit, a chip customized for a specific application requirement.

Why develop TPU?

In fact, this is because many of Google’s own products and services, such as Google Image Search and Google Translate, require the use of deep learning neural networks.

This places higher demands on computing power, which is difficult to maintain with ordinary GPUs and CPUs.

Therefore, TPU came into being.

The first generation of TPU was used in the famous AlphaGo. When it played against Lee Se-young in 2015, 48 TPUs were deployed.

The second-generation TPU was introduced to Google Cloud and used in Google Compute Engine (GCE), also known as Cloud TPU.

AlphaGo, equipped with TPU v2, defeated the then world Go champion Ke Jie using only 4 TPUv2s.

In 2018, Google released the third-generation TPU, which doubled the performance of the second generation.

The performance of each Pod has increased by 8 times, and each Pod can contain up to 1024 chips.

The fourth-generation TPU will not be officially released until 2021.