Google Cloud Tensor Processing Unit

2024: Google admits stealing AI technology after filing $1.67 billion lawsuit against it

On January 24, 2024, Google and Singular Computing entered into a settlement agreement with which they settled a multi-year trial related to an alleged infringement of rights to proprietary technology. The terms of the agreement were not disclosed, but the very fact of its achievement indicates that Google pleaded guilty to stealing the developments. Read more here.

2021: Fourth Generation Tensor Processor Announcement

On May 18, 2021, Google announced the fourth generation Tensor Processing Unit (TPU) tensor processors. The new solutions, according to the developers, are twice as fast as the previous model.

Introducing the new Tensor Processing Unit, Google CEO Sundar Pichai noted that these chips can be combined into blocks consisting of 4096 TPUs. The performance of one such unit can exceed 1 exaflops and correspond to the power of 10 million laptops with the most advanced technical characteristics.

This is the fastest system we have ever deployed at Google, and this is a historic milestone for us, "Pichai said. "You used to have to build your own supercomputer to get exaflops. We have already deployed many such systems, and soon dozens of TPU v4 modules will appear in our data centers, many of which will work about 90% without CO2 emissions.

Google unveils fourth-generation processor

According to the head of Google, the fourth generation of tensor processors will be available to cloud customers later in 2021. He did not give an exact date. It is also unknown when the new TPUs will go on sale for use on systems outside of Google. Previously, the company sold such processors in the form of modules.

TPU v4 modules or clusters are claimed to outperform previous generation TPUs in artificial intelligence and machine learning workloads, including object discovery, natural language processing, image classification, machine translation, and more.

As noted by TechCrunch, TPUs were one of Google's first custom chips. While competitors, including Microsoft, have decided to use more flexible FPGAs for their machine learning services, Google has relied on these custom chips beforehand. Their development takes a little longer, and they quickly become obsolete as technology changes, but can provide significantly better performance, the publication notes.^[1]

2018: Third generation: 100 petaflops of power and water cooling

As part of the Google I/O 2018 developer conference, the third generation of Tensor Processing Unit (TPU) tensor processors for machine learning was presented. The new solution, according to Google, is eight times more productive than its predecessor.

These chips are so powerful that for the first time we had to use liquid cooling in our data centers, "said Google CEO Sundar Pichai.

TPU 3.0 supports 100 petaflops processing. The TPU 2.0 version, introduced in 2017, had a rate of 180 teraflops. Each board connects to a PCI Express 3.0 x16 slot.

Google did not disclose detailed information about the new platform. In the image that the company showed during the conference, it was possible to see a board with four water blocks, that is, one board, as before, contains four tensor processors (ASIC), combined into one circuit. The main part of the processor is a matrix that performs multiply-accumulate operations.

Tensor Processing Unit processors eight times faster than Google TPU 2.0

It is also known that TPU 3.0 has twice the high-speed memory (128 GB) relative to the second generation, so the equipment has become better suited for large machine learning models working with gigantic data arrays.

Up to 64 TPU 3.0 modules can be installed in a single server rack, which is twice the size of its predecessor. It would be difficult to cope with the heat removal of such a system without water cooling.

Google customers will be able to use the power of TPU 3.0 through the Google Cloud, but the timing of the public availability of the novelty has not been announced. TPU 2.0 was launched as a service in February 2018, users can only rent one module at a time.^[2]

2017: First Generation Announcement

On May 17, 2017, Google announced the Cloud Tensor Processing Unit (TPU) supercomputer processor. The machine learning solution will be used not only by the company itself, but also by its customers who can access the new product through the Google Cloud.

Cloud TPU will not be sold directly to server manufacturers. They will be able to take advantage of processor performance by connecting to Google's cloud service to run their workloads and store data on the American company's hardware.

Cloud Tensor Processing Unit

Cloud TPU is the second generation of chips being developed by Google. The first version was presented in 2016. Then it was reported that TPU is a special purpose integrated circuit (ASIC) designed to speed up the processes of obtaining ready-made results in already trained neural networks. For example, the system comes into operation when the user initiates a voice search, requests a text translation, or searches for a match with the image.

Cloud TPU has become faster than its predecessor, with performance rising to 180 teraflops in floating point operations. Each processor contains a special high-speed network that allows you to create supercomputers, which Google calls the TPU Pod.

One such module contains 64 new processors and has a performance of 11.5 petaflops. This speed significantly reduces the training time of artificial intelligence systems. So, one of Google's supercomputers underwent one training procedure per day, using the 32 fastest GPUs on the market, while only an eighth of the TPU Pod allows you to carry out similar operations only from morning to evening.

One module, consisting of four Cloud TPU processors, is about 12,000 faster than the Deep Blue supercomputer, which gained fame after winning chess over world champion Garry Kasparov in 1997, said Urs Hölzle, senior vice president of technical infrastructure at Google.

Google did not disclose the timing, the cost of renting new processors, or the name of the manufacturer of these solutions. The company, which continues to buy chips from Intel and Nvidia, can save billions of dollars annually by using its own solutions, Bloomberg notes.

This market is developing rapidly. It is very important for us to promote machine learning for our own purposes and be the best in the cloud, "said Urs Hölzle.[3]

Notes