NVIDIA: from GPU development to a comprehensive AI infrastructure for data centers. Main announcements of GTC 2021

Jensen Huang presented the updated NVIDIA product portfolio at GTC. You can download the presentation from the performance by gallery-1 link

Native CPU for AI

One of the key announcements of the conference is the first native CPU processor from NVIDIA, which is expected to be widely available in 2023. It was named Grace in honor of Grace Hopper, Rear Admiral of the US Navy, programmer, creator of the world's first compiler of the programming language.

CPU Grace combines energy-efficient ARM cores with an innovative memory subsystem to deliver high performance with low power consumption that is critical for building next-generation super-performance systems.

Grace is created for use in data centers. First of all, it is designed for the most complex calculations, including natural language processing, the creation of recommendation systems and the construction of supercomputer centers for AI. All these tasks are related to the analysis of huge data arrays that require ultra-fast calculations and a large amount of memory.

For example, Grace is suitable for training next-generation models in natural speech recognition, which contain more than 1 trillion parameters.

The company also noted that the Grace processor was built for use in large systems designed to work with a new type of software - data-driven.

We are creating Grace to work with a new kind of applications related to understanding natural speech, AI models, recommendation systems that handle hundreds of petabytes of data, as well as for data-driven scientific calculations, "says NVIDIA CEO. - There are more and more such applications, this is a new segment. We expect to see a big leap here. Thus, Grace will be very useful for the industry.

NVIDIA adds that the Grace architecture is "completely unique," not unlike anything previously created. The new processor solves a problem that previously did not exist.

It is logical to imagine that CPU processors and system architectures designed 20 years ago are not intended for this new application environment. And we will continue to focus on the fact that previously did not exist, on solving new problems, "said the head of NVIDIA.

According to the developers, to solve the "heavy" problems of the Grace-based system in conjunction with the NVIDIA GPU platforms, they are able to provide 10 times higher performance than the latest NVIDIA DGX servers and workstations using CPU with the x86 architecture.

Systems based on Grace processors will use the fourth generation of NVIDIA NVLink bus, providing throughput at the level of 900 GB/s between the Grace processor and NVIDIA graphics accelerators. This is about 30 times higher than the current most advanced servers, say NVIDIA.

ARM is the most popular CPU in the world. Now they are widely used in mobile and embedded segments, and are just starting to grow in areas such as cloud computing, enterprise-level solutions and data centers. NVIDIA can accelerate the implementation of ARM architecture processors in the markets where it operates, according to the company.

Jensen Huang made it clear that with the development of its own CPU, NVIDIA does not seek widespread competition in the CPU segment with Intel and AMD and plans to focus on the niche for which the Grace processor was originally designed. He also recalled that the company has "excellent partnerships" with Intel and AMD, with which it closely cooperates in various areas - from PCs to data centers and supercomputer computing.

But here is a remarkable fact that during the day after the announcements of NVIDIA, its shares rose in price by 4%, and Intel and AMD shares fell in price by about 4% to^[1].

I must say that the development of NVIDIA own CPU processor based on ARM is undoubtedly an important event, but hardly absolutely unexpected. In 2020, the company announced its intentions to acquire an ARM developer for $40 billion. As of April 2021, NVIDIA is awaiting regulatory approval for this transaction.

Grace's first users are already pushing the boundaries of science and artificial intelligence: in 2023, the Swiss National Supercomputing Center (CSCS) and the Los Alamos National Laboratory plan to commission Grace supercomputers built by Hewlett Packard Enterprise.

The Swiss National Supercomputing Center, Hewlett Packard Enterprise and NVIDIA have announced the world's most powerful artificial intelligence-enabled supercomputer: the Alps system for advanced research in climate, physics and natural sciences with a 7-fold expansion of AI capabilities compared to the current world-leading AI system on MLPerf.

Alps will replace the existing Piz Daint supercomputer in CSCS and will serve as a universal system open to a wide community of researchers in Switzerland and around the world. Learn more at the link.

"Three-chip" road map

The Grace processor was the third chip in the NVIDIA line of data center solutions in addition to the GPU and DPU processors (Data Processing Unit, coprocessor for data processing). NVIDIA introduced the NVIDIA processor roadmap for data centers until 2025, where all three products are present.

Our roadmap in the field of data centers is now a rhythm consisting of three chips: CPU, GPU and DPU, "said the general director of NVIDIA. - Each of these architectures has a two-year cycle with intermediate releases. One year we will focus on x86 platforms, the other on ARM platforms. Thus, every year we will present the market with wonderful products.

As for DPU, in 2020, NVIDIA introduced the first such solutions in its portfolio - BlueField-2 and BlueField-2X, network cards with a fast ARM processor, which takes over the computing load, freeing up the resources of the server's central processor for other tasks. Initially, BlueField is the brand of Mellanox, which previously became part of NVIDIA.

And at the opening of the GTC conference, NVIDIA announced DPU BlueField-3, which should be available in 2022. According to the developer, BlueField-3 will be the first network card with a transfer rate of 400 Gb/s to the port over Ethernet and NDR InfiniBand. In addition, 16 ARM Cortex cores will be available, which will accelerate network computing, thanks to special hardware solutions. NVIDIA states that such a chip replaces about 300 traditional CPU cores. The chip is connected via PCI Express 5.0 and has its own DDR5 memory.

An updated NVIDIA DOCA - DOCA 1.0 Developer Toolkit (SDK) has also been announced, which is designed to create high-performance software-defined and optimized for DPU accelerated cloud services using standard APIs. DOCA in the NVIDIA product line can be called an analogue of CUDA for GPU platforms.

Not without updates in the NVIDIA GPU line. The company introduced two new, "junior" versions of its GPUs - the A30 and A10, as well as the A16 model. The first is for faster computing, AI and data analytics, the A10 is for AI element graphics, virtual workstations, and mixed computing and graphics workloads. And NVIDIA A16 is primarily focused on VDI infrastructure. All three models are based on the NVIDIA Ampere architecture.

Democratization of AI

NVIDIA's best practices for AI and HPC are embodied in its flagship DGX computing systems. They are presented in three versions. The first is the DGX Station, which is a personal supercomputer that can be connected directly to the desktop. The second is the DGX A100 to create a data center "out of the box." And the third solution - DGX SuperPOD - is a reference architecture for creating a full-fledged infrastructure for AI.

As part of GTC, NVIDIA announced a new version of DGX SuperPOD. Among the key changes - this product will include the DPU BlueField, due to which, according to the developers, the company for the first time in the world presents a cloud native supercomputer for AI.

You can access this supercomputer from the cloud in the same way as with any other computing in the cloud, you can share it with other people. And at the same time, all your data is safe, "said Jensen Huang.

NVIDIA also announced a new version of the DGX Station - DGX Station 320G, which is based on the Ampere GPU, which contains technologies that allow dividing the GPU into subsections, and has more memory compared to the previous model. Up to 28 data processors and analysts can use the same station at the same time.

From the presentation on GTC it follows that each such station provides performance up to 2.5 Pflops. True, the presentation does not specify whether performance on double or single precision operations is meant.

The company says that a cluster based on CPU with similar performance today would cost about $1 million, and DGX Station costs $149 thousand.

But this is not all. The company introduced a new class of NVIDIA-certified systems (Nvidia-Certified Systems) using the new GPU A30 and A10 to make AI available to organizations that launch their AI applications in the standard enterprise data center infrastructure. Such systems include mass models of enterprise servers, mainly in 1U and 2U form factors, from leading manufacturers, which were announced in January and are now certified to work with the NVIDIA AI Enterprise software package.

Among the manufacturers whose NVIDIA-certified servers support the NVIDIA EGX platform, which allows you to work with AI applications on the infrastructure used for traditional business applications, the following companies: Atos, Dell Technologies, GIGABYTE, H3C, Inspur, Lenovo, QCT and SupCT.

And Lockheed Martin, an American military-industrial corporation, and Mass General Brigham, a Boston-based network of nonprofit hospitals and doctors, are among the first to implement these systems in their data centers.

A few days after the announcement of the GPU A30 and A10, NVIDIA also announced that its AI investment platform, which was recently replenished with these processors, showed record performance in all categories in the latest version of MLPerf.

MLPerf is an industry-recognized benchmark for measuring AI performance in a variety of tasks, including computer vision, medical imaging, recommendation systems, speech recognition, and natural language processing.

NVIDIA achieved these results with the benefits of the NVIDIA AI platform, which covers a wide range of graphics processors and AI software, including TensorRT and NVIDIA Triton Infection Server, which are adopted by leading companies such as Microsoft, Pinterest, Postmates, USPS and WeChat.

Some results also include Triton Inference Server data, which simplifies AI deployment in applications by supporting models from all major frameworks running on GPU and CPU, and optimizing for various types of requests, including batch data, real-time data, and streaming. Triton's results show performance close to the most optimized GPU systems, as well as CPU systems with comparable configurations.

The company also opened up new opportunities by presenting the results obtained using Multi-Instance GPU technology of the NVIDIA Ampere architecture, while running all seven MLPerf Offline tests on one graphics processor using seven MIG instances. The configuration showed almost identical performance compared to one instance of MIG.

In addition to NVIDIA's own results, Alibaba, DellEMC, Fujitsu, Gigabyte, HPE, Inspur, Lenovo and Supermicro partners presented a total of more than 360 results using NVIDIA graphics processors.

In turn, the above-mentioned NVIDIA AI Enterprise software package is now certified for the world's most popular computing virtualization platform, VMware vSphere 7. The company also reported this as part of the GTC.

NVIDIA AI libraries accelerate each step, starting with data processing. NVIDIA AI can be integrated into all industries, popular tools and manufacturing processes. NVIDIA AI is in every cloud, and is used by major companies and more than 7.5 thousand startups around the world, "said Jensen Huang.

NVIDIA AI can run on any system that has an NVIDIA GPU

Pre-trained AI models

To further simplify the use of AI in the corporate sector, NVIDIA introduced a set of tools that allow companies not to design and train their own neural network from scratch, but to choose one of the many available in the NGC cloud catalog. The models presented in the catalog cover a variety of AI tasks, including computer vision and understanding of the natural language.

By selecting a model, the customer can tailor it to their specific needs using NVIDIA TAO (Train, Adapt, and Optimize). It is a GUI-based framework designed to create enterprise AI applications and services faster and easier.

It allows you to complete the pre-trained model from the NGC catalog on small sets of data that the user has, as well as train the model on data from various users inside encrypted enclaves in the GPU, without opening them to anyone involved in the process.

Once the model is optimized, when ready for deployment, it can be integrated with any infrastructure according to any usage scenario. And at the end stage, you can use the NVIDIA Fleet Command tool to deploy and manage the AI application on various devices with a graphics processor.

NVIDIA Jarvis

Based on the above-mentioned catalog, NGC NVIDIA introduced the Jarvis GPU-accelerated framework, which allows companies to use video and voice data to create dialogue AI services adapted for their industry, products and customers.

Pre-trained Jarvis models offer high-precision automatic speech recognition, language understanding, real-time translation for several languages. By using GPU acceleration, Jarvis is able to lay down the entire process, including listening, recognizing and generating a response, 100 milliseconds faster than blinking the human eye. At the same time, Jarvis provides 90% accuracy of speech recognition.

We taught Jarvis on the GPU for several million hours on more than 1 billion pages of text and more than 60 thousand hours of speech in different languages, "the director general of NVIDIA cited amazing data.

At the time of the announcement, Jarvis supports the following languages: English, which is especially pleasant - Russian, as well as Spanish, Japanese, German and French.

For the most efficient use of Jarvis in various industries, it can be castomized under industry jargon. NVIDIA, in particular, trained him for technical scenario and use in healthcare. And one of the "chips" of Jarvis - it can speak with emotions and expression, the level of which can also be adjusted. And which is also very important - it can be deployed, including in the cloud.

According to NVIDIA, since the launch of the early access program to Jarvis in May 2020, thousands of companies have requested access to it. Among them - telecom giant T-Mobile.

AI for 5G

As part of the GTC conference, NVIDIA also announced partnerships with companies that will develop solutions based on its AI-on-5G platform, designed to deploy AI applications over 5G networks. This will be done, in particular, by Fujitsu, Google Cloud, Mavenir, Radisys and Wind River. NVIDIA expects that this will accelerate the creation of smart cities, advanced hospitals and intelligent stores.

The AI-on-5G platform includes the NVIDIA EGX hyperconvergent computing platform, the NVIDIA Aerial development kit (SDK) for software-defined radio access virtual networks (vRAN) on 5G and enterprise applications for working with AI.

5G base stations with a large number of subscribers require computationally complex signal processing tasks with minimal delays. At the conference, NVIDIA introduced the Aerial A100, a solution that combines AI and 5G in a new type of computing platform. It contains the GPU and the DPU BlueField on the same map.

The NVIDIA Aerial SDK is a framework for performing these calculations on DPU and GPU, rather than telecom-specific equipment. At the same time, outside of peak load periods, GPUs can be used, for example, for edge AI calculations, camera maintenance in self-paid stores, autonomous vehicles. This will allow operators to get increased revenue by realizing the potential of services with a low delay. At GTC 2021, the NVIDIA BlueField-2 A100 was announced - a combined accelerator containing the NVIDIA A100 Tensor Core GPU and BlueField-2 DPU and allows you to implement protocols with strict delay restrictions, "5T for 5G."

According to the general director of NVIDIA, the Aerial A100, together with the EGX system, will be able to act as a completed 5G base station, which is also essentially a cloud native data center.

Cyber security

Did not bypass NVIDIA and the topic of cybersecurity. It has become a real challenge for companies because today, especially against the background of the development of cloud technologies, every computer in the data center is at risk of external influence.

At GTC, NVIDIA introduced Morpheus, a cloud-based cybersecurity framework based on GPU and DPU BlueField, which allows cybersecurity providers to develop AI-based solutions that can instantly detect vulnerabilities.

Morpheus uses machine learning to identify and prevent threats and anomalies that were previously impossible to identify, including leaks of unencrypted sensitive data, phishing attacks and malware, NVIDIA says.

Combined with DPU BlueField, this framework allows each computing node in the network to serve as a cyber-protection sensor, allowing organizations to analyze each packet at line speed without data replication. Traditional AI-based security tools typically select about five percent of network traffic data, so such threat detection algorithms operate with incomplete data.

Developers of network solutions and cybersecurity systems, software partners, startups and computer manufacturers can already apply for early access to the NVIDIA Morpheus platform, NVIDIA says.

Self-driving cars

NVIDIA has been working with the automotive industry for about two decades. Among the innovations for this industry that the company announced on GTC is the next generation of its DRIVE platform called Atlan. The developer calls it a "data center on wheels."

DRIVE Atlan essentially combines the entire computing infrastructure of a smart car in one chip. The platform can provide simultaneous operation of automotive self-government systems, intelligent on-board devices, multimedia applications with a high level of safety.

DRIVE Atlan includes next-generation computing cores with ARM architecture, deep learning accelerators, and machine vision. The reported performance exceeds 1,000 trillion operations per second.

DRIVE Atlan samples are expected to become available in 2023, and vehicles based on this platform are not earlier than 2025.

NVIDIA actively cooperates with automakers. In the next six years, cars equipped with NVIDIA Drive technologies will enter the roads from companies such as Volvo Cars, Mercedes-Benz, NIO, SAIC, TuSimple, Cruise, Zoox, Faraday Future, VinFast and not only.

Omniverse Enterprise - a platform for creating virtual worlds

In December 2020, NVIDIA opened beta access to the Omniverse design and collaboration platform. And within the framework of GTC, the company announced Omniverse Enterprise, a platform for collaborating with 3D graphics in various organizations. It includes a server to create and view applications and virtual workstation capabilities. The platform can be used in organizations of all sizes, allowing geographically distributed 3D development teams to work together on complex projects.

Omniverse is the fundamental platform of our strategy in the field of virtual and augmented reality, our strategies in the field of design and collaboration, "explained Jensen Huang. - It is also a platform for our strategy on virtual worlds metaverse, robotization and AI in autonomous machines. You will see more and more developments related to Omniverse. And it can be called one of the missing links, which is very important for the next generation of autonomous AI technologies.

According to NVIDIA, at the time of the announcement, more than 400 companies are using Omniverse. For example, BMW using this platform created a digital double of one of its factories.

And Bentley Systems, the developer of software for professionals in the field of construction and infrastructure management, will be the first third-party company to create an application package based on Omniverse.

20 years ago, all this was science fiction, 10 years ago it was a dream, and today we live in it, "said Jensen Huang at the end of his speech at the opening of the GTC.

According to NVIDIA, its online conference attracted an unprecedented number of participants - more than 180 thousand registered visitors, which is three times more than the largest GTC that took place earlier.

Read more news on the NVIDIA blog.

Notes