Microsoft has created one of the five most powerful supercomputers in the world

2020: Development of a supercomputer based on the Azure cloud

May 19, 2020 Microsoft announced the development supercomputer on the base clouds Azure in collaboration and exclusively for a non-profit OpenAI research organization, the founders of which are Elon Musk Sam Altman. It has become one of the 5 most powerful supercomputers in the world. The supercomputer is designed specifically for training the next generation of massive AI models capable of performing a wider range of functions and, accordingly, requiring large computing resources.

Historically, machine learning experts have developed small AI models that used sets of marked-up data to learn how to perform one separate task - to recognize an object, translate text, etc. The artificial intelligence community has proven that some of these tasks are better performed by one large model trained, for example, on billions of pages of publicly available texts. As a result, such a model understands the nuances of speech, grammar, concepts and context so subtly that it is able to perform several tasks perfectly: summarize long performances, moderate chat in online games, and even generate program code based on search results on GitHub. Microsoft, for example, has developed its own family of similar models - Microsoft Turing models - and a language model with 17 billion Turing Natural Language Generation (T-NLG) parameters.

The creation and training of such massive models requires the presence of a progressive supercomputing infrastructure or clusters of modern equipment connected using high-bandwidth networks. The supercomputer developed for OpenAI is a single system with more than 285,000 processor cores (CPU core), 10,000 graphics processors (GPUs) and a network connection speed of 400 gigabits per second for each GPU server. With its deployment in the Azure cloud, the supercomputer has the benefits of a state-of-the-art cloud infrastructure, including rapid deployment, data center reliability, and access to Azure services.

Developing a supercomputer for OpenAI is the first step towards making the next generation of massive AI models and the infrastructure needed to train them available to a wide range of organizations and developers worldwide^[1] to^[2].

2016: Microsoft-OpenAI collaboration begins

On November 15, 2016, it became known about the beginning of cooperation between Microsoft and the non-profit organization OpenAI, which develops artificial intelligence (AI) technologies. Under the announced partnership, Microsoft Azure will become the preferred, but not the only, cloud infrastructure for OpenAI projects.

OpenAI, co-founded by entrepreneur Elon Musk, will use Microsoft's cloud for its deep learning and artificial intelligence experiments.

In turn, Microsoft will be able, together with a new partner, to conduct research and create new tools and technologies that can be widely used in medicine, telecommunications, programming, etc.

Microsoft connected Elon Musk's AI project to its cloud

OpenAI chose Microsoft through our deep learning research, ongoing commitment to artificial intelligence, Azure's support towards open source technologies, and the unique combination in this platform of high performance computing, big data, and intelligent features such as Azure Batch, Azure Machine Learning, and Microsoft Cognitive Toolkit, Microsoft said in a release.

OpenAI announced its intention to use tens of thousands of Azure virtual machines to "increase the number of experiments being conducted and the scale of the models being tested." According to OpenAI co-chairman Sam Altman, cooperation with Microsoft best meets the organization's goals of "democratizing" artificial intelligence, and partners' views on these technologies largely agree.

Altman noted that OpenAI uses not only Azure, but also other cloud infrastructures, but most of the work will be carried out in the Microsoft cloud.^[3]

Notes