The supercomputer centers RAS in Moscow and Novosibirsk are integrated by a secure channel of data transmission

2017

In September, 2017 it was announced that between two large Russian Centers of collective use — the Interdepartmental supercomputer center of the Russian Academy of Sciences (JSCC RAS, Moscow) and the Siberian supercomputer center of the Siberian department of RAS (SSKTs Siberian Branch of the Russian Academy of Science, Novosibirsk) — creates a high-performance secure channel of data transmission.

The MVS-100K and MVS-10P computing systems installed in the Interdepartmental supercomputer center RAS in Moscow. Photo: servernews.ru

Project of creation of a secure channel of data transmission

Stages and contractors

In April, 2017 resources of two centers of collective use — JSCC RAS in Moscow and SSKTs Siberian Branch of the Russian Academy of Science in Novosibirsk — were united in the geographically distributed computer system, and since July, 2017 they were connected by the high-speed secure channel of data transmission (10 Gbps) created by specialists of group RSK, the companies S-Terra CSP and "Milecom" and also Institute of computing technologies of the Siberian Branch of the Russian Academy of Science.

Appointment

It is expected that use of qualitatively secure channel of communication will allow the Russian scientists to optimize the solution of relevant tasks in the field of fundamental and applied sciences, including such advanced directions of researches as:

• digital and intellectual technologies,
• hi-tech health care,
• increase in environmental friendliness and efficiency of power,
• development of an aircraft industry complex,
• mastering of space,
• artificial intelligence (AI, Artificial Intelligence),
• machine and deep learning (Deep Learning),
• work with Big Data and others.

Many of calculation tasks contain confidential information therefore creation of a secure channel of data transmission between two supercomputer centers was necessary for expansion of opportunities of use of the integrated computing resources. Further it will allow to integrate also other Russian regional supercomputer centers in the uniform protected data network.

Key tasks of our center — to provide new opportunities for carrying out research and development, to provide to research staff of RAS powerful resources for the solution of different scientific and applied tasks and also to provide the organization of effective work of the Russian scientists from different regions of the country. Putting into operation of a secure channel of data transmission between JSCC RAS and SSKTs Siberian Branch of the Russian Academy of Science, after recent upgrade united in a geographically distributed supercomputer complex, completely corresponds to current demands and the growing requests of our users not only in data processing fall forward, but also in providing the high level of confidential information protection — Boris Shabanov, the director of the Interdepartmental supercomputer center of the Russian Academy of Sciences noted.

When calculation tasks of our users require much bigger computing resources, than can provide SSKTs, we provide opportunities for use of additional capacities by redistribution of loading and applications launch on a supercomputer complex JSCC RAS. The secure channel of data transmission organized between our centers provides complete confidentiality of the transmitted data according to requirements of the scientific organizations and other our customers" — Boris Mikhaylovich Glinsky, the chief executive of TsKP "Siberian Supercomputer Center" IVMIMG of the Siberian Branch of the Russian Academy of Science emphasized.

Testing

As a result of the held testing an opportunity to transfer data on a secure channel with speed up to 4 TB per hour (about 9 Gbit / c) between two centers located from each other at distance about 3000 km was shown. Data protection is provided according to domestic cryptographic algorithms GOST P 34.10-2012, GOST P 34.11-2012, VKO_GOSTR3410_2012_256, GOST 28147 implemented in cryptogateways C-Terra based on the unified high-end servers of "RSK of the Tornado".

Objectives

Further in MSTs and SSKTs implementation of the protected VDI infrastructure for the organization of remote work with resources of the supercomputer centers in the conditions of the growing cyberthreats is planned. Protection of high-performance data transmission channel is implemented based on the specialized gateway of security "C-Terra Gateway 10G" which combines the innovative technologies of processing of network packets and GOST enciphering under standard protocols of IPsec. Achievement of high rates of performance of enciphering of transmitted data became possible thanks to, including, balancing by the VPN gateway of computing loading between cores of modern Intel Xeon processors.

Project on upgrade of supercomputer systems

At the beginning of 2017 the RSK group upgraded supercomputer systems in JSCC RAS (Moscow) and TsKP SSKTs Siberian Branch of the Russian Academy of Science (Novosibirsk). They became a basis of the geographically distributed computer system which gross peak capacity exceeded 1.1 petaFLOPS. As result, the Russian scientists from RAS organizations in Moscow and Novosibirsk and also from other regions of the country had opportunities for use of resources not only one, but also, if necessary, at once two supercomputer centers at the same time.

Course of projects

From the technical point of view, both projects in the supercomputer centers RAS in Moscow and the Siberian Branch of the Russian Academy of Science in Novosibirsk are remarkable the fact that in their framework implementation of server computing nodes with liquid cooling in the "hot water" mode based on the powerful 72-core Intel Xeon processors Phi 7290 and also on the basis of the 16-core Intel Xeon processors E5-2697A v4 is implemented. Besides, during projects for the first time in Russia and the CIS communication subsystems of two cluster complexes were implemented on the basis of the high-speed Intel Omni-Path internodal connection at data transfer rate of 100 Gbps.

High availability, fault tolerance and ease of use of the computing systems created based on solutions of RSK for high-performance computing are also provided thanks to the advanced management system and monitoring based on software "RSK BASIS". It allows to exercise control of both separate nodes, and all solution in general, including infrastructure components. All elements of a complex (computing nodes, power supplies, hydroregulation modules, etc.) have the built-in module of management that provides ample opportunities for detailed telemetry and flexible management.

The construct of a cabinet allows to replace computing nodes, power supplies and hydroregulations (at the condition of application of reservation) in the mode of hot swap without interruption of operability of a complex. The majority of components of a system (such as computing nodes, power supplies, network and infrastructure components, etc.) are the software-defined components allowing to simplify and accelerate both initial deployment, and service, and the subsequent upgrade of a system. Liquid cooling of all components provides long term of their service.

Results of works in JSCC RAS

JSCC RAS is one of the most powerful supercomputer centers of collective use in Russia in the field of science and education. The MSTs group consists of highly-skilled research associates, programmers and engineers. As of 2017, 184 groups of researchers solving problems of fundamental and applied orientation use resources of the center.

During upgrade the gross peak capacity of the updated supercomputer park JSCC RAS implemented based on RSK of the Tornado and RSC PetaStream computing systems with liquid cooling grew by 40%. As a result the overall performance of computing resources JSCC RAS exceeded 900 TFLOPS.

Results of works in TSKP SSKTS

As of 2017, 24 institutes of the Siberian department of RAS (about 200 users) use services TsKP SSKTs. After installation in March, 2017 of new supercomputer cluster system of NKS-1P based on the solution "RSK of the Tornado" with liquid cooling the general computing resources of TsKP SSKTs Siberian Branch of the Russian Academy of Science were increased almost twice — by 71% to the level of 197 TFLOPS.