The JURECA supercomputer in Yulikh rose from 80 to the 29th place in the rating of Top-500

On May 20, 2015 it became known of plans of T-Platforms company in the project of creation of the supercomputer for computer center of Forschungszentrum Jülich GmbH in Germany^[1].

Project Tasks

According to statements, future supercomputer can exceed MSU started in 2009 in Computer center on an indicator of peak capacity of SK "Lomonosov".

The T-Platforms company will deliver the supercomputer to computer center of the city of Yulikh (Germany). From which transaction amount of €17 million, 10% will receive ParTec – the German software developer, the managing director of the supercomputer.

The name of a product of delivery is JURECA. Under this name the generation of multi-user clusters of standard architecture which resources Yulikh provides to research teams of Germany acts. A system will succeed the cluster of JUROPA created in 2009 by the French company Bull. It is supposed that JURECA will exceed performance level of JUROPA six times^[2].

As a part of computing nodes with 12-core Xeon E5-2680 v3 processors on architecture of Haswell and 128 GB of memory of DDR4, in JURECA nodes with the increased amount of memory - up to 256 and 512 GB, the selected nodes of loading and postprocessing, visualization nodes with Nvidia K40 accelerators and memory to 2 TB will be created.

As interconnect in JURECA the EDR InfiniBand technology is used by capacity of 100 Gbps. System management is implemented by means of CentOS Linux and a combination of managing software of ParaStation Cluster Tools from the German partner of the supercomputer center Yulikh — ParTec company and the Clustrx tools from T-Platforms.

Peak capacity of the computer will be 1.8 PFLOPS with a possibility of increase on 0.4 PFLOPS for some applications (to cost €518 thousand in addition). The computer will consist of 1700 computing nodes.

For T-Platforms company it is the first foreign delivery of such scale. Thus, on the peak capacity future supercomputer in Yulikh will exceed Lomonosov (1.7 PFLOPS) started in 2009, but will be inferior to Lomonosov-2 with a peak capacity of 2.57 PFLOPS started in 2014.

Capacities of the supercomputer will be necessary for scientists of Germany for modeling of different processes: the supercomputer center services the research center in Yulikh specializing in natural sciences. Researchers of the whole world can rent computing powers of the supercomputer of Yulikh.

The project of the SK JURECA installation is complete

On November 18, 2015 the T-Platforms company announced start in operation of the JURECA supercomputer in the German Supercomputer center Yulikh^[3].

JURECA (2015)

She took the 49th place in the 46th edition of the world supercomputer ranking TOP500, having shown the level of efficiency of 84% in the Linpack test.

JURECA is the first large-scale foreign computing system created on the Russian equipment. Peak capacity of a system, taking into account nodes with accelerators ~ 2.2 petaFLOPS. CK JURECA - replacement to a multi-user cluster of JUROPA of production of the French company Bull. The JURECA system exceeded "Frenchman" on performance by 10 times at decrease in energy consumption by 30%.

A system will provide computing resources for research projects in the field of elementary particle physics, biophysics, physics of condensed mediums, studying of Earth and the environment, materials science, astrophysics, hydrodynamics. The analysis of large volumes of data within the project on a research of the human brain integrating specialists from 23 countries will become one of the largest tasks. An ambitious goal of this project — creation of virtual model of the work of a brain in general considering operation of genetic mechanisms, processes of molecular level and interaction of neural clusters. Creation and processing of such model integrating knowledge of work of a human brain accumulated worldwide requires the computing resources considerably exceeding modern supercomputers.

As a part of a new system – 34 racks with water cooling containing 1872 computing nodes with the 12-core Intel Xeon processors E5-2680v3, NVIDIA Tesla K80 and K40 accelerators in structure a blade systems T-Platforms of V-Class developed in Russia. Supercomputer nodes of six different types are integrated by high-speed interconnect of new generation of EDR InfiniBand from Mellanox company: at the moment JURECA is the largest supercomputer based on this new technology. Energy consumption of the supercomputer was about 1 MW.

A system works under management CentOS Linux and uses a combination of managing software of ParaStation Cluster Tools of the German developer ParTec and the Clustrx tools from "T-Platforms". The ParaStation TicketSuite and HealthChecker tools provide monitoring of a system and guarantee stability of its work, and high-scalable implementation of MPI library provides more effective implementation of parallel computing tasks.

A jump up on 51 positions in the rating of Top-500

The JURECA supercomputer created by the Russian company "T-Platforms" and working in the German supercomputer center in Yulikh rose to the 29th place in the 50th rating of Top-500 of the most powerful computing systems of the world which was published in November, 2017. For comparison, in the 49th rating published in June, 2017, JURECA took only the 80th place.

The JURECA supercomputer in the research center in Yulikh

According to developers, the supercomputer — in the three of the most powerful computing systems of Germany. In Linpack performance test a system showed performance of 3.78 PFlop/s with a peak capacity in 6.56 PFlop/s.

As explained CNews in T-Platforms, so notable jump up in the rating of Top-500 is caused by equipment of the JURECA supercomputer in 2017 the accelerator, or the booster. It represents a separate system in which Intel Phi 7250-F accelerators and interconnect of Intel Omnipath are used. Hardware platforms were created by Intel, and the Dell company was engaged in integration works.

As result, became possible to integrate in one system a cluster on standard processor architecture with InfiniBand interconnect, namely — JURECA and the booster on Intel Phi 7250-F and Omnipath. It allowed a cluster and the accelerator to exchange data, thus, all system can work on accomplishment of one general task.

The project on consolidation of systems is implemented with the assistance of Intel, the author of the concept — the director of the supercomputer center Thomas Lippert.

As explained in T-Platforms, similar consolidation represented technically difficult task. The main problem consisted in incompatibility of interconnects that did not allow to organize data exchange. However at start of the Linpack test on both components of a system specialists from Yulikh received result in 3.78 PFlop/s. It was supposed that the total power of a cluster and the booster will be about 6.5 PFlop/s. Thus, the result received in practice was about 60% of a theoretical maximum.^[4]

Notes