Severstal will organize "the lake of data"

On August 2, 2017 PJSC Severstal announced creation of the hybrid data warehouse (Data Lake). Infrastructure is focused on storage of an array of the technology data collected at the enterprises. They will be processed and used for project implementation of the company in the field of analytics of data, machine learning and artificial intelligence.

Project Tasks

The storage assumes hybrid architecture - a combination of two models of data storage – in own data processing center (DPC) and lease of capacities in cloud services. Project implementation is supposed based on mainly open-sourse of software products.

Severstal signed the contract with Lenovo Group for delivery of servers with the cumulative size of storage of 2 PB in own DPC. The cluster will have 30 TB RAM and 1200 cores of processor power for calculations.

The framework agreement is signed with Microsoft company about lease of computing powers in a cloud service of Microsoft Azure. The agreement will give the chance to take advantage of hybrid model and to get access to almost unlimited well protected resources of a public cloud of Microsoft which supports technologies of different producers, including technologies open source. The organization of dynamically measured storage which will be used, first of all, under project tasks when certain capacities are required for a specific time frame is supposed.

For transport of data it is going to use the solution based on open source software Apache Kafka and Spark which will allow to transfer stream data with a low delay and to analyze them in real time.

Practically all aspects of digital-transformation of the company come down to data processing. Therefore creation of the infrastructure capable to store and analyze a huge array of information collected by us at the enterprises – will lay the foundation for implementation of digital strategy of Severstal. And the hybrid architecture of the created storage will allow to solve most cost-efficient all complex of problems in the field of the machine learning and predictive analytics facing us and also to provide the high performance of processes of transfer and data processing and information security of the company. Igor Bardintsev, development director of digital technologies of JSC Severstal Management

In the lake of data of Severstal storage, first of all, of the data collected from sensors on industrial equipment (Internet of Things), servers of the industrial control system, MES systems is supposed. On the basis of the collected data it is going to implement projects on predictive analytics in such spheres as predictive repairs of the equipment, optimization of quality of the made products and others where perhaps and economically justified use of artificial intelligence.

The purpose of each digital-project – to bring efficiency of specific process to a maximum. We already implemented several interesting initiatives on CherMK, for example, the project on prediction of defects in workshop of cold rolling, we pilot several models in the field of predictive repairs on the Camp-2000, and we see that they bring visible results. But the more at us will be opportunities for collecting, storage and data processing, the we will be able to solve more similar problems. Therefore development of the data warehouse is a permanent process. Igor Bardintsev