Tele2 has increased the capacity of the big data platform using RT.DataLake

2022: Big Data Storage and Processing Cluster Expansion

Russian operator, mobile communication Tele2 announced on July 7, 2022 the successful completion of the expansion of the existing storage and processing cluster big data through the RT.DataLake solution. The total capacity of the updated implemented Hadoop RT.DataLake cluster was 2.4 PB. This made it possible to increase the useful capacity big data of the Tele2 platform by 40% and increase productivity for task calculations. machine learning Thanks to the expansion, the company has reduced dependence on foreign, ON gained the ability to increase computing power and scale the current solution without restrictions.

Tele2 has been using the Hadoop cluster for data storage and analytics since 2018. During this time, more than 100 data sources are integrated into the cluster, and the daily volume of integrated data reaches 100 TB. Dozens of business and technical teams use the big data platform every day, while the main internal client is the data analytics and monetization team. The load on the cluster grew continuously, and the free space decreased. Complex data integration processes made it difficult for the data scientists team to work. In this regard, the company decided to divide the architecture of the big data platform into a data processing segment and a data science segment.

To expand the big data platform, Tele2 chose the RT.DataLake product from Rostelecom based on Hadoop technology. This decision showed the best indicators for budget savings and total cost of ownership: the calculation in rubles for licenses excludes currency and sanctions risks, and the total savings over 5 years will be more than $3 million.

The RT.DataLake build is one of the most current commercial Hadoop distribution in the world - it consists of the latest stable versions of frameworks and components. This makes it possible to solve the needs of data engineers and data scientists Tele2, which are extremely demanding to regularly update versions of components in the cluster. Rostelecom, as a software vendor, has shown its readiness to customize the distribution kit for the needs of Tele2. This made it possible to provide the necessary set of used versions of the Hadoop component, implement the addition of functionality to the project source code and provide a set of deployment and management automation tools based on Ansible technology. Tele2 reliably ensures information security and data security by constantly conducting audits and improving methods of their protection. RT.DataLake is no exception. The solution implemented access differentiation based on Ranger technology, implemented the Kerberos authentication protocol and integrated with the corporate Active Directory service. The Rostelecom team regularly releases component update patches and information security threats.

Tele2 IT employees expanded the cluster without involving contractors. A great help in this process was provided by the Rostelecom team, which eliminated any shortcomings in the distribution in a matter of days. Our company has reduced its dependence on foreign software, got the opportunity to scale the big data Tele2 platform without restrictions and increase computing power, noted Alexey Martynov, Director of Information Technology Tele2.

After expansion, the big data Tele2 platform allows you to store 6.6 Pb of data, consists of 126 computing nodes with a total power of 9,000 cores and 86 TB of RAM. The power of the extended cluster allows you to comfortably work with the data scientists team and solve the most high-load problems. We are confident in the future and look forward to the full implementation of our plans to launch products based on big data analytics both for Tele2's tasks and for a wide range of external customers and partners, told Anton Merzlyakov, director of big data analytics at Tele2.