YTsaurus (YT)

History

2023: Source Code Publication

Yandex has revealed the sources of its main platform for working with big data YTsaurus. The press service of the company announced this on March 20, 2023.

As told in Yandex, the platform is suitable for a wide range of tasks, from analytics to training complex models with billions of parameters. For example, "Search" builds a search index using YTsaurus, and self-driving cars use the platform to process travel data and improve their algorithms. YTsaurus manages Yandex supercomputers, distributing the load so that their computing power is used most efficiently.

YTsaurus is Yandex's big data platform

By March 2023, Yandex has deployed the YTsaurus platform on tens of thousands of servers and processes data exabytes; every second employee of the company works with her. YTsaurus can be used as a classic MapReduce system, but it also supports other popular approaches to data processing - for example, it has integrations with ClickHouse and Apache Spark.

YTsaurus source code and documentation are available on GitHub. The code is distributed under the Apache 2.0 license. Anyone can use the platform or modify it for themselves.

Yandex has been developing YTsaurus - or YT, as we call it internally - since 2010. We started building our own ecosystem for big data, because none of the solutions on the market met all our requirements. Now YTsaurus is one of the key elements of Yandex's internal infrastructure. Dozens of developers are working on the platform, and its capabilities are constantly expanding, "said Maxim Babenko, head of the distributed computing technologies department, quoted by the Yandex press service on March 20, 2023.[1]

Notes