ClickHouse - Database Management System (DBMS)

Main article: Database Management System (DBMS)

ClickHouse is a database management system (DBMS) developed by Yandex, which allows you to store and quickly process large amounts of information for creating analytical reports. The technology was created for the Yandex.Metrica web analytics service, but it is also used in other Yandex projects, as well as for business analytics and infrastructure monitoring. ClickHouse can be used in telecommunications, advertising, online commerce and information security.

2021

Creation of the company ClickHouse, in which invested $50 million

On September 20, 2021, Yandex announced the creation of a new ClickHouse company, within which products will develop on the basis of the open DBMS of the same name. In addition to Yandex, $50 million was invested in this project, Benchmark Capital and Index Ventures venture funds invested in it. More details here.

Use for PAK Prana

On February 8, 2021, the company MOUTH announced that it was PRANA hardware and software complex switching to the Russian an analytical databases Yandex ClickHouse management system. More. here

2020: Yandex ClickHouse 20.8 with modifications from Arenadata

The next version of the column analytical DBMS with open source ClickHouse 20.8 from Yandex came out with modifications of Arenata - the Kerberos authorization functionality for Kafka. Arenata announced this on September 28, 2020.

Arenata is an active member of the OpenSource community. The company contributes to the development of many systems, including Yandex ClickHouse, on the basis of which the Enterprise-level DBMS Arena QuickMarts (ADQM) operates.

Now the user can easily set up authorization in ADQMM - the configuration file ClickHouse manages the librdkafka library, which provides interaction with Kafka. Parameters related to Kerberos also worked.

We had a rather ambitious task, "says Andrey Konyaev, product manager of Arenata QuickMarts. - The main difficulty was due to the fact that the ClickHouse initially includes a minimum number of external dependencies. The standard way for the product to use libraries is full integration. Another feature of the product is attention to testing. It took a lot of effort to create an environment of docker containers with Kafka, ZooKeeper and Kerberos KDC for tests of the functionality that appeared.

Arenata plans to continue refinements. In the near future - Kerberos authorization of access to HDFS.

The Argentata QuickMarts DBMS (ADQM) differs from the base ClickHouse in a number of parameters. The features of the system are the ease of installation and configuration from the Arena Cluster Manager orchestrator, native integration with Arenadata DB (Greenplum), Hadoop, Streaming as part of the single Arenadata EDP platform, user authorization, including support for integration with Kerberos Hadoop, as well as monitoring based on Graphite.

The company performs additional testing and quality control of releases. There is a possibility of operational refinement and development of functionality under the requirements of the customer. Another feature is the availability of operational Russian-language technical support.

2019: Inclusion in the Register of Domestic Software

On June 21, 2019, the company Yandex announced that register of domestic software it included - Yandex. Tracker a service for organizing the work of the team - database management system and ClickHouse. These products will now take precedence when selected. software for state-owned companies More. here

ClickHouse will be interesting to organizations that deal with data. This system can process huge amounts of information in real time and build analytical reports in different sections. Its scope: analytics of sites and applications, advertising, telecommunications, monitoring and telemetry processing, information security - and this is not a complete list. The system is easy to scale and allows you to store records of trillions of events.

2017: Galaxy adapted Yandex web analytics for industrial tasks

Corporation "," a Galaxy Russian developer of corporate information systems, optimized work with Yandex.Metrica for its customers. Experts have found various possibilities to use the web analytics technology ClickHouse (Yandex) to analyze the economic activities of enterprises. The technology can be used in both ERP Enterprise Resource Planning and Business Intelligence. BI

ClickHouse is a database management system that was originally developed for Yandex.Metrica web analytics. ClickHouse allows you to process huge amounts of data (over two terabytes per second) while maintaining fault tolerance, and does this much faster than other solutions present in the global market. To gain time for users of its business applications, Galaxy has optimized Yandex technology. To evaluate it, specialists used 13 million of their own journal entries and calculated the ClickHouse turnover in accounts and cash flow with funds. The result was convincing. If in a traditional accounting system similar problems are solved in minutes, then in a layout using ClickHouse - in seconds.

The solution allows the business user to work online with reports. For example, you can switch quickly from viewing the fund balances to the beginning of the month - to analyzing the turnover in the section of sub-accounts, etc. In a short session, the user can calculate several management solutions and choose the optimal one among them.

Interested users of ERP and BI solutions can work with the data in the test model of the Galaxy and make sure how quickly the ClickHouse copes with typical business tasks.

2016: Source Code Disclosure

In mid-June 2016, it became known about Yandex's plans to disclose the source code of the ClickHouse. This newspaper "Kommersant" was told in the Internet company itself.

According to the head of the ClickHouse development group Alexei Milovidov, the publication of the source code is "one of the most significant contributions to Open Source in the history of the company."

Yandex reveals the source code of the DBMS ClickHouse

Revealing the source code of the ClickHouse instead of making the product commercial, Yandex is conducting a "big experiment," said the head of the Russian representative office ESETDenis Mateev.

If he has positive results, then Yandex will improve the life of specialists in electronic commerce, analysts, media and software developers, "said Mateev.

In his opinion, ClickHouse has the potential to use in the Security Operation Center - monitoring centers, where information about events related to information security flows.

A security consultant at Yandex Alexey Lukatsky believes that if Yandex is ready to share its analytics and help its customers implement and configure the system, the company will be able to become a "very serious player" in the market for information security monitoring tools.^[1]

Notes