Apache Kafka

Apache Kafka is the distributed program message broker, the project open source^[1].

2020: Availability of Yandex Managed Service for Apache Kafka in Yandex. Cloud

The platform Yandex. Cloud opens access to Yandex Apache Managed Service for Kafka service which simplifies for developers and DevOps- engineers creation and management of applications in Yandex. A cloud, also does not require examination in infrastructure management of Apache Kafka and Apache Zookeeper. Service works in the preview mode, free access is provided on demand. It became known on July 28, 2020. In more detail here.

2019: Development model with involvement of many concerned parties

One of key needs of modern IT is the possibility of stream data transmission in real time. The Apache Kafka platform which can transfer stream data about the history of visits of websites, financial transactions and magazines of applications to the applications developed on the Apache Spark Streaming, Apache Storm or Apache Samza platforms is intended for these purposes. Kafka was developed by LinkedIn and in uses development model with involvement of many concerned parties. According to ASF, the platform is applied by many large companies, including Airbnb, LinkedIn, MailChimp, Netflix, the New York Times, Oracle, PayPal, Pinterest, Spotify, Twitter and Uber.

2015: Apache Kafka properties

It is developed within Apache Software Foundation. It is written in the Scala programming language.

Properties

• it was initially projected as distributed system which is easy for scaling,
• maintains high capacity, both from sources, and for systems subscribers,
• supports consolidation of subscribers in groups,
• provides a possibility of time data storage for the subsequent batch processing.

One of features - use of the equipment similar to transaction logs, used in database management systems.

In the beginning the product was developed by LinkedIn company, source codes of the project are open at the beginning of 2011.

The product is distributed under the license Apache License 2.0.

Notes