Apache Storm

Apache Storm is computing environment on the basis of Open Source for work in real time.

Apache Storm is easily integrated with other components, has high rates of performance and is rather just developed, creating distributed system of RT calculations for fast processing of big data streams, adding features to processing in Apache Hadoop.

Initially the project is created in BackType, then Twitter is purchased and in September, 2013 it is transferred to Apache incubator.

Use of Storm in a cluster of Hadoop provides effective processing of all range of workloads: from work in real time to interactive and processings by groups.

Scheme of interaction Apache Storm

Now Storm is developed on open model, among participants of staff of the companies Hortonworks Twitter Verisign, Yahoo and others. Testing of Apache Storm when processing one million 100-byte messages per second on one node (node) showed fault tolerance of the application, scalability between nodes of a cluster and simplicity of work.

Apache Storm supports integration about DBMS, parallelization, separation and repeated attempts at errors, when necessary.

2016: Apache Storm 1.0

On April 12, 2016 Apache Software Foundation announced release of release of Apache Storm 1.0.

Developers compare the project with Hadoop, only for data processing in real time (instead of batch mode).

The community of developers sees this release of Apache Storm significant in evolution of the project, with its exit in the platform the set of functions and improvements appeared. Among significant changes:

• qualitative increase in productivity (up to 16 times in comparison with the previous releases);
• integration of the optional heartbeat-demon of Pacemaker;
• API for the distributed cache (Distributed Cache API);
• support of HA Nimbus for high availability of a cluster;
• Native Streaming Window API for stream data processing;
• Stateful Bolt API with automatic check of a status;
• possibility of dynamic change of levels of logging;
• the mechanism of automatic Backpressure for reduction of capacity when exceeding of the configured thresholds;
• the scheduler of RAS Scheduler considering available resources in a cluster (RAM, the processor);
• function of the distributed search in magazines (Distributed Log Search) in the user interface Storm.