Google Mesa

Mesa is the distributed DBMS for processing of continuously updated information which is able to service trillions of requests within a day.

On August 8, 2014 the Google company announced technology of distribution of the data warehouse between several data centers using the architecture developed in the company capable to continue a way to much more large-scale, reliable and fast cloud analytical systems - Mesa.

Google reported: Mesa will provide storage of petabytes of data, updates millions of lines per second and services trillions of requests a day. Distribution of a system between several data processing centers is possible and it will continue work at shutdown of one of them. The technology is created for storage and the analysis of control and measuring data of business of online advertizing of Google, but, according to developers, a system can be used in other similar tasks.

Mesa helped Google to solve a number of operational problems with which traditional data warehouses and analytical enterprise-level systems did not cope.

Google was required that new data were subject to the analysis directly on receipt in base, and the majority of commercial data warehouses do not support continuous data-refresh. The company needed a reliable guarantee of coordination of requests - consistency, i.e. the identical request from the same source always should yield the same result irrespective of the fact which the DPC processes a request.

This compliance was always provided by only relational DBMS, but they do not cope with loading petabyte of data if the base is replicated between many servers in a cluster as it usually becomes at the enterprises for acceleration of a response and increase in coefficient of trouble-free operation. NoSQL bases, such as Cassandra, with ease cope with loading of huge amounts of data, but Google was required much higher level of consistency, than provide the majority of not relational DBMS. According to specialists of the company, any of commercial products and the Open Source projects did not meet the requirements and then Mesa was created.

In Mesa some other the technologies developed in the company including Colossus distributed file system, distributed system of data storage of BigTable and an analytical framework of MapReduce is used. The consistency helps to provide the distributed protocol of synchronization Paxos too created in Google.

Mesa has one more advantage in addition to scalability and consistency — it can work on standard servers, does not require the specialized expensive equipment. Because of it it can work as the cloud service scaled both up and down - as necessary.

Mesa does not support poll given using the SQL commands. Therefore developed technologies with support of languages of the requests similar to SQL for Hadoop - Hive, Impala and Shark.