ADB Arenadata DB

Arenadata DB (ADB) is a distributed DBMS using the concept of MPP (massively parallel processing, massively parallel computing) and based on open source DBMS - Greenplum.

Analytical massive-parallel DBMS are designed for storing and processing large amounts of data - from units to hundreds of terabytes of data. Such DBMSs are most often used for predictive analytics, regular reporting, analysis of customer outflows, and building corporate data warehouses.

2024

Run on dedicated Selectel servers

Selectel and Arenadata have announced a technology partnership. The company was informed about this on October 10, 2024.

Thanks to cooperation, companies will be able to offer the market a solution in the field of big data management - the Arenadata DB (ADB) analytical database of the Enterprise level on dedicated Selectel servers. Arenadata DB is designed to work with large analytical queries and is capable of containing tens and hundreds of TB of data in corporate storage. Ready-made configurations of dedicated Selectel servers allow you to quickly deploy the solution, integrate it with the company's existing IT systems and launch the project in the shortest possible time without the need to purchase your own server equipment. The solution can be especially relevant for banks and fintech, retail, telecommunications companies, production and mining enterprises, and the public sector.

According to established practice, Arenadata DB is deployed on-premium (on local client equipment), which requires additional investment in server equipment, or in the cloud, which can impose restrictions on the maximum size of the database. Leveraging Selectel's expertise, the solution offers tested dedicated server configurations that can be quickly ordered and scaled as the load grows - combining dedicated performance with cloud flexibility. The use of bare metal (dedicated servers) will allow customers to achieve higher and predictable performance and provide infrastructure-level isolation. This will also make it possible to place Selectel in the certified data center segment, which, together with the certificates of Arenadata DB itself, allows you to build a productive, secure and certified solution that meets the special requirements of banks and other large corporations and at the same time optimize IT infrastructure costs.

In recent years, we have been actively working with the Enterprise segment and developing our line of services, including in accordance with requests from large companies. We see that when launching solutions for working with large amounts of data, the business is increasingly demanding on the IT infrastructure. Thanks to our partnership with Arenadata and Selectel's expertise in bare metal, customers will be able to get a ready-made solution with maximum performance and deploy a database for storing and processing big data in compliance with all information security requirements and without capital investments in their own infrastructure, - said Alexander Tugov, Director of Service Development Selectel.

The development of the IT market is directly related to technological partnerships. And our collaboration with Selectel is a vivid example of this. The interaction of two companies from different directions will create a synergy that will allow customers to get the necessary solution in the field of storage and data management. It will make it possible to quickly process complex analytical requests, while providing maximum performance, flexibility and security, "said Anton Martynov, Director of Partner Relations at Arenadata.

Compatible with Polymatica Dashboards

SL Soft and Arenadata announced the compatibility of their products - the designer of information panels Polymatica Dashboards and database management systems Arenadata QuickMarts (ADQM) and Arenadata DB (ADB). Softline announced this on August 8, 2024. Read more here.

Arenadata DB as a replacement for SAP BW, Oracle and Teradata

Today's business is increasingly dependent on efficient data management. Against this background, companies are striving to implement products that optimize costs and improve performance. In Russia, this issue has become especially acute in the past few years in connection with the need to switch from foreign solutions to domestic ones. Arenadata DB (ADB) has become a popular replacement for data management systems such as Oracle and Teradata, as well as one of the most important components in replacing SAP BW. Read more here.

MWS Cloud Availability

MTS, a digital ecosystem, on July 26, 2024 announced the conclusion of a strategic partnership between MTS Web Services (MWS), part of the MTS group, and Arenadata. As part of the agreement, MWS launched five services on software products from Arenadata, including Arenadata DB (ADB). Read more here.

Confirmation of compliance with the requirements to DD 4 of FSTEC of Russia

The FSTEC of Russia confirmed the compliance of the Arenadata DB analytical DBMS (ADB) and the cluster column DBMS Arenadata QuickMarts (ADQM) with the requirements for level 4 confidence in information technology security tools (CD 4). Previously, both products were certified according to the 6th level of trust and technical conditions. The developer announced this on July 8, 2024.

As a DBMS developer, we are constantly working to improve the security of our products. This fact confirms that both Arenadata DB and Arenadata QuickMarts have successfully passed the regulator's tests for certification at a higher level of FSTEC trust. This is especially important for those of our customers who belong to the objects of critical information infrastructure, "said Alexander Ermakov, technical director of Arenadata.

Plan of integration with "Garda Masking"

The Gardaí Group of Companies, a manufacturer of a family of products for data protection both network security, and the developer of the collection platform data storage and Arenadata, are deepening their technological partnership in order to provide customers not only with the most rich functions, but also with well-protected storage, processing and analysis solutions. dataGardaí This was announced by the Group of Companies "" on March 26, 2024.

In the near future, partners will focus on integrating products for incident logging and auditing operations with databases and business applications, and real-time streaming analytics. The next project in line is to ensure that the Arenadata DB database management system works together with the Garda Masking data depersonalization solution.

Arenadata is a manufacturer of software products and solutions with a large set of functions for working with large amounts of information, their storage, analysis and statistical processing. Often, such arrays contain legally protected information, and security requirements for processing systems are increased. By combining the expertise of specialists and creating joint solutions, our companies will be able to cover the needs of customers in the complex, "said Pavel Kuznetsov, director of strategic alliances and interaction with state authorities of the Garda group of companies.

Similar technological alliances, which we can observe on the example of Arenadata and Garda, are increasingly becoming widespread in the Russian IT market, as they allow us to qualitatively expand the functionality of the software systems used. In our case, customers get a comprehensive solution for working with data, the security of which is significantly increased, - said Anton Martynov, director of work with partners Arenadata.

The experience of successful cooperation within the framework of Arenadata DB has contributed to the expansion of our technological partnership. And we are focused on developing scenarios for synchronizing data streaming processes in Arenadata Streaming and monitoring write and read operations using the Garda DBF data protection system, "commented Anton Kovalenko, Director of Product Marketing at Arenadata.

For our part, we are also working to expand the functionality for protecting systems of the Big Data class, in particular, based on information obtained from Apache Ranger logs. This contributes not only to the overall development of the product, but also expands the possibilities for technological partnership, "added Dmitry Larin, head of the product area for database protection of the Garda group of companies.

The technological cooperation of the companies actually began at the end of 2023, when joint test tests took place, confirming the compatibility of the analytical MPP DBMS Arenadata DB (ADB) and the security complex of the Garda DBF DBMS. This comprehensive application of the software stack enables customers to provide a high level of protection data stores against unauthorized access, as well as to prevent cases of corporate fraud and unwanted actions by users within the organization.

Available to users in VK Cloud in Kazakhstan

The Arenadata DB (ADB) cloud DBMS has become available to VK Cloud users in Kazakhstan. With it, companies can create a single enterprise data warehouse to solve analytical problems without the cost of implementing and supporting infrastructure. Using Arenadata DB accelerates sales analytics, purchasing planning, data collection for ML model training, and other business tasks. VK (formerly Mail.ru Group) announced this on March 5, 2024. Read more here.

Compatible with Astra Linux Special Edition 1.7

Arenadata has released an updated release of the Arenadata DB analytical distributed database (ADB), which includes support for the Astra Linux operating system. The developer announced this on February 20, 2024.

Arenadata DB is compatible with Astra Linux Special Edition 1.7 in the basic mode of operation of the IPS ("Eagle"). This gives ADB users more flexibility in terms of choosing system software for deploying company products, while using reliable solutions included in the register of domestic software.

Also among the added features of ADB:

• Set the custom JAVA_HOME path for the cluster. From this release in ADB, you can use both OpenJDK 8 in the delivery from the OS vendor, and Java 8 and 11 versions from other sources. Astra Linux requires Java pre-installation (recommended by Axiom Java 8);
• Docker CE support. Installation is available on servers with pre-installed Docker Community Edition;
• generation of PXF logs in JSON format. Users will be able to organize the collection of PXF logs in the standard JSON format for log collection and analysis systems (for example, ELK), which simplifies and speeds up this process.

This release of Arenadata DB has expanded the range of possibilities in terms of choosing the operating system used to users for whom a high level of information security is important. Another important change: our customers on Astra Linux will now be able to use the domestic assembly of Java 8 from the register of domestic software, - said Alexander Ermakov, CTO of Arenadata.

The changes also affected ADB-related products. In Arenadata DB Backup Manager (ADBM) - a pgBackRest-based fault-tolerant system for managing binary ADB backups - it is now possible to restore mirrors from binary backups in parallel with primary segments. This refinement will help significantly simplify and speed up the process of restoring a PROD- or DR-cluster using physical backups. Also added is the automatic installation of the Skip validation flag (without the ability to reset) in the Restore modal window for recovery in stopped ADB clusters.

The Arenadata DB Real-Time Request Monitoring System (ADBC) has added the Sessions tab on the Monitoring page with the ability to interrupt sessions. Therefore, users have the opportunity to collect data on sessions, which allows administrators to control and save connection pool resources (connections to DBMS). The filtering process on agents for collecting system metrics is also optimized, which allows you to optimize or reduce the consumption of hardware resources.

Availability from the CROC Cloud

Croc Cloud Services and Arenadata entered into a partnership agreement in which Arenadata products became part of the services provided to customers on the basis. In CROC clouds particular, provider Arenadata Hadoop (ADH) is now available to cloud customers. CROC announced this on January 17, 2024. More. here

2023

Compatibility with "Garda DBF" DBMS security system

The Russian vendor Arenadata and the developer of information security systems, the Garda group of companies, as a result of test tests, confirmed the compatibility of the Arenadata DB analytical MPP DBMS (ADB) and the Garda DBF DBMS security complex. Arenadata announced this on November 8, 2023.

The comprehensive application of the software stack will give customers the opportunity to ensure a high level of protection of their data warehouse from unauthorized access, as well as to prevent cases of corporate fraud and unwanted actions by users within the organization.

As part of the tests, the companies conducted checks: · detection of the Arenadata DB system and its settings in automatic and/or manual mode in the Garda DBF product; · the Garda DBF system to determine the users connected to the ADB database; · Audit event management · generated by Arenadata DB · the impact of the Garda DBF product on Arenadata DB reliability and performance parameters; · interaction of the Garda DBF connection agent with the Arenadata DB virtual database hosting environment.

In addition, the Arenadata and Garda teams tested Arenadata DB for configuration capabilities, security policies, network traffic storage policies and thresholds when using DBMS, and statistical anomaly search mode for the product.

The tests performed gave expected and positive results, therefore they are considered successful. Their uniqueness was that previously the Garda DBF agent was not tested for compatibility with analytical massively parallel. DBMS Similar tests were carried out only with SMP systems Postgres and. MS SQL

{{quote "The database is a fairly vulnerable component of the infrastructure, since applications require extensive rights that are not always convenient and possible to limit by protecting the database itself. Especially if we are talking about privileged users, for example, administrators who can do almost everything in the database. At the same time, the value of information for reading or writing can be too great to be trusted even by administrators. "Garda DBF" allows you to exercise control, including proactive, at a deeper level than the DBMS itself does. Therefore, for us this is an interesting product that complements and expands the functionality of Arenadata DB, "commented Sergey Zolotarev, Director of Strategic Development of Arenadata.}}

The data in our time, during the so-called VI technological structure, have actually turned into a new "currency" and are of independent value. Thus, violation of the state of their security - illegitimate changes, data leakage or their destruction - carries risks that are already tangible, flowing into the real, physical world. In such a situation, developers of data storage, processing, transmission and analysis systems and developers of security tools themselves dictate the need to create technological alliances to ensure data security, starting at the level of the complex solutions architecture itself. Realizing the responsibility to users, we are working in detail on the compatibility of Garda protection tools with DBMS manufacturers. The specialized expertise of the Arenadata DB development team allows us to jointly offer consumers a comprehensive solution that ensures both solving business problems and a high level of protection of information circulating in the system, "said Pavel Kuznetsov, Product Director of the Garda Group of Companies.

Cloud Availability Cloud.ru

and Provider cloudy AI-technology Cloud.ru has become a strategic partner Russian of the vendor ON for storages and processing. data Arenadata Arenadata products can now cloud Cloud.ru be used in. Arenadata announced on August 29, 2023.

According to TAdviser representatives of Arenadata, the following products of the company are available in the Cloud.ru cloud: ADB (Arenadata DB), ADH (Arenadata Hadoop), ADQM (Arenadata QuickMarts) and ADS (Arenadata Streaming). Read more here.

Availability in beeline cloud

The product ecosystem Arenadata is now available to customers. cloudy provider beeline cloud Arenadata announced this on August 3, 2023. As TAdviser representatives of Arenadata specified, among the products available through the beeline cloud: Arenadata DB,,,, and Arenadata Hadoop. Arenadata Streaming Arenadata Postgres Arenadata QuickMarts Arenadata Cluster Manager Read more here.

Rubbles MLOps Suite Compatibility

IT companies Rubbles and Arenadata have made their software products compatible. Now the Rubbles MLOps Suite platform and Arenadata data storage and processing solutions can work seamlessly in a single software package. Arenadata announced this on July 19, 2023.

A single software package combines Rubbles MLOps solutions and Arenadata products, including Arenadata DB. Read more here.

Obtaining an FSTEC certificate

Arenadata DB (ADB) received Certificate of Conformity FSTEC RUSSIAN FEDERATION No. 4675 dated June 2, 2023.

The document certifies that Arenadata DB is certified according to the requirements of the 6th level of trust and technical specifications. According to the certificate, the ADB meets the information security requirements established in the document "Information Security Requirements Establishing Levels of Trust in Information Security Tools and Information Technology Security Tools."

On the basis of technical conclusions, Arenadata DB was entered into the state register of the certification system for information protection tools in accordance with information security requirements dated June 2, 2023. The received FSTEC certificate is valid until June 2, 2028.

The FSTEC certificate clearly demonstrates the level of maturity of Arenadata DB and reflects the wishes of users to domestic solutions. Among our customers there are representatives of government companies and departments, critical information infrastructure (CII) facilities. They are subject to special information security requirements, including the use of certified security equipment. With the help of the certified Arenadata DB, our customers can fulfill such instructions, - commented Alexander Ermakov, Technical Director of Arenadata.

2022: Gagar Server and Disk Array Compatibility Testing > n

server The equipment manufacturer GAGAR>N , together Russian with the data platform developer Arenadata and IT the company "," T1 Consulting tested the equipment for working Big Data with and confirmed the effectiveness of its use for creating domestic complexes storages and processing. big data This was announced on July 26, 2022 by representatives of "."T1 Consulting

In connection with the increased need of customers to build an IT infrastructure based on Russian T1 Consulting hardware and software systems, he organized the development of PAC using Arenadata DB and Arenadata Hadoop software on the x86 server platform and GAGAR > N disk arrays. Before forming a ready-made proposal, specialists conducted testing to verify the compatibility of computing equipment with software solutions. Read more here.

2021: Cloud Availability Mail.ru

On July 9, 2021, Mail.ru Cloud Solutions (MCS) announced an expansion of cooperation with Arenadata. The Arenadata DB cloud analytics database based on Greenplum in Enterprise has become available on the MCS cloud platform. Platform users will be able to test this solution for free for three months.

The Arenadata DB Cloud version of Enterprise can be integrated with other Mail.ru Cloud Solutions services - for example, raw data storage, storefronts, orchestration systems, and so on. This approach allows you to form an analytical platform for solving problems of any complexity, which will become the infrastructure basis for working with big data and will expand the client's local systems with advanced cloud capabilities. Thanks to the built-in ClickHouse and Kafka connectors, the integration of the Enterprise version will be much easier than the open version of Greenplum.

Advanced support and training from the solution developer will be available to users of the cloud Enterprise version. Mail.ru Group and Arenadata will advise customers on building big data platforms and help integrate with other solutions, including those deployed on the local infrastructure. Users of the Mail.ru Cloud Solutions platform can get free access to Arenadata DB in the Enterprise version for three months, leaving a request on the solution page.

We offer our customers technologies in the form of ready-made solutions. Arenadata DB Cloud version of Enterprise in conjunction with other MCS services helps to quickly create a full-fledged platform for working with big data. This analytical base allows you to work with Big Data in the cloud using simple SQL queries. Thus, big data tools are becoming more accessible than ever, "said Dmitry Lazarenko, Product Director at Mail.ru Cloud Solutions.

Arenadata DB version of Enterprise became available on the MCS platform as a PaaS service. Thus, the administration of the DBMS completely falls under the responsibility of the provider, who is responsible for its availability.

Services are also available on the MCS platform that allow you to increase the speed of Arenadata DB Cloud, such as Low Latency NVMe ultrafast drives. Their use, together with similar DBMSs, is effective for high-load systems in some industries (for example, banks and retail) and allows you to get a response rate of no more than 0.5 ms.

Among the first customers to start using the Arenadata DB as a service was Auchan. Arenadata DB Cloud will allow you to organize a data laboratory for various departments of the company: logistics, commercial management, marketing. They will have the opportunity to conduct research, use ad-hoc analytics, ad-hoc reporting.

The launch of the Enterprise version of Arenadata DB in the cloud Mail.ru Cloud Solutions allows the business to get a comprehensive platform for working with Big Data without effort. With special DBMS connectors for the most well-known data solutions, it quickly integrates with other MCS services and on-premium customer storage systems. Russian business receives a fully managed analytical DBMS of an enterprise level, which does not require a long configuration: experts from Arenadata and Mail.ru Cloud Solutions have already chosen the optimal thin settings for the solution, - emphasizes Alexander Ermakov, CTO of Arenadata.

2020

Use Low-latency SSD to increase bandwidth

The cloud platform Mail.ru Cloud Solutions (MCS) on December 16, 2020 introduced the next type of Low-latency NVMe disks on the Russian market - ultra-fast network drives technology.

The developer of the data collection and storage platform, Arenadata, used the Low-latency SSD solution to increase the bandwidth of the massively parallel Arenadata DB [[<L|database]]. Read more here.

It is critically important for us to ensure high performance of the disk subsystem, regardless of the load level, "said Alexander Ermakov, Technical Director of Arenadata. - When testing Low-latency NVME under high load on the cluster, there was no longer an effect of its fall, the stability of the disk subsystem and the servers themselves significantly increased. The response was not more than 0.5 ms.

Compatibility of Arenadata DB version 6.7 with Alt 8 SP OS

DBMS Arenadata DB is running in a secure "" environment operating system. Alt 8 SP This became known on August 31, 2020.

The created software stack will be primarily useful for organizations that are obliged to store and process large amounts of information in the environment of the Russian OS certified by the FSTEC of Russia.

Alt 8 SP OS and Arenadata DB DBMS are developing on the basis of free software. This makes it possible for the creators of the operating system, IVK and BASEALT, together with the developers of DBMS from Arenadata, to use the best practices of international projects, exchanging developments with them.

We have entered into a partnership with BASEALT and IVK at the numerous requests of our clients and partners, - comments Alexander Ermakov, Technical Director of Arenadata. - Our team conducted functional testing of Arenadata DB version 6.7 to expand the cluster and upgrade the system based on the Alt 8 SP OS. We plan to apply this experience to other "BASEALT" software products to provide customers with complete tools to work with.

The Alt 8 SP operating system allows you to create a trusted digital environment for working with limited access information containing service, commercial and other types of secrets, as well as personal data, - said Grigory Sizonenko, General Director of the IVK group of companies. - In combination with Arenadata DB, it will become the basis for the digital environment of organizations that need to analyze gigantic data sets and reliably protect the results.

Arenadata DB is one of the key software products in the vast ecosystem of software compatible with the Alt family of operating systems, "emphasizes Alexey Smirnov, CEO of BASEALT. - We are steadily increasing the ecosystem, first of all, by ensuring compatibility with the most developed and reliable software products.

In the future, Arenadata plans to confirm the compatibility of Alt OS with other products included in the unified Arenadata Enterprise Data Platform.

Arenadata DB 6.8.1 exit with updated Command Center

Arenadata on July 8, 2020 presented to the market an updated version of its analytical database management system - Arenadata DB. For Enterprise Bundle subscribers, a solution for advanced monitoring of requests and consumed resources is now available.

Arenadata DB Command Center (ADBCC) is software that will complement the Enterprise version of the product. It allows you to see in real time the progress of current requests, their plan, stages of execution, distribution by cluster and other valuable metrics, as well as view historical data. To update the service, subscribers are advised to contact their account manager.

The product has undergone several changes. Support for Arenadata Cluster Manager (ADCM) has appeared - you can connect Command Center in two clicks in the web interface. Added user types with flexible permissions to view and remove requests. Additionally, the team made a number of small improvements for user convenience and search accuracy.

With the Arenadata DB Command Center, customers operating DBMSs in product environments can significantly reduce the cost of writing and optimizing complex queries. We tried to make Command Center what we would like to see the perfect plan visualizer. All two months that the product was in a closed beta within the company, our developers used it to analyze requests, and they were satisfied. Of course, much remains to be finalized, and we hope for the help of our users in determining the necessary functionality. Dmitry Pavlov, Arenadata's chief product officer, added.

Cloud Mail.ru Cloud Solutions Launch

On April 29, 2020, it became known that Mail.Ru Cloud Solutions (MCS) is launching Arenadata DB in the cloud as a service - an analytical database based on the distributed open source database Greenplum, designed to store and process large amounts of data and easily scalable within a single interface without rebooting the cluster. The service allows you to deploy a database for storing and processing big data in a few clicks without investing in your own infrastructure and its support.

Arenadata DB is a fast cluster solution that can deploy distributed databases. It allows you to store and process large volumes of structured and semi-structured data and build models based on them, for example, for BI. Unlike the use of similar analytical databases On Premises, Arenadata DB as a service allows you to speed up the construction of complex analytical queries up to 5 times thanks to the ability to quickly scale up to hundreds of nodes in the MCS cloud infrastructure.

Cloud access to the database frees the business from the need to buy servers, build and maintain infrastructure, purchase licenses: in MCS, the company pays only for used capacity with per second charging.

As part of Mail.ru Cloud Solutions, we give companies cloud access to the most popular databases, and now the list has been replenished with an analytical database. We take over infrastructure and administration - and our customers will be able to focus on solving business problems, - comments Ilya Letunov, head of the Mail.ru Cloud Solutions platform.

We see that recently Greenplum has been actively spreading: it is being introduced by an increasing number of Russian companies of various sizes. The ability to take advantage of it in the cloud pushes the boundaries for those of them who could not try out the product on Bare-Metal. The advantages of this approach, for example, elasticity and time payment for the power used, have already been appreciated by many companies. In our opinion, Mail.ru Cloud Solutions was able to build a reliable and convenient cloud, becoming one of the leaders in working with Enterprise. We hope that our joint product with MCS will bring a number of innovations to the Russian market and help customers achieve their goals, "says Dmitry Pavlov, Product Director of Arenadata.

One of the main advantages of the service is close integration with other Big Data services available in the cloud platform Mail.ru Cloud Solutions and used to build data lakes (Data Lake): Hadoop, Spark, Airflow and Kafka.

MCS clients working in the cloud with Cloud Big Data, when connected, Arenadata DB will also be able to significantly reduce the data transfer time between solutions, since they are within the same network.

Regarding MPP systems (Massively Parallel Processing), Arenadata DB is easily scalable: for this it is enough to add physical or virtual servers to the cluster. The MCS client will be able to install and configure all the necessary databases in one interface to organize data storage and work efficiently with Big Data.

When connecting the Arenadata DB cloud, companies can choose the version to use: Community, Enterprise Trial or Enterprise. During the first month, when Arenadata DB aaS is connected on demand, companies are also given the opportunity to try the Enterprise version besplatno.​

Based on the cloud service for analyzing big data from CROC Cloud Services

Croc Cloud services in partnership with Arenadata are launching a service based on Big Data technology. Using the Arenadata DB cloud system, large companies will be able to process current and complete data without significant investment in their own IT infrastructure to launch services and improve customer experience. This became known on February 11, 2020.

Big data technologies are increasingly used in the corporate segment for business analytics and marketing tasks, especially in industries such as telecom, retail, finance and industry. Companies seek to identify hidden patterns and use them to grow their business and increase profits.

For example, using big data analysis, you can launch promotions for specific goods based on the weather forecast, offer suitable tariffs depending on the geolocation of the subscriber, predict the failure of industrial equipment using information obtained from sensors at the facility, and increase the efficiency of credit scoring. Using such solutions from the cloud makes them more accessible to a wide range of organizations.

The launched KrokCloud services service will be in demand both by customers who are just starting to implement a big data analysis system, and by those companies that already have experience with Big Data solutions. Unlike on premiere solutions, Big Data class cloud systems do not require investment in the operation and modernization of their own computing resources.

This cloud service is based on the Arenadata DB product - an enterprise solution certified by the global community of system developers open source with (ODPi, Open Data Platform). This is a Russian development adapted to work in the cloud. Arenadata DB perfectly copes with the tasks of building corporate (data stores QCD), analytics using - BI intuals, AD-HOC queries data science and enterprise level. As the amount of data grows, the user simply adds servers to the cluster - the ADB will take care of everything else itself.

Business interest in Big data is very high. In the market, we see more and more practical cases of applying the technology in marketing, managing customer experience, and in product development. For companies that are looking for an effective predictive analytics tool, reporting and customer outflow analysis, such services will be interesting for their convenience and economy. The capabilities of the service and its potential for business can be tested on demand - for this we will configure Arenadata DB depending on the customer's requirements, told Sergey Zinkevich, product manager of CROC Cloud Services.

We are confident that cooperation with CROC Cloud services will benefit both us and the provider, but primarily users of the Arenadata product. They will be able to choose exactly how to use the data analysis solution - in their infrastructure or from the cloud. And by choosing SaaS, customers can be sure of the constant availability of the service and the other benefits that our partner guarantees: the quality of support, the scalability of the cloud infrastructure, cost transparency and access to additional services, such as setting up and creating network connectivity. noted Anton Martynov, Arenadata Partner Director

2019

2018

Availability in Kazakhstan

On November 14, 2018, the company, a Arenadata Russian developer of a multi-purpose data platform, announced availability in the Kazakhstan Russian distribution (ADH) market Arenadata Hadoop , as well as analytical massively parallel DBMS Arenadata DB. More. here

Arenadata DB. Description

As of November 2018, Arenadata DB (ADB) is relational DBMSs with a massively parallel Shared Nothing architecture designed to store, process and analyze large amounts of structured and semi-structured data. Using the processing power of hundreds of servers, an advanced query optimizer and a flexible data backup system, ADB can significantly improve performance and reliability by preserving legacy ANSI SQL applications (fully compatible with PostgreSQL) access to data.

The ADB architecture is a classic cluster: several servers-segments, one master server and one standby, connected to each other by fast networks (10G Ethernet or Infiniband). Each server segment has several segments (instances) PostgreSQL containing data. In the event of a failure of one or more segments, they are marked as failed and instead their mirror segments are launched, replication data for which occurs using the technology used in PostgreSQL DBMS (Wright Ahead Log, WAL - all changes to tables and indexes are written to the file only after they are logged).

The use of several interconnects allows you to increase the bandwidth of the channel of interaction of segments with each other and ensure the failover of the cluster by redistributing traffic. The distribution of segments across network interfaces is selected individually and can be adapted to cluster tasks - for example, all main segments can be forced to use one network interface, while backup segments will use the second.

In ADB, a classic data separation (sharding) scheme is implemented - each table consists of N tables placed on N cluster segments. The logic of dividing the table into segments is set by the distribution key (field). You can specify a different type and compression level for each individual column in the table. In addition to the types of compression initially available in Greenplum - zlib (one of the most widely used compression libraries, in particular, is used in Linux distributions) and RLE delta compression - ADB has a zstandard algorithm developed by Facebook and implemented by the Arenadata team, which provides almost four times higher performance than zlib.

ADB uses polymorphic data storage, for example, one table can be divided into vertical sections (batches), some of which will be stored as rows and some as column objects. In this case, for the user, such a table will look like one object.

Security in ADB is achieved by encrypting data and server-to-client connections over SSL at all stages of their lifecycle. In addition, all internal interactions of ADB DBMS components (segments, mirrors and wizards) can also be encrypted using SSL, and data stored on cluster disks can be encrypted using PGP keys (at the table or column level in tables). All this allows you to exclude situations of finding data in unencrypted form.

Data visibility and access rights are delimited by the Role Based Access Control (RBAC) role model, which allows you to implement flexible access delimitation rules that change dynamically during the operation of the storage platform and data processing. For example, you can create schemes to restrict access to tables and other DBMS objects, as well as to rows and columns of individual tables.

ADB implements a parallel data exchange protocol with third-party systems - PXF (Platform eXtension Framework), which provides interaction with the external system at the same time of all cluster segments. If the source system is also a cluster, then cluster interaction on both sides can be used, which increases performance, and the speed of interaction will increase as clusters expand.

A flexible backup system allows you to deploy a cluster with a predetermined level of fault tolerance, allowing the DBMS to work even if half of the servers out of the cluster fail. And more ADB storage strategies provide the performance you need at every stage of the data lifecycle - from getting new online data, storing master data with different levels of compression, to exporting archived data to a Hadoop cluster.

The ability to integrate ADB with other systems allows you to use this DBMS to build universal storage and processing platforms, such as the Arenadata Enterprise Data Platform (EDP), an open, scale-out solution for storing and processing large amounts of data of any type. The platform works with loads from OLTP to OLAP, supports access to data in both SQL and working with Python libraries.

The Arenadata EDP platform consists of three closely related components using a parallel access framework: the Arenadata Hadoop cluster, the ADB cluster, and the Arenadata In-memory Grid cluster. In the ADB DBMS, tables are created, data sources for which are both data from the DBMS itself, data from the Hadoop HDFS cluster and data from the In-memory Grid cluster RAM. Nifi, an open ETL/ELT processor, is used to control internal processes and data loading processes, and Apache Zeppelin is used to access user data and their analytical processing.

To effectively use the DBMS, management and monitoring tools are required - the ADB has a package of administrator tools: monitoring software, DBMS management and sending notifications.

High speed of processing complex requests, linear scaling, lack of specific hardware requirements, open source code, integration flexibility allow Arenadata DB to be used as an analytical data store for corporate information systems.