Vesoft: Nebula Graph DBMS

Main article: Database Management System (DMS)

2023: Chinese DBMS developer used by China Mobile and WeChat officially comes to Russia

In January 2023, Vesoft, the Chinese developer of the open source graph DBMS Nebula Graph, launched a distributor in Russia - Factor Group. In this status, the company will represent the interests of Vesoft not only in Russia, but also in the near abroad.

The Factor Group explained to TAdviser that they serve as the official regional representative of the vendor, the center of competence and technical support in the region in Russian. This includes working with customers, partners, developing partner programs, organizing training for customers and partners, promotion/development/consulting/implementation/support in Russian. In addition, the Factor Group is also developing an Open Source community of users of the open product branch.

Nebula Graph is a relatively new solution in the market: the first stable release of this DBMS took place in mid-2020^[1]. However, by 2023 it already has large users. Among them are companies such as WeChat, China Mobile, Tencent, Meituan, JD Digits and Kuaishou. Nebula Graph is used in the work of social networks, recommendation systems, knowledge graphs, cybersecurity, investments, AI, etc.

The DBMS is distributed in a free edition (Community Edition) and a commercial version (Enterprise Edition). The latter provides more functionality and professional technical support.

Global analysts record an increase in demand for graph databases in the world "(photo source - Nebula Graph)"

The collaboration between Factor Group and Nebula Graph will allow both companies to deliver advanced graph computing technologies to customers in the region without any restrictions and provide better database support to promote digital transformation of organizations in various industries, says Allen Du, vice president of sales at NebulaGraph.

Nebula Graph uses the RocksDB storage mechanism to provide low latency, high bandwidth read and write. The developers claim that their solution provides parallel access, fast graph processing and efficient memory use, can store and process graphs with trillions of edges (links) and vertices (objects).

Graph databases are sometimes distinguished as a separate class of solutions, although most often they are classified as a broader class - NoSQL. A feature of graph databases is the ability to process graphs with minimal delay, which is extremely important for solving various problems in the fields of fintech, AI, information security, etc. They are used, for example, for risk management tasks, fraud detection, for client analytics and recommendation systems.

Analysts at The Insight Partners note an increase in demand for graph DBMS in the world and predict the average annual growth rate of this market in dollars by about 22% in 2022-2028, from $1.85 billion to $7.22 billion. The need for graph DBMSs spurs, among other things, an increase in demand for systems that can cope with processing requests with minimal delays, according to The Insight Partners^[2].

And Gartner predicts that graph technology will be used in 80% of data and analytics innovation by 2025. In 2021, according to analysts, this figure was 10%^[3].

Among the key global players in the graph DBMS market are companies such as Amazon Web Services, Oracle, Teradata, IBM, Microsoft, Callidus Software, MarkLogic Corporation, Neo4j, OpenLink Software and several others. One of the most famous graph DBMS - a freely distributed Neo4J, was released back in 2007.

Many foreign players left Russia in 2022. Nebula Graph will be able to make a competitive replacement for similar solutions, the Factor Group believes.

If we talk about the differences between graph databases and SQL databases, then in general, graph databases are a rather niche solution, notes Viktor Smirnov, director of business development in the field of creating CROC software. For example, unlike document-oriented ones, which have gained popularity in terms of distribution of NoSQL. Such databases have a rather limited scope, where they can be really effective relative to traditional relational databases. And here, of course, we are talking about really large amounts of data.

The main example is the social graph, which is implemented by social networks familiar to us. Or other systems where you want to consider many predefined criteria when building analytics. And to do this, you need to store a large number of relationships between data. And just the graph database can manage interconnected data much more efficiently. The prospects for application are seen mainly in replacing the solutions of Western vendors who have left the market, - notes Viktor Smirnov.

A graph in mathematics is a network of related objects. Graph databases are designed to store and study information about such networks, explains Ivan Panchenko, deputy general director of Postgres Professional. For example, they allow you to quickly respond to requests such as "what is the closest route between Johannesburg and Anadyr" or "find mutual friends of Ivanov and Petrov." Therefore, graph bases underlie social networks, where connections are formed between users according to a large list of criteria.

However, ready-made boxed solutions of the social network, including the Russian VK, are not used, preferring their own databases, says Ivan Panchenko.

Graph bases are also often used in knowledge bases, science, and above all, in the study of social phenomena.

Since the count bases in our country have long been widely used, the emergence of a Russian distributor of another DBMS (by the way, freely distributed) will not radically change the landscape, although it will improve the conditions for working with this DBMS in Russia, - believes the deputy general director of Postgres Professional.

2022: Nebula Graph 3.2 Graph-Oriented DBMS Release

On July 19, 2022, it became known that the release open DBMS of Nebula Graph 3.2 was published, designed for efficient storages large sets of interconnected, data forming a graph that can number billions of nodes and trillions. communications The project is written in the language C++ and distributed under 2.0. license Apache Client libraries for accessing DBMS are prepared for languages, and. Go Python Java

The DBMS uses a distributed shared-nothing architecture, which implies the launch of independent and self-sufficient processes for processing graphd requests and storage processes stored.information The meta-service is engaged in orchestration of data movement and the provision of meta-about the graph. algorithm A RAFT-based protocol is used to ensure data consistency.

The main features of the Nebula Graph are:

• Provision safety by granting access only to authenticated users whose privileges are set through the Role-Based Access Management System (RBAC).
• The ability to connect different types of storage engines. Support for extension of query generation language by algorithms.
• Provide minimal latency in reading or writing data and maintain high throughput. When tested in a cluster of one graphd node and three nodes, the 632 GB stored database, including a graph of 1.2 billion vertices and 8.4 billion latency edges, was at the level of several milliseconds, and the throughput was up to 140 thousand requests per second.
• Linear scalability.
• SQL-like query language that is powerful and easy to understand. Operations such as GO, GROUP BY, ORDER BY, LIMIT, UNION, UNION DISTINCT, INTERSECT, MINUS, PIPE (using a result from a previous query) are supported. Indexes and user-defined variables are supported.
• High availability and fault tolerance.
• Support for creating snapshots with database status slicing to simplify backup creation.
• Ready for industrial use (already used in the infrastructure of JD, Meituan and Xiaohongshu).
• The ability to change the storage scheme and update data without stopping or affecting the operations being performed.
• Support for TTL to limit data lifetime.
• Commands for managing storage settings and hosts.
• Tools for managing work and scheduling start-up (COMPACT and FLUSH are still supported from work).
• Searches for the full path and the shortest path between specified vertices.
• OLAP interface for integration with third-party analytics platforms.
• Utilities for importing data from CSV files or from Spark.
• Export metrics for monitoring with Prometheus and Grafana.
• Nebula Graph Studio web interface for visualization of graph operations, graph navigation, design of data storage and loading scheme^[4].

Notes