Data Fabric

The concept of Data Fabric and the main elements of this concept

The introduction to the concept of Data Fabric for TAdviser was prepared by Svetlana Vronskaya, author of the telegram channel Analytics Now.

Data Fabric, which for some reason is often incorrectly translated as a "data factory," has nothing to do with the plant. Data Fabric is a data fabric, and it is a complete information management architecture with full and flexible access to work with it.

It is a standalone ecosystem that is used to maximize access to enterprise data. Data Fabric makes it easier to search, process, structure, and integrate information with other information systems.

What does Data Fabric consist of? Firstly, at each step of information processing, machine learning is used - from the analysis of the obtained data to the optimization of their processing algorithms. In addition, all data consumers, all data sources are connected through end-to-end integration (including through databases and data warehouses and other Data Lake) through APIs. Instead of a single block of the software solution, a micro-service architecture is used. The primary focus of IT is on cloud solutions, information flows are orchestrated, and information quality is improved after unification and virtualization. In addition, data is easily accessed and secured for each of the user groups.

The Data Fabric architecture works in the DataOps concept. Quickly organize responses to any changes in the data, increase the level of forecasting, optimize the processes of storing, processing and servicing resources.

A distinctive characteristic of Data Fabric is the active use of Big Data and artificial intelligence technologies, in particular, machine learning to build and optimize control algorithms and practical use of data. In addition, the Data Fabric concept is complemented by semantic graphs that allow you to define, standardize and agree on the meaning of all incoming data in business terms that are understandable to end users.

Simply put, a data fabric is a system at the level of your entire organization, where everything is subordinate to data and conclusions based on them.