AI in analytics: What's beyond BI?

The Business Intelligence (Business Intelligence) software market BI is one of those that shows convincing double digits of growth. So, according to data to a special study conducted by the company, the Navicon market for domestic analytics systems will grow this year by 10-12. For comparison: the global BI market, researchers predict stable growth in the coming years at the level of 8-10% per year. But already in 2023, when more successful migration cases appear on our market, and domestic software the business ceases to fear the problems of "raw," the software market development rate will increase to at least 30-35%, analysts at Navicon say.

Specifics of the development of the Russian BI market

The unconditional driver of the increase in the Russian BI market is the needs import substitution states and business. RUSSIAN FEDERATION Navicon notes that the demand for import-independent the Russian solutions from customers has grown by 50%, and about 40% of customers announced plans to migrate to the domestic one in the ON near future. First of all, we are talking about segments, and. banks retail telecom

However, the positive impact of import substitution, which began this spring, was a continuation of the increased attention to analytics that appeared during the pandemic. Large companies that had accumulated significant amounts of data by that time were looking for ways to more effectively manage resources and financial flows, establish logistics, Navicon says, while giving preference to the leading three foreign vendors: Microsoft, Tableau, QlikTech. The share of domestic software in the Russian BI market until February 2022 was, according to Navicon estimates, no more than 10%.

The situation changed when foreign vendors began to stop their activities in Russia. And in this there are, in addition to the opening window of opportunities for historical vendors, certain risks: large customers are accustomed to a certain level of service and reliability of global brands.

At the same time, Sergey Gromov, an expert in the field of BI, believes that in many ways domestic platforms have reached a certain level of maturity and in the current conditions can become an effective alternative to the solutions of foreign manufacturers who have left the market due to sanctions. In April, he presented the third part of the annual study "Gromov's BI Circle," traditionally dedicated to BI systems created by Russian developers.

Sergei Gromov emphasizes that the next testing was carried out in conditions close to "combat," that is, real use in business. To this end, the tasks that solve BI systems in retail were used - in this segment business analytics has been implemented for a long time and actively.

The study covered 40 domestic solutions, including: Visiology, Modus BI, Luxms BI, Forsyth. Analytical Platform, Triaflai, DataLens, Krista BI, PolyAnalyst, N3. Analytics, Polymatica Analytical Platform, Alpha BI, Analytical Workspace, Almaz BI, Cubisio, Visary BI, 1C: Analytics, Pulse Affairs/Pulse Region, Dataplan and others.

Domestic BI platforms, although they are mostly young, have developed in recent years in conditions of rather fierce competition with the decisions of leading global vendors, says Sergey Gromov. - This could not but have a positive impact on the quality and functionality of Russian solutions.

Thus, there is room for growth, each platform can find weaknesses, but in general it was possible to achieve a lot, the expert believes, and this should facilitate the migration of organizations to domestic solutions.

A characteristic feature of the Russian BI market is a high share of customers from the public sector and a company with state participation. It was they who provided the demand that provided the basis for the development of domestic developments, comments Lyudmila Ostrovskaya, project manager of the Integrated Design of Information Systems Department of FORCE - Development Center. At the same time, they will compete on the field freed from foreign vendors not only with each other, but also with free software. Indeed, confirms Dmitry Sysoev, director of the management consulting department of Norbit, in recent years there have been many quite advanced BI-products using open source software.

The characteristics of an advanced analytical system include a wide range of analyzed data.

On the way to the Data Factory

As a fresh example of a developed analytical system, the Poseidon system launched at the end of April can be cited. President RFVladimir Putin signed Decree No. 232 on the Poseidon state information system in the field of anti-corruption on April 25, 2022. This GIS will control the income and expenses of officials, carry out prevention in the fight against corruption, and be used to disclose other offenses. Earlier, to control the income of civil servants, the Help-BC software platform was used, with the help of which civil servants formed income declarations.

Decree No. 232 provides for information interaction of the Poseidon GIS with other information systems, information from which can be used to combat corruption, including federal state bodies, government agencies of the constituent entities of the Russian Federation, the Bank of Russia, state corporations (companies), state extra-budgetary funds, and other organizations. The Poseidon system will carry out an automated analysis of the entire body of available information.

Another example is a project to create an intelligent system for managing fields and field data, which is carried out by Etton Group of Companies. The system will process all information from the wells, from well characteristics to production parameters. Based on a comprehensive analysis of the data, the analytical system will propose an optimal well operating mode. At the same time, a whole range of intelligent mechanisms is used for data processing: algorithmic machine learning, data management scenarios (rule-based management), creating models to increase the speed of calculations. Even an internal declarative language is provided that simplifies descriptions of data processing operations.

The developers of the system say that it will be able to identify patterns that connect the data with each other, and calculate the forecast for all parameters during the planning of various methods of impact on the well or formation. In general, digitalization processes accompanied by data accumulation have become a driver for the development of the so-called Data Factory. In fact, these are ecosystems that combine data collection from a variety of sources and information systems, pipelines for transferring and processing information, APIs. Analysts Gartner in 2020 called Data Factory one of the key trends in the field of Data Analytics.

For example, the Delta BI system developed by Navicon is built on this principle. It provides an integrated approach to working with data. In particular, the basic supply includes more than 200 ready-made connectors for various information systems. The company says that the system implements the processing of "broad" data, including both sources of both structured and unstructured data, in order to increase awareness of the context and decisions made.

Source: Navicon Company

Delta BI, according to developers from Navicon, makes it possible to make management decisions literally in real time, and the Self-service approach allows business users to work with analytical tools.

The hype wave has already passed. Companies understand what business effect can be expected. Technologies and approaches are understandable and known to everyone. Specialists are still very expensive, but are no longer rare. Therefore, there is no need to talk about breakthroughs in the near future, rather the period of adaptation of what we already know how is ahead.

Data for Today's Analytics

The reason is their availability, reliability (or ability to soberly assess it) and relevance (temporary and contextual) in relation to the business task.

According to the expert, our market as a whole is more characterized by an insourcing model IT and a less mature market for external data, due to which their use is more limited compared to the western market. Indeed, often existing internal data is more detailed and the restrictions on their use are less regulated by legislation (on personal data, for example) than external ones.

However, some of the tasks or simply cannot be solved or they are solved much less efficiently without external data, Alexander Khledenev points out, for example, forecasting sales based on internal historical data without taking into account macroeconomic data or forecasts of seasonal sales without taking into account the factors that form this seasonality. A certain watershed can be drawn between the analysis of internal, operational activities and external business communications, which include client analytics (micro-segmentation and personalization, scoring of customers (leads), product analytics (price, assortment analysis), sales analytics, market analysis, etc. "At the same time, external data is obtained from open sources (open databases, state statistics, demographics, etc.), collected from social networks, bought from supplier companies or on specialized marketplaces, and also obtained as a result of an exchange between companies (integrated or partners)," the expert notes.

Smart scoring models designed to analyze the creditworthiness of a potential borrower have advanced far enough in this part. bank For example, credit institutions that create their own ecosystems receive much more data about their customers and their potential opportunities than fellow competitors. But they also strive to fill their data lakes additional information. It comes from information : social networks the composition of friends, which of them is already a client of the bank, in which groups a person is, how often he goes to social networks, confirmation of family ties, etc., they say in the company "VS Lab."

There is also information about the employer, both from databases SPARK(,, Unified State Register of Contour Focus Legal Entities and Unified State Register of Legal Entities) and from MEDIA social networks. From the databases, information is taken about the state of the company, its activity in, state procurements known data on the turnover and number of the company. Rumors about the deteriorating or improving situation of the company, scandals related to the director and founders are collected from the media and social networks - at the same time, things that can affect scoring stand out. For example, a person has an excellent, salary good credit history, but the media say that information his employer is, the accounts bankrupt are frozen, and soon all employees will go outside. This, of course, will lower the scoring score for such a person.

This data is provided either by paid services or various OSINT systems (Open source intelligence - information collection and analysis of intelligence based on open sources). Information is automatically taken from OSINT systems algorithms AI in marked-up form.

Also, sources can be data from exchanges engaged in the sale of cookies, because they collect all the interests of the bank's client, his search queries and much more. With this data, for example, AI can find out that the borrower is a regular client of an online casino, and at the same time came to receive a loan - the bank will respond accordingly.

Data from mobile operators, for example, on the movement of a particular phone number, its activity, who and when calls him, how long this number has been for this subscriber - all these are commercial services that sell data in large volumes, specialists from VU Lab notice: these data are embedded in the AI mechanism, supplementing and enriching its model.

The main reason why AI achieves amazing results in this area is the large amounts of disparate data that are collected and analyzed by different components of systems, says Alexander Khledenev. - The more data, the more sophisticated combinations can build AI, reveal more patterns and connections, learn the maximum necessary information about the client.

An example of such a system HFLabs KYC is a company solution HFLabs that belongs to the class "." Know Your Customer The system automatically checks customers and counterparties against sanctions lists and the official register before the financial transaction is carried out. She works with all lists, Rosfinmonitoring performs the check necessary for 115-FZ ("anti-money laundering" legislation), finds people included in the sanctions lists USA,,,,. European Union UN Switzerland You can find out Great Britain if a person is bankrupt and has not presented an invalid passport.

Each individual request is processed in less than a second. And the speed of regular verification for the entire client base, even if it is 50 million people, is about three hours, the company says.

The system will see that the husband of a potential borrower has already overdue two loans, and will not give permission to lend. The daughter of the client who insured the meat processing plant crashes the car every month, but the company will still approve CASCO.

Source: HFLabs

Another example of smart continuous analysis big data is an intelligent system for monitoring competitors' prices. PowerPrice This development of the company Napoleon IT is being implemented, in particular, Tyumen in the company. ALOEsmart

The Napoleon IT PowerPrice platform provides AI customers with a continuous stream of competitor price data and automates the pricing process by determining optimal prices according to a given strategy. To obtain data on competitors' prices, computer vision technology is used: using a mobile phone, you need to take several photos of shelves with products, and the system will extract data on goods and their price. This mobile application, integrated with the ERP system, helps to quickly collect data on current prices and promotions of other stores and immediately act in accordance with current data.

Napoleon IT says that today the system recognizes 3 million images a day.

Alexander Khledenev from VS Lab believes that, from a technological point of view, continuous analysis of internal and external data in real time, comparing them with historical data, followed by embedding the results in business processes (the concept of Continuous Intelligence) can be called one of the most significant achievements of modern data processing processes. He cites as a unique example the experience of Hershey, which during the pandemic found an increased demand for one of its products: S'more is a traditional American dessert that children cook at the stake in the yard. The company predicted the timing and volume of its consumption. And, contrary to previous strategy and the risks of cannibalizing sales before Easter, it filled store shelves with these products in regions of the United States where people found themselves tied to their households.

In other words, the company was able to increase production, stocks, as well as quickly track the reaction to its marketing company and change the appeal to consumers to a more adequate situation. And this led to an increase in sales by 70 million. dollars the company's main product is chocolate bars, which people bought in addition to dessert.

Trust in AI

Today, on the agenda is the problem of trust in the work of intellectual mechanisms and complex models. We can say that some technological limit has been reached for the complexity of problems that AI is able to solve with humans. Indeed, companies are striving to include more and more data of different types in analytical processing, an increasing number of employees are involved in working with data in order to make even more effective and timely decisions. The data should remain up-to-date, accurate and consistent, as well as correctly used by all participants in the process. The correct operation of a complex model, especially one built using machine learning, is a separate problem that does not yet have a complete and final solution.

In the public sphere today it is customary to focus on the so-called "problem of alignment." We are talking about the alignment of algorithms for data analysis and decision-making by the AI program and a person brought up in line with universal human values. The "leveling problem" for Americans is as follows: the system automatically collects resumes, and years later it is found that gender biases are "protected" in the selection criteria. For Russia, this problem has the form of a choice task for an unmanned car: who to crush in the event of a difficult road situation, if there is no other way out: an old man or a child? But the point, of course, is not the so-called ethics of AI.

It has not yet been possible to create an interpreted and trusted AI (trusted AI), says Mikhail Dudalev, head of the data analysis department at Fuzzy Lodge Labs, that is, at the moment there are no approaches to ensuring that models of any complexity can be guaranteed that they will work adequately in any conditions and give correct results. Even in banks, no one will undertake to 100% guarantee that AI did not miss the fraudster's credit application containing a specially selected set of parameters. Timur Aitov, deputy chairman of the Chamber of Commerce and Industry's committee on the financial market and credit institutions, jokes that in this sense, AI can only be believed.

Work on the explanatory function of ML is carried out, for example, by Qlik. Last summer, it acquired NodeGraph, a developer of customized metadata management solutions, to increase data transparency and thereby trust in data as a source of decision-making. The integration of NodeGraph with the Qlik platform will enhance its capabilities through interactive Data Lineage, performance analysis, and data governance. This will allow Qlik customers to get a more complete overview of the structure of data pipelines: from source systems to transformation and use, Qlik says, that is, it will contribute to the implementation of an "explainable BI."

The depth of understanding of data in NodeGraph will help companies expand access to data, according to the company, and will also provide programs for upgrading analytics with its transfer to the cloud. For this purpose, easy connection of NodeGraph to cloud platforms is used: AWS, Google Cloud, Microsoft Azure and a number of analytical products of third companies, including SSIS, Snowflake, Microsoft Power BI, Tableau.

According to Qlik representatives, the acquisition of NodeGraph contributes to the development of the Qlik Active Intelligence concept: decisions are made based on the analysis of verified data updated in real time, which increases the value of data in the business.

Trust in data, in turn, is a key element in creating an Active Intelligence culture, in which data becomes an integral tool in solving business problems, "said Mike Capone, CEO of Qlik.

Perhaps the organic integration of analytical operations into business processes should be considered the main sign of the modern stage of development of BI systems. According to Gartner's forecast, 75% of enterprises by the end of 2024 will have "working" solutions based on AI.

Decentralization in Analytical Computing

According to experts, a noticeable technological trend in the coming years will be the transition from factographic analysis (what is the essence of the situation?) To predictive analysis (what can happen?) And scenario modeling (what will happen if...?), As well as further improving the search for dependencies in data and assessing their impact on business. In other words, business intelligence in the sense of Business Intelligence becomes one of the key business functions directly integrated into the business decision-making system at all levels of the organization. The tasks inherent in BI for collecting data and visualizing them in the form of dashboards remain in the past. And what is ahead, beyond the borders of BI?

Analysts at PWC are talking about three waves of customer analytics.

During the first wave several decades ago, businessmen formulated questions to experts who organized the search for the necessary data and their processing. The second wave is characterized by various options for centralized data collection, analysis and reporting on a top-down basis. The third wave is associated with the emergence of personal computer tools and corresponding tools such as spreadsheets.

Now is the time for decentralized analytics, with faster data processing, and technology for storing and scaling data in distributed environments. Innovation in decentralized analytics has evolved faster than the respective centralized options, a trend PwC believes will continue in the future.

Source: PwC, ABBYY, 2013

In line with decentralized analytics, experts talk about the significant role of boundary computing. Indeed, the emergence of more and more IoT applications that require low latency, autonomy, and security becomes critical to physically bring analytical computing closer to where data is generated and used.

Gartner researchers speak in their report "Top 10 Trends in Data Processing and Analytics," published in 2020, about neuromorphic chips that will bring artificial intelligence to the boundaries of IT systems and deploy it on peripheral equipment.

PWC believes that the use of methods for working with large amounts of data, in particular NoSQL and in-memory databases, advanced statistical packages (including, for example, open source software, based on the R language), visualization tools with interactive graphics capabilities and more intuitive user interfaces than before, is key to the next generation analytics.

The movement beyond classic BI systems today is seen in several directions related to the use of intelligent mechanisms for business analysis.

Decision Intelligence

Decision Intelligence (DI) is a decision-making technology that combines key knowledge from the applied direction of data science, social sciences and management science. That is, DI, in addition to quantitative ones, operates with qualitative, "emotional," factors, says Emil Hasanov, product manager of the IT company Nauka, in her article "Decision Intelligence: artificial intelligence with a human face, "IT Manager magazine, January 2022. With its help, company leaders can make decisions in accordance, for example, with the political situation or sentiments in society or other details of environmental reality that are important for a particular segment of activity. This is the fundamental difference between DI systems and BI systems: they are based on knowledge about business, and not on the perfection of algorithms.

Source: it-world.ru/cionews/business/182334.html

From the point of view of the AI technology stack, the DI system includes:

• Machine learning (for algorithmized processing of structured data and formation of solutions according to specified parameters).
• In-depth training (to develop proposals taking into account previous decisions and their results).
• Visual simulation of solutions (visual analytics).
• Modeling complex systems (quickly building complex business logic based on available data, rules and goals).
• Predictive analytics (creating the most accurate predictions by building self-learning mathematical models).

DI is an attempt to enhance the benefits of AI with the capabilities of human thinking, which opens up a number of business benefits, says Emil Hasanov.

They give an example of the task of improving the accuracy of solutions for streaming services.

Existing AI software products generate sentences based only on a person's preferences, which are extracted from information set by the user himself, browsing history, and search query data. DI can add consideration of external factors, for example, time of year, current weather at the place of stay, upcoming holidays, etc. At the same time, at the initial stage, employees are engaged in the formation of input data and setting a goal. And then the system automatically analyzes the data and generates solutions.

Source: it-world.ru/cionews/business/182334.html

Navicon positions its Delta BI system as a representative of the Decision Intelligence class. It was originally developed as an IT solution focused on supporting management decisions online, which is extremely important in conditions of economic instability. It uses composite data from many data sets of various types and AI solutions to create a flexible, convenient, user-friendly interface, the company says.

Gartner predicts that by 2023, a third of large companies and corporations will have analysts specializing in Decision Intelligence. For those organizations that want to use Decision Intelligence tools in their organizations, Gartner reminds that solution management and modeling technologies, if they need to use multiple logical and mathematical methods, must be automated, documented and tested in practice.

Analytical Solutions for Business Analysis

The main direction for business intelligence systems today is the management task class described by the abbreviation BPF (Budgeting, Planning and Forecasting), which implies planning, budgeting and forecasting. Moreover, the most popular functions, according to Synthellis analysts, were the preparation of operational budgets and forecasts, as well as modeling scenarios in the "what if" format. To solve these problems, EPM BPMCPM(Business/Corporate/Enterprise Performance Management) class systems are aimed at analyzing corporate efficiency.

In fact, systems of this class ensure the interconnection of management processes at the strategic and tactical level through the automation of business processes on the basis of a single data warehouse. This approach implies an approach to supporting management functions based on a single data model, according to Intersoft Lab.

This level of analytical research requires technologies of a different level - Business Analytics (BA). Unlike BI platforms, which provide mathematical data processing, BA tools require the conversion of data into business indicators, which requires application functionality, says Julia Amiridi, Deputy General Director for Business Development at Intersoft Lab.

Industry experts see wide potential for the use of machine and AI algorithms in BA systems. A survey of European companies conducted in 2020 demonstrated the high interest of corporate financial services in planning and forecasting using machine learning.

Source: Upgrading BPF (budgeting, planning and forecasting) - trunk trend, it-weekly, 01.01.2022

In light of this trend, we can expect an increase in demand for automation of the predictive function using ML and AI methods in such datable industries as financial the sector,, etc., telecommunications retail according to Intersoft Lab.

Russian consulting group "Consist Business Group" presented its own product in the CPM/EPM category "TURBO Budgeting." As Ilya Dolgoborodov, Product Director of TURBO Budgeting, said at the Information Technologies in Budgeting conference, this is a universal designer for managing the entire budget cycle, which allows you to create forms, reports and graphs on the same screen with common controls without programming.

The system solves the tasks of strategic management, supporting the company's strategy, forming budgets based on the company's development plans (bottom-up) and performance targets (top-down), and also analyzes the company's performance indicators. To this end, the product provides consolidation, systematization, analytical processing of operational information and its visualization into sets of interactive analytical panels. Enrichment of data from third-party applications for model formation is supported. Financial models are used to reconcile proposed changes in the business.

You can use the version, scenario (what if...) and the actual analysis plan with a set of planning scenarios (optimistic-pessimistic). Integration with artificial intelligence systems is used to build predictive models.

According to the company, the powerful and fast calculation system of the TURBO X platform, on which the TURBO Budgeting solution is based, is built on a combination of a multi-dimensional OLAP in-memory cube and a relational base, contains ETL tools for embedding in the company's infrastructure.

Beyond the borders of dashboards

In the process of developing the functionality of business intelligence systems, the dashboard system - a key element of BI - remains as one of the proven analytics tools.

In our opinion, the main task of the BI system is to help managers make quick and high-quality decisions without the risks associated with errors due to incorrect data, "he says. And he adds: And in order to assess the quality of analytics, it is necessary to determine the metrics of the system. It can be a monetary benefit from decisions made, the number of days without emergencies and much more. If the created dashboards are not in demand by managers, then they are not useful.

Analysts Gartner in the study "10 main trends in data processing and analytics" note that over time, the amount of time that users of analytical systems spend in dashboards will decrease. The reason is that the tools of augmented analytics, natural language processing, streaming analytics and collaboration in the future for 3-5 years will be fully automated and can be configured to solve specific problems that arise in front of specialists. Thus, analytics that take into account the specific context of the application of analytical methods and tools, the decrease in the role of dashboards as predetermined toolbars, will decrease. In line with such a development of events, it is your hand to submit to the self-service analytics managed by the business user himself.

From dashboards to self-service analytics

The topic of self-service analytics, which is implemented at every workplace of each employee, is very popular today. But how to provide each employee not only successfully visualized reporting (dashboard), but also personalized information that turns dashboard analytics into a decision support system (SPS) by managers or direction managers?

It is important that the SPPR and BI are a single whole: from data to recommendations to action, ideally - to the formation of an order, - Artem Grishkovsky is sure.

In particular, the regional information and analytical systems of situation centers on the Triaflai platform are implemented on the principle of not just a dashboard with a picture, but as a system of recommendations based on data obtained from various sources, expert assessment and the knowledge base accumulated in the regions, - notes Artem Grishkovsky.

By 2023, Gartner predicts, a third of large companies and corporations will have analysts specializing in decision modeling tools on their staff.

"Civic" analytics

Citizen data science, a concept introduced by Gartner, defines a new direction that includes tools and methods that allow users who do not have professional skills in data science to solve complex problems of extracting analytical information from data. This "democratization" of analytics occurs as a result of the unification of two previously autonomous worlds: data and analytical processes.

Implementation of end-to-end workflows using augmented analytics tools blur the differences between these areas of work. As a result, Gartner predicted in 2020, over the next 3-5 years, not only professionals, but also the so-called "civilian developers," that is, employees of business divisions, will be able to work with analytical applications.

So, in March of this year, Forsyth released the FlyBI business analytics and data visualization product, implemented in the civil analytics paradigm. FlyBI is a tool for solving Data Discovery problems, focused on the independent work of users from various areas of business - self-service BI. It makes it possible to combine heterogeneous data sources: integrate with corporate information systems and work with heterogeneous data sources, including data stores, databases, individual files, and, moreover, with the ability to form a single data model. Moreover, it is possible to quickly connect to the necessary data sources and analyze them on the fly. And this, the company says, is the key difference between FlyBI and traditional BI systems, which provide a predetermined set of calculation methods and output reports without the ability to add new data sources or quickly create a report for a new task.

FlyBI includes a set of built-in and external visualizers and is capable of supporting up to 200 active connected users. It can be used as a separate independent solution, as well as in integration with the Forsyth platform . Analytical Platform. " In the future, the company intends to add means of collaborative work to the FlyBI product, which will turn it into a platform for joint research by a team of analysts.

Industry experts agree that a civil analyst is, first of all, a specialist analyst, because the requirements for the analytical competencies of employees are growing as the applied data processing tools become more complicated., Alexey Vyskrebentsev head of the center for expertise of the company's solutions, Forsythe"" explains that the analyst performs a number of intellectual tasks: from preparing and structuring data and formulating a mathematical problem to, in fact, conducting calculations. Professional analysts will not be needed only in the case, says the specialist, when the intellectual itself ON will be able to perform all these operations in automatic mode: having received raw data at the input, formulate analytical concepts, collect datacets for them, make calculations and draw conclusions. The most creative part of these works - improving the quality of data and its reliability, improving old models and creating new ones - will not be completely automated for a long time.

However, the developers of analytical solutions are already moving along this path.

Automation of Human Intellectual Skills

Today, at a practical level, we are talking about consolidating capabilities in a style: smart self-srvice analytics + BI + DSS. Theoretically, this triad fits well into the concept of hyperautomatization put forward by Gartner. It involves the formation of a "digital workforce" based on smart software robotization (RPA/IPA). How realistic is the idea that analytical technologies at the level achieved today can create professional intelligence, experience, skills of a digital employee?

Today it is difficult today to give a clear description of the image of a "digital assistant" that fully replaces employees and closes the need for automation not only routine, but also operations containing an intelligent component. This image will be formed through experiments and project experience, says Artem Grishkovsky, commercial director of Trusted Environment.

However, Triaflai - Business Intelligence Platform (BI) and Decision Support Platform (DSS) - shows trends in embedding tools that complement the classic portrait of BI products.

Artem Grishkovsky identifies four aspects of such development:

1. Extremely important is the set of technological means that will ensure confidence in the data, since without confidence in the data, decision-making by both man and "robot" is not possible, "says.
2. The solution is based on a model - a digital image of the business process. Therefore, we need tools that will allow you to flexibly, quickly configure models, make adjustments, create so-called "digital twins."
3. Knowledge bases are needed, that is, expert assessments, historical data that, in reference to multifactorial models, will allow you to adjust, train models and, already in robot mode, offer solution options.
4. In itself, the solution proposed by the system does not make sense if it is not embodied in the control impact - an order, which also requires the introduction of new tools.

According to Artem Grishkovsky, the Triaflai product is being built as a comprehensive platform for automating decision support, including these above tools. At the same time, the emphasis is currently on the automation of the work of analysts and experts. In the implementation projects that are implemented on the Triaflai platform, knowledge bases are formed, which in the near future will allow us to talk about the possibility of complete automation of decision-making, the commercial director of Trusted Environment is sure.

How the task of increasing confidence in data in large analytical systems is solved at the practical level can be judged by the example of VTB. The Bank has built work with credit risk factors (FCR) on the basis of machine learning, which made it possible to increase by several percentage points the Gini indicator, which assesses the accuracy of forecast models. At the time of the launch of the project in 2018, 22 credit risk factors for corporate clients were selected to calculate the scoring score.

Source: VTB blog on Habr.com, 2018

The Corporate Information Store (CIR) - the bank's primary data store - has become one of the data sources for the model. In fact, Data Lake = FIR + external data sources. Data from external sources reflect affiliation, B2B connections, etc.

The Rating Calculation System (WBS), one of the main databases used to assess the risks of corporate clients, contains business information about the ratings of enterprises, financial statements, etc. WBS data are supplemented with information from various files, including data for the current work of data scientists.

The bank says that the Hadoop cluster is easily scalable, which provides freedom to increase the amount of data consumed by models. Moreover, individual models can perform their calculations in parallel. But the most important thing is the Jini indicator. At the same time, analysts do not need to contact IT specialists with a request to write SQL queries to FIR in order to then process models at their workplaces. Analysts can make requests on their own, which means that the speed of processes is radically reduced.

Smart Human Analyst Assistant

Of course, artificial intelligence equal to human intelligence will not be for a very long time. Therefore, a "smart enterprise," as well as a "smart office," is the intelligence of real people, not software robots, "says Valery Andreev, Deputy General Director for Science and Development of IVK.

In particular, algorithms will not be able to replace a competent analyst who sees the whole picture of what is happening and notices the appearance of important anomalies, said Marina Romanova, head of the consulting department for analytical solutions of the ERP division of T1 Consulting.

Moreover, intelligent IT, providing a person with smart auxiliary tools, sets a new bar for the quality of the analyst's work, Artem Grishkovsky is sure: not just to answer the questions posed, referring to the available data, but to generate new non-trivial knowledge and proposals for their use for the benefit of the company. For example, today, they say in the company "VS Lab," with the help of a product of the OSINT class, you can, for example, to determine the technological potential of a particular new niche of the market, identify the key directions of technological development, understand how these directions are related to each other, what specific people and companies deal with these issues, and how they are related. Шаблон:Quote'In other words, AI can carry out a detailed business analysis of a technological niche that seems promising, but the final decision on the start of actions and their character remains with the person early, "says Alexander Khledenev.

One of the steps to automatically build new knowledge from existing data is to combine structured and unstructured data for collaborative analysis.

Indeed, the mechanisms working today for analyzing documents and texts in the natural language, including multimedia data sources: search for references, categorization and extraction of facts, semantic analysis to determine emotional coloration, assessment of interest, relationships, identification of patterns, etc. - require the obligatory participation of the person responsible for the formation of queries and meaningful formation of schemes of subject areas (data sources, criteria and features, mandatory entities, attributes and connections). Automation is possible in the way of using the formalism of ontologies to describe a subject area characterized by a certain logical structure.

Highlighting the structure of the subject area (ontology) is the first step in bringing unstructured data to a structured form, says Artem Grishkovsky in his article "Integrated processing of unstructured data (Open systems. DBMS).- Each individual subject area is only a subset of an unstructured data set, therefore, to maximize the possible data coverage and, as a result, more complete analysis, it is necessary to allocate the maximum possible number of different subject areas that will participate in the analysis.

The generated structures successfully participate in cross-analysis with structured data sources.

Source: Artem Grishkovsky. Integrated Unstructured Data Processing//Open Systems. DBMS, 2013, NO. 06

Within this approach, ontology is a logical structure that is associated with one or more physical structures data storage in a database. Ideally, an ontology is an abstract logical layer that separates a user analyst from storage structures. Thus, the analyst performs all work with data, the formation of queries and reports in terms of the ontology scheme, that is, the very subject area in which a person works.

Among Russian developers of analytical systems, a solid experience in creating products of this kind has been accumulated. For example, the OntosMiner linguistic processor, designed to build complex text monitoring and data analysis systems (developed by Avicomp Services, now part of the United Instrument Making Corporation. OntosMiner analyzes text using ontologies and dictionaries available for user editing, as well as special heuristics.

Ontological models are able to remove the ambiguity of words, which, in particular, helps to implement automatic translation of texts. For example, the developers of the product demonstrated its capabilities for recognizing words in languages ​ ​ that the system did not learn. For example, the editors of CNews told in 2015 how, after analyzing a large number of text documents in Russian about Chinese persons (in particular, texts about Song Qingling, the wife of the Chinese revolutionary Sun Yat-sen), the system begins to recognize the hieroglyphic style of their names. This is because in hieroglyphic names, the system finds the same semantic connections as in names written in Cyrillic or Latin.

The developers themselves call the ability to configure it by the customer one of the most important properties of their system: OntosMiner allows you to fine-tune the linguistic processor for specific goals of the company using machine learning technologies.

Semantic Archive Platform is an analytical tool developed by Analytical Business Solutions, which allows you to automate the entire technological chain of solving analytical and intelligence problems, from collecting the necessary information, its intellectual analysis to convenient reporting.

The platform makes it possible to analyze and use heterogeneous information with maximum completeness for timely adoption of optimal management and business decisions, the company says:

• conduct a comprehensive check of counterparties, partners, employees in different countries of the world, conduct competitive and business intelligence;
• search for any information about individuals and legal entities, form dossiers and assess the risks of doing business;
• conduct a deep analysis of the relations of the owners of companies, investigate the connections of large holdings and corporations, etc.;
• monitor and analyze the political, social and economic situation;
• investigate corruption, terrorist, criminal and financial schemes.

Source: Analytical Business Solutions

Among the technologies used: Text mining (extracting valuable information from a large number of unstructured data), OSINT (using all open sources of information: media, social media, online databases, forums, blogs, etc.), ontological model and the possibility of changing the ontological model of the database without involving programmers. Thus, companies can create their own knowledge bases in various areas of activity.

Another technological direction demonstrates steady growth in the field of analytics - graph solutions. Sergey Gromov talks about the growing level of maturity of Graph Processing solutions in business analytics.

For example, the Cronos-Inform group of companies, which develops application software for processing structured and full-text information, as well as creating automated information collection, processing and analysis systems based on it, uses the CronosPRO tool database as the core of the system, which uses a network data model to support the tasks of modern high-performance analytics.

According to the company, the CronosPRO system is distinguished by a combination of high speed with economical placement of data on disk: dynamic compression allows you to reduce the amount of data by 1.5 to 2 times.

Next Review Material "Artificial Intelligence Technologies" > >
> Home Review Page "Artificial Intelligence Technologies" > > >

Other materials of the review "Artificial Intelligence Technologies"

• Data processing in deep neural networks: achievements and calls of the current moment
• AI: from data to knowledge
• Processing of natural language documents and texts
• Smart Process Processing
• Smart Voice Services
• Intelligent video analytics
• New Challenges for Information Security Tasks
• Smart manufacturing
• Virtual Assistants
• Smart city
• Conclusion. Where does the world of practical AI implementations roll?

Other materials on the topic of AI

• The artificial intelligence market in Russia has reached a turning point. TAdviser 2020 Review
• Artificial intelligence (AI, Artificial intelligence, AI)
• Artificial Intelligence (Russian market)
• Artificial Intelligence (Global Market)
• Computer Vision: Technology, Market, Outlook
• Video Analytics Systems Video Analytics Systems and Projects Catalog
• National Strategy for the Development of Artificial Intelligence
• Machine Learning, Malicious Machine Learning, Data Labeling
• RPA - Robotic Process Automation
• Video analytics (machine vision)
• Machine intelligence
• Cognitive computing
• Data Science
• DataLake
• BigData
• Neuronets
• Chatbots
• Smart speakers Voice assistants
• Artificial intelligence in various fields: in banks, medicine, radiology, retail, military-industrial complex, production, education, Autopilot, transport, logistics, sports, media and literature, video (DeepFake, FakeApp), music
• Self-driving cars in the world
• Self-driving cars in Russia