Translated by

Polymatica Analytical platform

Product
Developers: Polymatica
Last Release Date: 2019/08/12
Technology: BI,  Data Mining

Content

The analytical Polymatica platform (before Polymatica) is intended for the multidimensional analysis of indicators and metrics without detailing loss. A system integrates functionality of Data Science, Data Mining and BI. In a basis of the platform the high processing speed of large volumes of diverse data and automation of mathematical algorithms and statistical models are put.

The platform is developed by the Russian company "Polymatica" from scratch and without use of modules of third-party producers. Are its cornerstone Multisfera — own technology of multidimensional data processing in real time.

2019

Functionality Scenarios for automation of work with data

On August 12, 2019 the Polymatica company announced updating of the analytical Polymatica platform within which the user can get acquainted with functionality Scenario allowing to save results of analytics and to exchange them with colleagues.

The functionality Scenarios allows to automate routine operations on work with data. Works as record of actions of the user with a possibility of the subsequent reproduction on the updated data array. This functionality will allow to save time of analysts and business users by preparation of identical analytical cuts – they will be able to receive them on clicking of one button and to exchange with other users of Polymatica, the developer claims.

The Polymatica company noted also other updates of the platform:

  • Diagrams for a clustering

In the analytical Polymatica platform by one button the intellectual algorithm of a clustering allowing to collect in groups objects with similar characteristics is available. A system itself will offer optimal quantity of clusters, and will show the most typical representatives.

Diagrams for a clustering in Polymatica

Pools – the evident image of clusters in two measurements where the size of a circle can be used as the additional parameter showing proximity to the center of a cluster.

Diagrams for a clustering in Polymatica

3D - pools – the evident image of clusters in three dimensions where the sphere size can be also used for identification of provision of an object in a cluster.

Diagrams for a clustering in Polymatica

Corridors – the image of clusters with display of limits of values of the elements which got to a cluster.

  • Polymatica as source of data on ODBC

The analytical Polymatica platform uses ODBC both for data loading, and for transfer of the received results in the third-party systems, for example, in forms of the regular reporting or the customized dashboarda. Such approach simplifies solution integration in multicomponent systems where Polymatica carries out tasks in operational multidimensional analytics of data bulks, the solution designer emphasized.

  • Interval updating

For system administrators the feature for interval updating of the multisphere is added. In the interface it is possible to configure removal and rewriting of data a certain time frame. It allows to obtain correct data in the automatic mode, without rewriting all array.

  • Use of selection when calculating in horizontal direction

The Polymatica company noted that traditionally work with data is carried out pokolonochno, for example, distribution of a share of clients in the annual profit of the company: at the left there are clients, opposite to everyone their shares which as a result make unit or 100%. In the provided version if desired to compare monthly, instead of profit in a year 12 months – 12 columns appear. Calculation in horizontal direction allows to look at distribution of shares by each client monthly. For convenience of users in the analytical platform an opportunity is supported, without rebuilding the table, to execute selection and grouping of clients if it is necessary to consider any additional parameters, for example, the range of the average amount of the check or time of purchase.

Diagrams in Polymatica
Diagrams in Polymatica
  • Diagrams

Diagrams Surface and Spheres are added, in diagrams Fine-molded and Circles appeared trend lines, hints are added to diagrams Radar, Chordate and Dendrogramm and settings of change of flowers are improved.

Diagrams in Polymatica
Diagrams in Polymatica

Appointment. Architecture. Technologies

According to information for January, 2019, first of all, Polymatica is focused on independent work of users (analytical self-service). The platform provides a set of visual analytical tools and simple methods of advanced information analysis in use, at the same time the user of a system always works with data without aggregation. Due to use of hybrid architecture of the platform and computing resources of graphic processors response time of a system remains at the level of several seconds even at simultaneous processing of millions of objects and billions of records.

According to results of the research "IBS Examination", the solution allows to perform analytical transactions with "live" data (2 billion lines) in 3 seconds on all array, the transaction Data mining (a clustering of objects, 2 billion lines) approximately in 17 seconds on all array.

Appointment

Possibilities of Polymatica cover a broad spectrum of tasks, including data validation on consistency, data processing and their statistical analysis, visualization of the received results on interactive charts, charts and web cards and also carrying out deep researches of multidimensional data using methods of advanced analytics.

Clients of Polymatica company use the analytical platform for efficiency evaluation of the companies/organizations and accomplishment of social and economic programs, trackings of the statuses of project implementation and approval of agreements, optimization of purchases and stock balance, consumer segmentations for providing targeted offers.

The platform proved for the projects implemented within accomplishment of programs of transition to digital economy.

Architecture

The analytical Polymatica platform represents the client-server solution which can be unrolled in the closed circuit or in a cloud. The Polymatica server works running the operating system based on Linux (Ubuntu and dr), including the certified versions for the state agencies and the organizations working with the classified information, for example, of AstraLinux, Smolensk. Access to functionality is provided via the browser on http or https.

Flexible configuration of access rights of users to the data which are contained in multispheres up to restrictions for separate dimensions is provided. For exchange of analytical cuts the built-in system of profiles which allows to appoint roles to users according to the regulations existing in the organization is used. For accomplishment of routine transactions scenarios which allow to write the sequence of actions for receiving a necessary analytical cut are used and after data-refresh to create it on clicking of one button.

Data-refresh can be performed by two methods: complete rewriting and incremental updating — adding only of new lines. The system administrator can configure the frequency and time of updates.

Polymatica allows to load data into a system from different sources. Import from relational databases MySQL MS SQL Oracle PostgreSQL, the distributed storages Teradata and solutions based on Hadoop, business applications, CRM- systems, 1C and other information systems and also flat tabular files is supported. Data loading in a system and creation of multispheres is performed by writing of the standard SQL query to to the database, the possibility of automatic update data in the multisphere according to the schedule is provided. During the work with tabular files direct import of data is used, and for "non-standard" sources the company provides tools for low-level data loading.

Multispheres are responsible for data processing in Polymatica. Among advantages of multispheres developers of the platform noted the following:

  • Work with dimensions and facts as simple pulling.
  • Creation of hierarchies during operation.
  • An opportunity to create compound dimension.
  • Use of GPU for acceleration.
  • Data compression up to 12 times.
  • Data loading up to 500,000 records per second.

Modules of the Polymatica platform allow to expedite intelligent data analysis of the multisphere — multidimensional calculations on millions of records take no more than a minute, at the same time 100% reliability and recurrence of the received results are guaranteed. Bidirectional coupling of modules with data of the multisphere provides users of a system with a powerful mathematical apparatus and visualization. Use of modules does not demand from the user of profound mathematical knowledge, long processes of search of the necessary model and setup of parameters — start of the module and calculations are executed after clicking one button.

Visualization plays a large role at data analysis — on the diagram easy to see peaks and deviations, to detect patterns. The analytical Polymatica platform supports different schedule types which work by the principle of interactive interaction with the selected dimensions, displaying changes in real time.

The data containing the addresses or coordinates can be localized on the geographical web card.

The platform includes the following modules Data mining:

  • Clustering — distribution of objects by groups on the basis of one or several parameters, the choice of optimal number of groups and calculation of clusters. The irreplaceable tool for segmentation of buyers and other objects of a research for creation of a portrait or targeted influence.

Application of the Clustering DataMining-method in Polymatica
  • Associative rules — calculation and identification of patterns of behavior of objects and determination of popularity and reliability of joint emergence of events. Including it is used for studying of joint purchases in retail.

  • Forecasting — determination of parameters of objects in the future on the basis of available historical data.

The received analytical cuts can be transformed to reports, using different methods.

The web interface interacts with the server by means of sending API commands for execution of operations and obtaining calculation results. At the same time all loading on accomplishment of calculations lies on the server, the client software program only displays data retrieveds and does not carry out any resource-intensive tasks.

Existence of API commands of the server allows to use possibilities of Polymatica for the solution of highly specialized tasks, to build in analytical modules of the platform corporate portals, business applications and internal information systems. The platform supports a possibility of development of add-on modules for the solution of industry tasks.

Integration into the platform of the Python console allows to expand possibilities of the solution for performing more difficult calculations and a covering of broader spectrum of tasks.

Technologies of the Polymatica platform. GPU Acceleration

The technology of acceleration of calculations on the graphic processors Nvidia is used in Polymatica for providing comfortable response time of a system during the work with multispheres of the big sizes, from 100 million records. When receiving a request for accomplishment of analytical transaction a system automatically determines the most effective option of carrying out calculations (the processor, the video card or the hybrid mode) and distributes calculations between available resources.

The platform at the hardware level supports work on a server cluster, integrated on InfiniBand technology. In this case a system automatically creates a pool of available physical resources and executes task allocation between them.

2018: Polymatica 5.4

In December, 2018 the Polymatica Rus company released update for the flagship product — the analytical Polymatica platform. Version 5.4 release the company completed transition to own technology of Multisfera (instead of an OLAP cube) allowing to provide the high processing speed of Big Data and also presented the updated color and functional scheme for visualization.

In version 5.4 the appearance of the interface according to the changed corporate style and approach to a usability is updated. This updating starts a series of the global changes directed to improvement of the user experience for more effective work with the analytical platform.

Main changes in version 5.4:

  • For formation of diagrams the color palette developed by the European designers for creation of bright and stylish visual representations of data is used.
  • Diagrams are added: the chart with areas, the stream chart, point, the radar, the chordate chart.
  • Such functions as switching and rotation of diagrams, object management of the diagram through a legend, drill down are available to users.
  • Incremental updating: during creation and change of the multisphere it is possible to set parameters of the incremental updating allowing to add only new lines of data now.
  • Expansion of formats of export of a settlement zone: the feature to execute export in the CSV format is added.
  • Forecasting for several steps: now the user can select necessary quantity of forecast elements.

From version 5.4 it is recommended to use Google Chrome.

2017

Polymatica 5.1.8

Technologies of neural networks become more and more popular, even more often apply them to business solving of tasks. In high gear also the market of Big Data grows. To technologies which allow to analyze quickly and qualitatively Big Data interest in the market sharply increased in 2017. According to a research of the analytical agency Research and Markets of investment into the world market of Big Data in 2017 made 57 billion dollars. Technologies and products for processing of Big Data remain demanded in the different industries worldwide.

The Polymatica company announced in December, 2017 updating of the analytical platform – Polymatica 5.1.8. The platform uses Data mining technology and neural networks thanks to which became possible analytics in real time and processing of all available amount of data in minutes.

The updated Polymatica allows to solve such problems as: detailed justification for acceptance of management decision in the conditions of "compressed" terms, decrease in loading and cost reduction on IT department, process optimization of analytics, creation of the convenient detailed reporting and others.

Analytical platform. Description

According to information for March, 2017 Polymatica is the industrial BI solution for analytical work with big arrays of diverse data in real time. Possibilities of the platform cover a broad spectrum of tasks, including data processing and their statistical analysis, visualization of the received results on interactive charts, charts and web cards, preparation of report forms using tabular representations and graphic components and also carrying out deep researches of multidimensional data using methods of advanced analytics[1].

The platform is completely created based on own developments of Polymatica Rus. From scratch and without use of modules of third-party producers technologies of analytical data processing in real time (OLAP-in-memory) and acceleration of calculations using graphic processors (GPU-acceleration) were implemented and also advanced methods of intellectual information analysis (Data mining) are developed.

Independent approach to creation of a product allowed to avoid critical shortcomings of the existing BI solutions, such as technological limits on volumes of the processed data, limitation of functions in work with data, complexity of use of separate tools and, as a result, a small number of end users of such systems.

Polymatica is focused first of all on independent work of users (analytical self-service). The platform provides a set of visual analytical tools and simple methods of advanced information analysis in use, at the same time the user of a system always works with data in initial detailing. Due to use of OLAP technology and computing resources of graphic processors response time of a system remains at the level of several seconds even at simultaneous processing of millions of objects and billions of records.

At the same time the platform supports processes of the regulated data processing. Setup of access rights of users to initial data of an OLAP cubes at the different levels is provided, for exchange of report forms the built-in system of profiles which allows to appoint roles to users according to the regulations existing in the organization is used.

Architecture of the analytical platform Polymatica

Architecture of the analytical platform Polymatica

As of March, 2017 Polymatica is constructed based on a multi-tier architecture which each component, in turn, includes several modules. The modularity of architecture provides convenient scalability of the platform in process of growth of requirements to its opportunities: the number of users, volume of the processed records or the number of connections to data sources and also allows to use only necessary components of the platform in work), significantly increasing efficiency of use of the solution, both from technical, and from the economic point of view. Deployment and full-function system operation in the closed circuit (without Internet access), start of separate modules on different physical devices is supported.

Principal component of the platform is the analytical OLAP server integrated with the modules Data mining. The server of Polymatica works running the operating system based on Linux (Ubuntu, Red Hat, CentOS and others), including the certified versions for the state agencies and the organizations working with the classified information, for example, of AstraLinux, Smolensk.

Polymatica imposes available technical requirements to the hardware. So, for example, the server with two processors, 256 Gbytes of RAM and the video card Nvidia from 16 Gbytes of video memory is necessary for work with an OLAP cube up to 2 billion records in size, and during the work with a "small" OLAP cubes, to 100 million records, rather dual-processor server with RAM 32 Gbytes, existence of the video card at the same time is not required.

In the platform effective use of resources of RAM and the hard drive is implemented: during the work of the user in a system only data of an OLAP cube "active" at the moment are loaded into RAM of the server, and when importing from a source data compression more than by 12 times is provided.

Polymatica allows to load data into a system from different sources. Import from relational databases of MySQL, MS SQL, Oracle, PostgreSQL, the distributed Teradata storages and solutions based on Hadoop, business applications, CRM systems and other information systems and also flat tabular files is supported. Data loading in a system and creation of an OLAP cubes is performed by writing of the standard SQL query to the database, the possibility of automatic update of data cubed on the schedule is provided. During the work with tabular files direct import of data is used, and for "non-standard" sources the company provides tools for low-level data loading.

On client side the user performs all operations in the web browser standard manipulations with a mouse (drag and drop, selection, a call of a context menu, the left/right click). The adaptive system interface allows to work comfortably in a system on any device: computer, notebook, tablet PC or phone. Use of all popular web browsers with default settings is supported, additional requirements to installation Active-X, Flash or other components are absent.

The client software program (the browser or the application) interacts with the analytical server by means of sending API commands for execution of operations and obtaining calculation results. At the same time all loading on accomplishment of calculations lies on the server, the client software program only displays data retrieveds and does not carry out any resource-intensive tasks.

Existence of API commands of the server allows to use possibilities of Polymatica for the solution of highly specialized tasks, to build in analytical modules of the platform corporate portals, business applications and internal information systems. The platform supports a possibility of development of add-on modules for the solution of industry tasks.

Technologies of the platform Polymatica. Data Mining

Data Mining-modules of the platform Polymatica allow to expedite intelligent data analysis of an OLAP cube - multidimensional calculations on millions of records take no more than a minute, at the same time 100% reliability and recurrence of the received results are guaranteed. Bidirectional coupling of modules with data of an OLAP cube provides users of a system with a powerful mathematical apparatus and effective visualization. Use of modules does not demand from the user of profound mathematical knowledge, long processes of search of the necessary model and setup of parameters - start of the module and calculations are executed after clicking one button.

Application of the Clustering DataMining-method in Polymatica
Application of the Clustering DataMining-method in Polymatica

For March, 2017 the platform includes Data-Mining-modules: "clustering" – distribution of objects by groups on the basis of one or several parameters, the choice of optimal number of groups and calculation of clusters; "associative rules" – calculation and identification of the main patterns of behavior of objects and determination of popularity and reliability of joint emergence of events; "forecasting" – determination of the most reliable values of keyword parameters of objects in the future on the basis of available historical data.

Technologies of the platform Polymatica. GPU Acceleration

The technology of acceleration of calculations on the graphic processors Nvidia is used in Polymatica for providing comfortable response time of a system during the work with an OLAP cubes of the big sizes, from 100 million records. When receiving a request for accomplishment of analytical transaction a system automatically determines the most effective option of carrying out calculations (the processor, the video card or the hybrid mode) and distributes calculations between available resources.

The platform at the hardware level supports work on a server cluster, integrated on InfiniBand technology. In this case a system automatically creates a pool of available physical resources and executes task allocation between them.

Possibilities of the platform

(data are relevant for January, 2017)

  • accomplishment of rules E. Kodda for OLAP data processing
  • analytical self-service of the user
  • there are no restrictions for amounts of data with preserving of detailing

  • combination of functions of visualizers and analytical products
  • repeated acceleration of the solution of applied tasks
  • partner modules
  • OEM is the server

  • OLAP transaction with "live" data (2 billion lines) – 3 seconds on all array
  • the transactions Data mining (a clustering of objects, 2 billion lines) – of 17 seconds on all array
  • compression of data in 12+ of times without loss of initial detailing

Examples of the analysis of complete amount of data

  • state structures (3+, 23+ one billion records)
  • banks from TOP-10 (3+ one billion records)
  • telecom (2+ one billion records)

2016

In the register of the Russian programs

The Polymatica Rus company acquired the official right to providing the software to the state organizations. The Ministry of Telecom and Mass Communications of the Russian Federation included the reporting and analytical platform Polymatica in the Unified register of the Russian programs for electronic computers and databases.

According to the order of department, Polymatica was included into the list of the Russian programs of the class "Systems of Collecting, Storage, Processing, Analysis, Modelling and Visualization of Data Arrays". Inclusion of Polymatica in the official register meets the new requirements about transition of the state organizations of the domestic software. According to the order of the Government of the Russian Federation, since January 1, 2016 all Russian state agencies and state institutions are obliged to buy software from the companies which are included into the register of domestic software.

Polymatica 5.0

Polymatica 5.0 will leave at the end of June, 2016 and will include a number of developments of the company as for the Russian, and world market of the intellectual analysis Big Data and systems business analysts. Feature of the reporting and analytical platform "Polymatica 5.0" is removal of technological and functional limits on processes of the analysis and information processing. Using this version of Polymatica users will be able to process at the same time all array of the available data, receiving complete freedom in work with information. Thanks to technologies of online data processing in RAM (OLAP in-memory), accelerations of calculations using graphic processors (GPU-acceleration) and to the web interface response time of a system remains at the level of several seconds even during the work with billions of records.

Multilayer interface

  • instead of tabs – layers
  • conveniently for mobile devices
  • it is more than information at a single glance

Among other distinctive features of the platform there are a multilayer interface and support of user data. In "Polymatica 5.0" the possibility of simultaneous operation with several interactive layers for completeness of information analysis and also an option of input of additional dimensions and the facts which do not contain in initial data source will be implemented. Users of a system will have an opportunity "on the fly" to create data structures and to carry out difficult calculations to several clicks at the different levels of information processing.

  • advanced core
  • new format of an OLAP cubes
  • levels of calculation of the facts
  • support of user data
  • multilayer interface

Support of user data

  • assortment planning (fashion-retail)
  • forecasting and modeling
  • budgeting

Polymatica. Description

As of June, 2016 the reporting and analytical platform Polymatica represents the system of a class Business Intelligence, systems Data Mining, OLAP-servers and interactive visualizers. In a basis of the platform the processing speed of large volumes of diverse data and usability of mathematical algorithms and statistical models are put. The OLAP server has no restrictions on depth and detailing of the analysis of initial data. According to results IBS of the research "Examination", the solution allows to perform OLAP transaction with "live" data (2 billion lines) in 3 seconds on all array, transaction Data Mining (a clustering of objects, 2 billion lines) approximately in 17 seconds on all array and provides a compression of data in 12+ of times without loss of initial detailing.

Notes