Evraz and GlowByte have developed a federated real-time monitoring system

2023: Kolmogorov Predict Testing

EVRAZ, supported by the Advanced Analytics GlowByte practice, has piloted the Kolmogorov Predict solution for distributed real-time monitoring of the quality of ML models into its ModelOps infrastructure. This was announced on January 30, 2024 by representatives of GlowByte.

Kolmogorov AI software was developed by the Russian vendor Data Sapience. Kolmogorov Predict extends the capabilities of EVRAZ's ModelOps enterprise platform, allowing real-time monitoring and analysis of machine learning models even if they are used in a number of mathematical optimization algorithms, when the number of predictions per second can exceed tens of thousands.

Monitoring metrics are calculated asynchronously with model execution services and do not affect their operation in any way. This approach allows you to maintain the efficiency of monitoring and at the same time not overload model execution services.

The solution is easily integrated into existing Python model learning pipelines. Thanks to this, the system allows you to quickly connect monitoring not only for new models, but also for models already operating within the information systems, inside the OpenShift or Kubernetes platforms.

The system provides flexible options for configuring and monitoring metrics using an intuitive user interface.

Due to the active adaptation of data analysis and artificial intelligence technologies, the ability to promptly monitor and manage machine learning models and advanced analytics is becoming key to the successful conduct of any business. Thanks to cooperation with EVRAZ colleagues, we were able to add important distributed and federated computing functionality to the core of our Kolmogorov platform and, in particular, to the Predict model monitoring module, "said Mikhail Zaytsev, owner of Kolmogorov, Data Sapience.

We used to use standard tools to monitor models. Since they consider each prediction independently, the requirements for the resources used grow directly in proportion to the number of predictions. As part of the Lean approach, the widespread use of such tools is unacceptable, and the lack of monitoring for optimization creates the need to involve a developer to analyze any ambiguous moments during operation. The new system is able to consider thousands and millions of predictions performed within the framework of optimization algorithms as a single entity. This system was helped by colleagues from GlowByte to develop and pilot. Thanks to it, we will be able to implement monitoring without a multiple increase in resource requirements, including on projects using optimization methods, "said Andrei Zubkov, leader of the AI department, head of the EVRAZ information systems development department.