Retrainable AutoML Framework

2024: System Implementation

On June 25, 2024, Alfa-Bank announced the creation and implementation of the first fully automatic retraining system in Russia for artificial intelligence models, the Retrainable AutoML Framework. This innovative development is designed to solve the problem of degradation of the quality of AI models in the process of their use, the press service of the credit institution says.

The Retrainable AutoML Framework is a comprehensive solution that includes a continuous monitoring system for models running in industrial systems and a feedback mechanism. The system automatically monitors the performance of models by comparing their forecasts with actual results.

𝓲

Unsplash

A feature of the Retrainable AutoML Framework is an end-to-end automation process - from feature extraction to seamless replacement of an outdated model with a new one in an existing business system. At the same time, the participation of a person in decision-making is minimized.

An important advantage of the system was the ability to retrain the model directly in the industrial environment without the need to return to the development environment. This was made possible by creating a universal model image, separate from the model itself and an archive with an environment.

The introduction of the Retrainable AutoML Framework allowed to increase the average annual quality of models by 5-7%. The system is able to timely detect a decrease in the accuracy of forecasts and initiate a retraining process, which allows you to maintain the effectiveness of models at the initial level. Konstantin Chetin, Head of the Development of Models of the Client Base of MMB in Alfa-Bank, at the end of June 2024 noted:

The system has already proven itself perfectly. Therefore, now we are scaling its application in the bank. The main goal is to transfer up to 70% of all working machine learning models in the bank to automatic retraining by the first quarter of 2025.

The projected financial effect of the implementation of the Retrainable AutoML Framework for 2024 is a savings of 200 million rubles. This is achieved by reducing the time spent by highly qualified specialists on model maintenance and improving the accuracy of forecasts.^[1]

Notes