MISIS, MIPT and T-Bank AI Research: Ensemble Neural Network (SDDE, Salience Diversified Deep Ensembles)

Main article: Neural networks (neural networks)

2024: SDDE Neural Network Presentation

Students of MISIS University and MIPT, together with scientists from the non-profit artificial intelligence research laboratory T-Bank AI Research, proposed the SDDE ensemble neural network (Salience Diversified Deep Ensembles), which more accurately identifies objects in images not uploaded to databases. MISIS announced this on October 31, 2024. In the future, this algorithm will help develop the field of unmanned vehicles and medical diagnostics, where it is important to distinguish between unidentified elements and graphic artifacts.

𝓲

MISIS

With the increase in the amount of data, there is a need for more reliable neural networks that can not only classify new objects, but also recognize the technical interference that inevitably occurs when obtaining an image. The aggregate of all unknown information is referred to as out-of-distribution (TDD) data. Human factors can lead to undesirable consequences if DVR is detected. The creators of the algorithm solved this problem with the help of an ensemble model variety, which reduced the correlation between occurrences and increased the overall accuracy of the system.

The SDDE ensemble neural network consists of several models that are trained on subsets of individual databases, which allows each of them to focus on the unique characteristics of images. This is achieved by diversifying the attention maps of each model - a concept that allows you to understand where the neural network is looking. As a result, the diversity of the ensemble increases and the neural network determines objects in images with a minimum error. To assess the effectiveness of the neural network, the researchers conducted tests on several databases: CIFAR10, CIFAR100 and ImageNet-1K. The SDDE ensemble neural network performed best compared to similar algorithms such as Negative Correlation Learning and Adaptive Diversity Promoting.

One of the most important tasks in the development of machine learning models is to correspond to the real probability of the one given by the neural network. That is, the neural network is as confident as it is easy for it to predict the target for this sample. Usually networks do not doubt their predictions at all. In the framework of this study, we proposed a method for diversifying ensembles based on logits - that is, the values ​ ​ that the neural network issues before turning them into probabilities. This change made it possible to increase the accuracy of the "opinion" of the neural network when detecting data outside the distribution, which is critical for the use of models in real conditions. For example, in autonomous driving mode, it is necessary to accurately identify objects on the road in order to prevent accidents. In medical diagnostics, however, an extensive database is required for the correct diagnosis. Uncalibrated models may be overly confident in their incorrect assumptions. Our neural network lacks excessive confidence, which allows it to more adequately evaluate its calculations, "said Maxim Zhdanov, a 3rd year student at the Institute of Computer Science, NUST MISIS.

To better detect interference artifacts, the researchers used the Outlier Exposure approach, which is to train the model on special datasets containing examples of TDD.

Earlier, scientists from the University of MISIS and the Higher School of Economics have already presented a new neural network LAPUSKA, which copes with an improvement in image quality 2 times faster than similar products.