MISIS and HSE: Method of processing video based on neural networks

The main articles are:

• Neural networks (neural networks)
• Video analytics (terms, applications, technologies)

2023: Introduction of Neural Network-Based Video Processing Method

Scientists at MISIS University, together with colleagues from the Higher School of Economics, have proposed a method for processing video based on neural networks, which will help to distinguish the main thing from videos and thus significantly save time. This is especially true for various areas where a rapid analysis of a large number of video materials is required, for example, in video surveillance systems, educational projects or sports events. MISIS announced this on November 28, 2023.

Every day consumption video content is growing rapidly. According to to data Cisco the Global Networking Trends Report, in 2022, video accounts for more than 80% of all Internet traffic. Therefore, researchers are actively developing tools to automate the search for the main information one among the abundance of video content.

Using video generalization or summarization, you can compress the original content, while maintaining its basic essence. There are two main approaches: creating a static sequence of key personnel and forming a short video, where important points are arranged in chronological order. The essence of the method presented by researchers at NITU MISIS and the Higher School of Economics is based on the fact that key points are selected from the entire video, and its total duration decreases. This allows you to save the main content and at the same time make the video more compact.

The developed model includes a multi-layered multilevel attention module, similar to a transformer, which allows simultaneous processing of input elements and prevents slowdown caused by recurrent neural networks used in previous approaches. A feature of the model is the use of a positional encoder, which takes into account time information and increases the quality of generalization. This technology was tested on two reference data sets and showed high efficiency, - said study co-author Ilya Makarov, director of the artificial intelligence center NUST MISIS, head of the AI in Industry group of the AIRI Institute.

The results of the study confirmed that this video generalization model gives not only competitive results, but also surpasses the selected analogues. This opens up opportunities for the use of video content and makes it more accessible to a wide audience.

The study was carried out as part of the strategic project of the University of MISIS "Digital Business" under the program of the Ministry of Education and Science of Russia "Priority 2030."