MTUSI: Deviant Behavior Detection Algorithm

Main article: Video analytics (terms, applications, technologies)

2023: Creating an Algorithm

MTUSI scientists have created an algorithm for detecting deviant behavior based on real-time video with support for several cameras and several people, based on the assessment of a person's pose and the open source form of the OpenPifPaf algorithm. The university announced this on June 6, 2023.

The problem of ensuring the safety of residents of megacities is becoming especially urgent today. Often dangerous situations arise due to the deviant behavior of people: fight, attack, being in the wrong place. Modern cities are equipped with video surveillance systems: they are used to control urban life, or they can be directed to detecting potentially dangerous situations in real time.

Due to the huge number of videos, the task of detecting dangerous situations requires the use of modern intelligent technologies that allow automatic analysis. The primary task is to teach the program to determine the position of a person's body in images and videos, - said the dean of the Faculty of Information Technologies of MTUSI, Ph.D. Mikhail Gorodnichev.

There are several libraries that allow you to evaluate a person's pose from a video image. After studying various options, scientists decided to use OpenPifPaf to work on a project to create an algorithm.

{{quote 'Data is the basis of any system built using machine learning. The main criteria for their quality are breakdown accuracy and diversity. The most popular data sets for position recognition are MPII and COCO, they differ from each other in the markup and number of images. When implementing the algorithm for highlighting deviant behavior, the COCO set was used, - explained Ksenia Polyantseva, senior lecturer at the Department of Mathematical Cybernetics and Information Technologies at MTUSI. }}

In addition to the OpenPifPaf library, the project used Python auxiliary libraries: torch, argparse, math, OpenCV, matplotlib, PIL. For the practical application of the algorithm, an application was created that recognizes a person's fall from video.

The main information is displayed above the video image: the number of frames per second, the total number of personnel processed, the predicted state of the person, which can be either "Normal," "Fall Warning" or "Fall." In the event of a human recognition error (including if there are no people in the frame), the state is displayed as "No."

In addition, a web application was created that responds to deviant behavior, captures it, writes the event to the database and displays a notification. Each line contains information about the sign of deviant behavior (at this stage only the fall of a person), the date and time of fixation, the camera number.

Employees of the Department of Mathematical Cybernetics and Information Technologies of MTUSI Marina Moseva and Artem Pavlikov emphasized that the proposed algorithm works with fairly low hardware requirements, and the program does not require a GPU. However, the algorithm has a tendency to false positives due to unbalanced training data, so further training of the system is needed for best results.