Innopolis University has developed an algorithm for linguistic analysis of documents for the Ministry of Internal Affairs of the Russian Federation

2020: Development of an algorithm for linguistic analysis of documents

Innopolis University on November 30, 2020 announced the development of a solution for the Department of Information Technology, Communications and Protection of the Ministry of Internal Affairs of the Russian Federation.

The university developed an algorithm that conducts linguistic analysis of documents, after which it converts the first-person narration into a third-person text: for example, from the combination "I saw that Ivanov approached me" in "He saw that Ivanov approached him." The Innopolis University team trained the neural network (BERT architecture) on the dataset of news reports with a volume of 12 GB, it marks the belonging of the talker to the desired subject, determines the shape of the word and morphological categories.

The results of the neural network were used to write a Python algorithm based on heuristics and rules of the Russian language. The solution takes into account the specifics of departmental texts, and is also able to process artistic texts. The algorithm processes pronouns, verbs, prepositions, quotes, direct speech, determines the affiliation of pronouns to names and identifies heroes, dates, amounts of money, locations.

Employees of a Russian IT university suggested introducing an algorithm into a comprehensive service with a web interface, where the user can insert text, download text files of different formats, audio files for recognizing speech and images with text. A plugin has also been developed for LibreOffice with the selection of modified parts of text.

The developed solution was tested by experts from the Ministry of Internal Affairs of the Russian Federation. The algorithm showed excellent results using their examples. In 48 hours, we have developed a cross-platform autonomous product that is ready for implementation in the department and is able to spare employees of internal affairs bodies from routine tasks, "said the team leader, an employee of the Center for Artificial Intelligence of Innopolis University Semyon Kiselev.