Raft Digital Solutions Raft Digital Solutions

2024: Creation of LLM Security Lab with ITMO and Napoleon IT

Raft, together with the AI Talent Hub of ITMO University and Napoleon IT, created the LLM Security Lab. This laboratory will become a key center of research in the field of security and ethics of generative AI, as well as the application of large language models (LLM) in business. Napoleon IT announced this on August 21, 2024.

The purpose of the laboratory is to support scientific and applied research of students, training specialists in the field of working with LLM models and AI solutions, as well as the development of new technologies and approaches in this rapidly developing field.

Analysts predict that Gartner 80% of all product development processes will use generative artificial intelligence by 2025. In this regard, securing LLM applications becomes a critical task. Half of the companies that have already implemented generative AI face serious security problems, which confirms the relevance of the creation of the LLM Security Lab.

LLM Security Lab is aimed at developing expertise in the security of LLM applications. The laboratory was created on the basis of AI Talent Hub, a master's degree in machine learning developed by ITMO University and Napoleon IT. Raft is investing in this project to explore and address AI security issues in depth.

Key areas of research in the laboratory include:

• Monitoring and Analysis: content Developing tools for tracking attacks and analyzing message toxicity that allows companies to protect their reputation and improve user interaction.
• Simulating cyber attacks and: personal data protection exploring new types of attacks, developing solutions to prevent leaks confidential information and secure AI products.

In addition, within the framework of the laboratory, educational programs will be developed for the LLM direction in the AI Talent Hub. They are aimed at training specialists in various aspects of working with large language models. In particular, the training will include:

• Training of researchers who will develop and implement open-source solutions that contribute to progress in the field of AI.
• Training ML engineers who can integrate new technologies into business processes, including bringing LLM into production and optimizing them.
• Training of AI product developers who can create and implement innovative solutions based on LLM and generative AI in various industries.

One of the key courses in the LLM direction will be "LLM Security." His program covers current security and ethics issues in the field of AI, including model vulnerabilities and methods of protecting them. The course combines theoretical training and practical exercises, allowing students to prepare for real business challenges in AI security.

Students of the course have already tested the alignment of open and private models, and also conducted successful tests of the LLM chatbot for a large beauty retail company.

Over the past year, we have seen massive adaptation and increased interest in generative AI technologies from businesses. An important aspect of this adaptation is the protection of company information and trade secrets. We see how methods of attacking large language models and complex IT systems are actively developing, which integrate them into business processes. In our laboratory, students will work on application tasks, audit LLM applications and develop specialized solutions aimed at improving security, "commented Evgeny Kokuikin, head of Raft AI products.

The AI Talent Hub platform was conceived as a meeting place for technology companies, a university and talents, within which new knowledge, new technologies, innovations are born. We welcome a new partner to work on the development of the use of generative networks in corporate business, "commented Konstantin Price, Commercial Director of Napoleon IT.