Nvidia NeMo Framework

The main articles are:

2023: Discovery of a vulnerability that could lead to security breaches and disclosure of confidential information

Researchers have discovered a potential software vulnerability artificial intelligence AI () Nvidia that could lead to a security breach and disclosure of confidential. information This became known on June 12, 2023.

Nvidia has developed the "NeMo Framework," a system designed to work with large language models that are the basis for AI products such as chatbots. This system is used in business to combine the company's own data with language models in order to provide answers to questions, thus replacing the work of support representatives or providing simple health advice.

However, researchers at Robust Intelligence, a company headquartered in San Francisco, found that security restrictions set for the safe use of AI could be easily circumvented. They figured this out after spending several hours analyzing data using Nvidia's system.

In one of the experiments, the researchers were able to force the Nvidia system to replace the letter "I" with "J," which led to the disclosure of personal identifiable information from the database. The researchers found that they could bypass security restrictions in other ways, such as making the model deviate from what it is not supposed to do. Echoing Nvidia's example of a narrow discussion of the employment report, they were able to translate the model into topics such as the health of a Hollywood movie star and the Franco-Prussian War - despite bans designed to prevent artificial intelligence from going beyond specific topics.

The ease with which researchers have circumvented security measures underscores the challenges artificial intelligence companies face in trying to commercialize one of the most promising technologies that have emerged in Silicon Valley in recent years.

Researchers advised their customers to avoid using Nvidia's product. Nvidia said it had fixed one of the main causes of the problem,^[1] analysts said^[2].

Notes