OpenAI Classifier

2023: System Release

On January 31, 2023, OpenAI, the developer of the ChatGPT chat bot based on artificial intelligence, released a specialized classifier (OpenAI Classifier) capable of distinguishing human-written text from material created by a neural network.

The tool is a language model, the training of which used pairs of texts - one of them was written by a person, and the second was generated by an AI system. Such pairs of materials had the same theme, but were taken from various sources. A confidence threshold is set for the classifier to minimize the number of false positives.

ChatGPT developer releases open tool to identify texts written by artificial intelligence

During testing using texts in English, the tool correctly identified 26% of materials generated by artificial intelligence, marking them as "probably written by AI" (true positives). At the same time, in 9% of cases, the classifier incorrectly determined the text created by a person as written by a neural network (false positives). It is noted that the accuracy of the tool increases with an increase in the length of the analyzed material.

OpenAI says the classifier can be useful for teachers to identify fake student work. In addition, the tool will help in identifying misinformation published in various sources. However, it is emphasized that the classifier is not completely reliable: it should not be used as the main decision-making tool, but as an addition to other methods for determining the source of text. In particular, the system is very inaccurate when processing small text fragments - less than 1000 characters. In addition, the text generated by the AI can be edited to trick the classifier. In its current form, the system is optimized for English-language materials and does not give reliable results for other languages.^[1]

Notes