GPT-4 (neural network)

History

2023

Turbo GPT-4 launch

On November 6, 2023, OpenAI announced the GPT-4 Turbo, a more powerful, functional and cheaper version of its large language model (LLM). GPT-4 This "super network" has received an updated knowledge base, which contains information about various events on a global scale until April 2023.

GPT-4 Turbo is available in two versions: one is exclusively for text analysis, and the second understands the context of both text and images. The cost of using the new LLM is $0.01 per 1000 input tokens (approximately 750 words). In this case, tokens are fragments of raw text: for example, the word "fantastic" will be divided into "fan," "tas" and "tic," that is, into three tokens. The price of output tokens - those that GPT-4 Turbo generates based on input - is set at $0.03 per 1000. In the case of image processing, the cost depends on their size. So, according to OpenAI, transferring an image 1080×1080 pixels in size to GPT-4 Turbo will cost $0.00765.

OpenAI announces GPT-4 Turbo

We have optimized performance, so we can set prices on Turbo GPT-4 three times lower for input tokens and two times lower for output tokens compared to GPT-4, says OpenAI.

GPT-4 Turbo has an extended context window - 128 thousand tokens, which is four times more than GPT-4. This is the largest contextual window of any commercially available LLM. The volume of 128 thousand tokens corresponds to about 100 thousand words or 300 pages, which is equivalent to approximately the content of the book "Gulliver's Travels" by Jonathan Swift. The new language model, compared to GPT-4, copes better with tasks that require careful adherence to instructions, such as generating certain formats - for example, "always respond in XML."^[1]

An AI system has been developed on the GPT-4 that quickly trains robots to perform tasks better than humans

On October 20, 2023, the research division of Nvidia Research announced the development of an artificial intelligence-based Eureka system designed to train robots in complex skills. As a result, machines are able to perform some actions even better than people. Read more here.

GPT-4 fooled AI-based defenses: model replaces weapons with apples

The scientist from Google demonstrated how the GPT-4 model bypasses the protection of other models, machine learning which emphasizes the importance chat boats as assistant researchers. This became known on August 1, 2023.

To do this, the researcher asked GPT-4 to develop a method of attack and explain how it works.

AI-Guardian was developed by Gong Zhu, Shengzhi Zhang and Kai Chen and introduced in 2023. AI-Guardian was designed to detect modified images that cheat the classifier, and was GPT-4 involved in bypassing this detection.

For example, adding additional graphic elements to the "STOP" sign can be confusing for self-driving cars. This is one example of a malicious image modification that is scanned by artificial intelligence in a car.

Carlini's work provides Python code proposed by the GPT-4 to bypass AI-Guardian protection measures against attacks. GPT-4 generated scripts and explanations for setting up images to deceive the classifier. So, the classifier may think that a photograph of a person with a weapon is a photograph of a person with an apple. The attacks reduce AI-Guardian's resilience from a reported 98% to 8%. The authors of AI-Guardian admitted that the developed bypass method successfully deceives AI-Guardian protection.

To bypass AI-Guardian protection, it was necessary to identify the mask used by AI-Guardian to detect hostile examples, showing models a variety of images that differ in only one pixel. This "brute force" technique, described by Carlini and GPT-4, ultimately allows you to identify the bypass activation function so that you can then create images to bypass it.

Carlini expects the further development of large language models (LLM).

As the calculator has changed the role of mathematicians, significantly simplifying the execution of mechanical calculations, so today's LLM models simplify the solution of programming problems, allowing scientists to spend more time developing interesting research questions, concluded Carlini[2].

Russian online school Skyeng has introduced GPT-4 technology in training

The online English school Skyeng has launched a virtual interlocutor "Keshu" based on the chatbot GPT-4. The company told about this in mid-March 2023. Read more here.

Creating a Product

On March 14, 2023, OpenAI the chatbot developer company ChatGPT unveiled a new version of its AI-based language model, designated GPT-4.

It is reported that GPT-4 is a large multimodal model trained on a huge amount of data that was not only taken from open sources on the Internet, but also licensed by the developer. These are correct and incorrect solutions to mathematical problems, reasoning of various characters, contradictory and consistent statements and much more. To train the neural network, the Microsoft Azure cloud infrastructure was used. According to OpenAI, in many real-world scenarios, GPT-4 demonstrates "human-level performance."

GPT-4 is a large multimodal model trained on a huge amount of data

The new model can take not only text, but also images as input. In a number of tasks, for example, when processing documents with text and photos, diagrams or screenshots, the GPT-4 demonstrates the same capabilities as when entering only text. Image processing functions are currently being tested, and therefore are not available to the general public.

As OpenAI notes, in a relaxed conversation, the difference between GPT-3.5 and GPT-4 can be barely noticeable. Key differences appear when the complexity of the task reaches a certain threshold - the GPT-4 model is more reliable, creative and capable of processing much thinner instructions than GPT-3.5.

At the same time, the developers say, the new model has the same disadvantages as previous versions. In particular, GPT-4 can "hallucinate" (invent facts) and make mistakes in reasoning. Sometimes a neural network can make simple logical errors and accept obvious false statements from the user. Therefore, as OpenAI notes, great care should be taken when using the output of the language model, especially in "high-stakes" tasks^[3]

Notes