Google Lumiere

2024: Neural Network Launch

In January 2024, Google launched an open neural network to create text-based video. The service was named Lumiere.

Unlike competing projects, Lumiere creates video from start to finish in a single process, the developers said. Other similar services first generate key parts of personnel, after which they increase their resolution.

Google launched a neural network to create text-based videos

Lumiere works in several modes, for example, converts text to video, converts static images to dynamic images, creates videos in a given style based on a sample, allows you to edit an existing video with written prompts, animates certain areas of a static image or edits video fragmentally - for example, it can change the subject of a wardrobe on a person.

According to the project's website, the T2V AI model that underlies Lumiere has been trained on 30 million text description videos. The videos are 80 personnel long at 16 frames per second and have a resolution of 128 x 128 pixels.

Judging by the demonstrations, Lumiere has the most advanced drawing capabilities. You can close part of the image, and Lumiere will automatically fill this area - so organically that the intervention of artificial intelligence will be impossible to notice. Videos created with Lumiere are realistic, but very short. Their duration does not exceed 5 seconds. It is assumed that in the future the time limit will be increased.

By January 2024, Lumiere is available to everyone, but is a research project, which eliminates the need for Google to restrict the system within the framework of copyright, disinformation, security, hatred, nudity, privacy policies, etc. Hypothetically, it can be used to create high-quality video deepfakes on absolutely any topic.^[1]

Notes