Large Reconstruction Model (LRM)

2023: Product Announcement

In mid-November 2023, researchers from the Australian National University (ANU) and Adobe Research announced the creation of the first artificial intelligence system capable of generating 3D models based on a single 2D image. Moreover, this process takes only about 5 seconds. The technology was called the Large Reconstruction Model (LRM).

The developers say that previously, 3D modeling software worked effectively only with objects of a certain category with pre-described shapes and structure. Then came the Dall-E and Stable Diffusion systems, using diffusion models to generate images from different points of view. However, their work results are limited by the capabilities of pre-trained algorithms. The new technology offers much broader functionality.

Neural network operation diagram

LRM is based on a scalable neural network containing 1 million data sets with 500 million parameters: these are standard images, three-dimensional figures and video materials. LRM can reconstruct highly accurate 3D figures based on a wide range of 2D originals - both real-world photographs and images created with generative AI. This is achieved by using an extensive database with millions of image parameters and the ability to predict the neural radiating field (NeRF). It is argued that the system works effectively even if the original 2D images have a low resolution.

It is expected that a large model of reconstruction will find application in a variety of areas - from virtual and augmented reality to the gaming industry, cinematic animation and industrial design. The researchers say their algorithm is able to change the approach to the process of creating high-quality 3D models.^[1]

Notes