Nvidia Neuralangelo

2023: Software Announcement

On June 1, 2023, the company Nvidia unveiled an open one artificial intelligence called Neuralangelo. This system is capable of forming detailed three-dimensional structures based on ordinary 2D images. You can generate, for example, realistic virtual copies of buildings, sculptures and other objects in the real world.

Using 2D video as the basis, Neuralangelo selects multiple personnel in which a scene or object is captured from different angles. This provides information about the depth, dimensions, and shape of the object. After determining the position of the camera in each frame, the AI creates a rough three-dimensional representation of the scene - like a sculptor who begins to carve a shape from a large piece of material. Further, special algorithms optimize the structure and increase detail.

Open source software for converting 2D images to 3D is presented

The end result is a 3D object or a large-scale volumetric scene. Such generated structures can be used in virtual reality applications, in games, metaverse, in the development of robotics and in many other fields.

Neuralangelo's distinctive ability is to accurately reproduce textures of complex materials such as shingles, glass and marble. In this regard, the new neural network significantly exceeds the AI models of the previous generation. The system is capable of reconstructing interiors and exteriors of buildings with high efficiency, reproducing color shades, etc. Moreover, the source 2D material can be recorded using a camera of a regular smartphone or other gadget, which greatly simplifies the process of generating 3D content.

The tool will eventually allow developers to import detailed objects - whether small statues or massive buildings - into virtual environments for their projects, said Ming-Yu Liu, senior research director at Nvidia.[1]

Notes