F3RM (Feature Fields for Robotic Manipulation)

2023: Product Announcement

On November 2, 2023, scientists from MIT talked about a new method for training robots using artificial intelligence. It was named Feature Fields for Robotic Manipulation.

Researchers talk about the principle of the method, comparing it with a person who came to visit a friend in another country. In the morning, a person tries to make breakfast and opens a refrigerator for this. Initially, most elements seem incomprehensible to him due to unfamiliar packaging. Despite visual differences with familiar elements, a person gradually understands what is used for.

MIT scientists talk about a new method of training robots

This hypothetical situation inspired scientists to create the Feature Fields for Robotic Manipulation (F3RM) model. This system turns 2D images of the main models on which AI is trained into 3D models so that it is easier for the robot to identify and interact with the objects surrounding it. F3RM can understand open language instructions, which helps them in a real environment where there are thousands of objects around them.

Creating robots that really can generalize when in real conditions is a very difficult task. We are trying to understand how this can be done, and therefore we are trying to reach a new level of generalization, from three to four objects to everything we can find in our research center, commented Je Yang, one of the authors of the study.

Thanks to F3RM, robots are able to interpret open text instructions using natural language. As a result of this, machines can understand less specialized requests of people and perform these tasks. For example, the user may ask to "raise a tall cup," and the robot will find and raise an item suitable for the description.^[1]

Notes