Visual-Robotics Bridge (VRB)

2023: Product Announcement

In mid-June 2023, a group of researchers from Carnegie Mellon University introduced the Visual-Robotics Bridge (VRB) model for training robots in household chores, such as picking up a phone, opening boxes, etc.

Until 2023, scientists trained robots, physically showing them how a particular task is performed, or training them for several weeks in a simulated environment. Both of these methods are time-consuming and resource-intensive and often unsuccessful.

Research team unveils Visual-Robotics Bridge (VRB) model to train robots on household chores

A team at Carnegie Mellon University claims that their proposed model, VRB, is able to get the robot to learn the task in just 25 minutes, and that's without human involvement or a simulated environment. This work could radically improve robot training methods and could allow robots to learn from the vast number of videos and videos available on the Internet.

VRB is an advanced version of In-the-Wild Human Imitating Robot Learning (WHIRL), a model that researchers previously used to train robots. The difference between WHIRL and VRB is that in the first case, it requires a person to perform a task in front of a robot in a specific environment. After observing a human, the robot can perform the task in the same environment. However, in VRB, a person is not required, and in a certain practice, the robot being trained can simulate human actions even in conditions other than those shown in the video.

The VRB model works on the basis of affordances, a concept that explains the possibility of action on an object. Designers use this concept to make the product user-friendly and intuitive.

In the study, scientists at Carnegie Mellon University first forced robots to view several videos from large sets of video data, such as Ego4d and Epic Kitchen. This extensive data was developed to train AI programs in human action. They then used affordance to make robots understand common ground and the steps that make the action complete, and finally they tested the two robotic platforms in different real-world conditions over 200 hours. Both robots successfully completed 12 tasks that people perform almost daily in their homes, for example, opening a can of soup, picking up a phone phone, lifting the lid, opening the door, pushing out a box, etc. In the next stages, the developers hope to use VRB to train robots in more complex multi-step tasks.^[1]

Notes