RHyME (Retrieval for Hybrid Imitation under Mismatched Execution)

History

2025: Product Announcement

Cornell University researchers have developed an innovative artificial intelligence system, RHyME (Retrieval for Hybrid Imitation under Mismatched Execution), allowing robots to master new tasks after watching a training video once. Scientists reported this in April 2025, publishing the results of the study and demonstration materials.

According to the developers, aditional training of robots required the creation of accurate step-by-step algorithms, and any deviation from a given program - for example, falling a tool or shifting an object - led to the impossibility of performing a task. The new RHyME technology fundamentally changes this approach, allowing robots to learn by observing human actions, similar to how humans learn from each other.

𝓲

Фото: Freepik

Co-author Kushal Kedia explained that one of the most tedious aspects of working with robots was the need to collect a huge amount of data for each action. At the same time, people learn completely differently - just watching the actions of others and repeating them.

The main advantage of the RHyME system is the ability of the robot to reproduce the task even if its actions do not coincide with the actions of a person on the training video. This is especially important because the movements of people are too smooth and varied for direct copying by mechanical systems.

Traditional algorithms did not work if the demonstration in the video was at least slightly different from the physical capabilities of the robot. RHyME allows the machine to analyze its own "memory" in search of similar movements and actions in order to adapt what it sees to its capabilities and fill in the missing elements.

For example, if a robot shows a video where a person takes a cup from the kitchen and puts it in the sink, the system will not require an accurate repetition of all movements. Instead, RHyME will allow the robot to turn to other records where it has already performed similar actions, such as lifting objects or placing them on the surface, and based on this experience to form the desired behavior.^[1]

Notes