Droidlet (robot development platform)

2021: Announcement of an open platform for creating robots

At the end of July 2021, it became known that Facebook posted in open access Droidlet, a platform for creating robots that use natural language processing and computer vision to understand the world around them. According to the developers, their solution simplifies the integration of machine learning algorithms into robots, helping to quickly create prototypes of software.

As of July 30, 2021 robots can be forced to vacuum the floor or perform a dance, but they are not capable of more. This is because they cannot process information at a deep level because they cannot recognize what a chair is or, for example, do not understand that hitting a spilled soda can will lead to even more disorder.

Droidlet allows developers to create robots that perform tasks in the real world (right) or in simulated environments

Droidlet is rather a way to test different models of computer vision and natural language processing. The platform allows researchers to create systems that can perform tasks in the real world or in simulated environments, such as Minecraft or Facebook's Habitat, supporting the use of the same system on different robots by replacing components as needed. The platform provides a dashboard to which researchers can add widgets and debugging and visualization tools, as well as an interface for correcting errors and the annotation itself.

The platform consists of a set of components, some of them heuristic, and some trainees who can be trained on static data when convenient. Training is also possible on dynamic data.

The Droidlet platform supports researchers. In general, reducing friction when integrating machine learning models and new capabilities, both scripted and trained, into their systems, as well as providing user experience for interaction. As more researchers work with the platform, they will improve existing components and add new ones that others, in turn, will be able to add to their own robotics projects... With Droidlet, robotics researchers can now take advantage of significant progress in AI and create machines that can effectively respond to complex voice commands, Facebook writes on its blog.[1][2]

Notes