VR bow to launch swarm of drones

2022: VR Bow Development for Drones

At the end of October 2022, Skolkovo reported that it had developed a method for deploying a swarm of rescue and research drones. Using a virtual reality helmet and tactile interface, the operator simulates a series of bow shots, directing each drone to a given point. Deep reinforcement learning is used to prevent drone collisions. "

The drone swarm operator uses a VR helmet, marker gloves and a tactile display. Around the "archer" there is a motion capture system consisting of several IR cameras - they register the position of gloves and drones. Thanks to the tactile display, the operator feels how the tension of the virtual string changes when the distance between the gloves changes, and uses this feedback to set the distance to the point where the drone should be. At the same time, the VR helmet in real time visualizes the ballistic trajectory of the drone. When the operator unclamps the fist, this action registers the camera, and the drone is guided along a given trajectory.

In Russia, developed VR-bow for launching drones

As soon as the drone reaches the target, it is transferred to the traditional mode of control and proceeds to fulfill its mission. This can be a rescue operation, forest fire detection, monitoring the state of natural resources, environmental pollution, determining the growth index (NDVI) of crops, geological exploration, inspection or maintenance of infrastructure facilities.

According to the developers, thanks to the use of deep training with reinforcement in the new method, drones are able to anticipate collisions with other drones and adjust their trajectory accordingly.

The goal of our research is to overcome an even more complex challenge: how a user without drone control experience can deploy a whole swarm in a matter of seconds, "says Associate Professor Dmitry Teteryukov, head of research and laboratory of intelligent space robotics Skoltech.