Robotic cane

2021: Announcing Robotics with 3D Camera for the Blind

In mid-September 2021, a robotic cane with a 3D camera is presented, which can accurately guide the user to the chosen place, avoiding obstacles. The novelty has already begun to be used by blind people.

Equipped with a color 3D camera, an inertial measurement sensor and its own on-board computer, a robotic cane can offer blind and visually impaired users a new way to navigate the premises. In tandem with the architectural drawing of the building, the device can accurately guide the user to the right place using sensory and auditory prompts, while helping him to avoid obstacles such as boxes, furniture and protrusions. The development of the device was carried out jointly by the National Eye Institute (NEI) and the National Institute of Biomedical Imaging and Bioengineering (NIBIB). Details of the updated design were published in the IEEE/CAA Journal of Automatica Sinica.

Robotics with 3D camera for the blind

Many people in the visually impaired community consider the white cane to be their best and most functional navigation tool, despite the fact that it is a technology a century ago, said Kang Ye, PhD, lead author of the study and professor of computer science.

Although there are mobile phone-based applications that can provide navigation assistance, for example, to help blind users stay at pedestrian crossings, large spaces inside buildings pose a serious problem, especially when these spaces are unfamiliar. Early versions of the robotic cane from Kang Ye began to solve this problem by including plans for buildings, and the user could also tell the cane where he or she wanted to go, and the cane using a combination of auditory signals and a robotic tip could lead the user to the destination. But when used at long distances, inaccuracies in determining the user's location could accumulate, as a result, the user was in the wrong place.

To solve this problem, Kang Ye and his colleagues added a color depth camera to the system. Using infrared light, like the front camera of a mobile phone, the system can determine the distance between the cane and other physical objects, including the floor, doorways and walls, as well as furniture and other obstacles. Using this information, as well as data from an inertial sensor, the on-board cane computer can apply the exact location of the user to an existing architectural drawing or floor plan, as well as warn the user about obstacles to his path.

Before the system enters the market, several problems have yet to be solved, for example, it is still too heavy for regular use and the professor's team is looking for a way to reduce the size of the device. The device can become a key tool of independence for blind and visually impaired people, without losing characteristics.^[1]

Notes