Sensor-Tekh: Charlie

The Charlie speech recognizer for independent dialog between people with a hearing disorder, deafblinds and people without restrictions of health.

2020

Function of remote communication

On March 25, 2020 the Skolkovo Foundation announced that the Charlie speech recognizer based on artificial intelligence from the resident of fund, Sensor-Tekh company, received special function of remote communication. Now people can study and work with a hearing disorder remotely. It will make their life more comfortable in the conditions of a coronavirus pandemic.

For example, the teacher at institute can put Charlie near himself during the lecture, and students with a hearing disorder, having connected to the program for the special link, will read the decrypted speech of the lecturer of the house on the screen of the smartphone or the computer online. In the same way it is possible to hold working meetings, meetings, any educational actions or performances. Only Internet access is for this purpose necessary.

Now, when very many switch over to the mode of remote work and study, leave on self-isolation, it is extremely important to support also people with disabilities. Therefore we quickly made such function for Charlie. I and will use it in the nearest future. All my students in Baumanka are transferred to remote training, in inclusive groups there are a lot of deaf children. I hope that they will estimate a product, – Denis Kuleshov, the director of Sensor-Tekh Laboratory, the teacher of MSTU of N.E. Bauman speaks

The Charlie device reproduces on the screen the decrypted speech online, and not hearing person can just read what he is told, without resorting to the help of the third parties. The gadget is often installed by houses for communication with relatives. It can be used in public places: in banks, in hospitals, at stations. Charlie already actively works in Jobcenter of the population of Moscow "My work". With its help specialists of department of employment of disabled people effectively communicate with people at whom hearing is broken, and help them to find work. Also pilot project of testing of Charlie is let into MFC of Perm. It is planned that after tests the device will be set in special windows for service of people with disabilities.

Charlie - one of the most advanced Russian developments for people with a hearing disorder. Now thanks to function of remote communication the device can be used also at distance. So, not hearing people will be able independently to read what to them is told by those who are not present nearby. Especially now, during a pandemic when we want to hear and see every day our relatives, friends, teachers, Charlie will help to feel not in complete isolation, to communicate, study, – Shcheglova Yulia, the project manager of a cluster of biomedical technologies of Skolkovo Foundation speaks

Technical characteristics and scenarios of use

Technical characteristics for March, 2020:

• Quickly: The translation of the speech in the text in real time
• Conveniently: 4 microphones for capture of the speech in "mode 360"
• Simply: Text output to the TV-screen, tablet or Braille display
• Reliably: Independent operation throughout all working day

Scenarios of use:

• Personal use

The person with a hearing disorder or the deafblind can purchase the device for the house. The text recognized by Charlie can be output to the TV-screen, the tablet or Braille's display to understand about what relatives and relatives speak.

• Use in public places

Charlie can be set in public places, for example in banks, stations, the centers of social service of the population. Besides, such technologies are very demanded at the charitable and commercial organizations which constantly work with people with the broken hearing.