Cognitive Feedback

The main articles are:

• Neural networks (neural networks)
• What is the Internet of Things (IoT)
• Internet of Things in Agriculture (IoTAg)
• IT in the agro-industrial complex in the world
• IT in the agro-industrial complex of Russia
• Artificial intelligence in agriculture (agro-industrial complex)

2023: Cognitive Feedback Technology Development

The Russian developer of autonomous control systems for agricultural transport, Cognitive Pilot (a subsidiary of Sberbank and Cognitive Technologies) announced on April 12, 2023 the development of R2D-class technology (Robot to Driver - interaction of a robot with a driver) for agro-robots. Its analogues for April 2023 are not known.

In fact, we are talking about the formation of market R2D solutions. " According to the estimates of the InterAgroTech association, its volume Russia in 2026 may exceed 6 billion, rub reported to Cognitive Pilot.

The volume world technology market of equipment for, pilotless agricultural machinery according to to data analysts ReportLinker, in 2022 was estimated at 79.5 billion. dollars It is expected that by 2030 this figure will amount to $231.8 billion, with a dynamics of 14.3% per year.

Together with the creation of autonomous management systems for agricultural machinery based on AI, the company is actively engaged in the development of service, analytical, telematic and other solutions that make working in this zone effective and comfortable. The company confirms its trendsetter status and continues to open technology markets, said Cognitive Pilot CEO Olga Uskova.

One of the significant problems with the industrial use of smart autopilots for agricultural machinery was the lack of feedback from the combine harvester or tractor driver with the robot.

Machine operators are largely ready to trust artificial intelligence, but in private conversations they often complained about the lack of feedback from the autopilot, information about how reliably its work is going, whether a problematic or abnormal situation has arisen. Sometimes the combines were reinsured and switched to manual mode, when it seemed to them that the camera was clogged or the site was complex, which reduced the efficiency of the work, explained in the Cognitive Pilot company.

To solve this and related problems, Cognitive Pilot has developed Cognitive Feedback technology, which allows you to determine the level of reliability of AI throughout the autopilot and interact with the machine operator - inform it about the absence of problems, as well as potential cases of unreliable operation of artificial intelligence, warn about the occurrence of problematic situations, which can be caused by severe weather conditions (thick fog, dust), poor field condition, contamination of the camera sensor, etc. Then, in this area, the combine must switch to manual control mode.

To build a system for assessing the reliability of the system of autonomous control of agricultural machinery based on the Cognitive Agro Pilot AI, the company's specialists created a neural network architecture. Its capabilities can be clearly demonstrated on the example of smart harvesting of ordinary crops (corn, sunflower, etc.). Within the framework of this architecture, it became possible to determine each isolated component of the inter-row (see Figure). Thanks to this, the network recognizes rows in more detail, which has a positive effect on the accuracy and smoothness of the combine control.

Compare the level of detail of rows on different networks. On the left, an approach is used where each component is isolated from each other. On the right is the classical semantic segmentation approach. In the far plane, it is clearly seen that the semantic segmentation network is not able to separate rows.

Compare the level of detail of rows on different networks. On the left, an approach is used where each component is isolated from each other. On the right is the classical semantic segmentation approach. In the far plane, it is clearly seen that the semantic segmentation network is not able to separate rows.

Further, based on the analysis of each component of the row, a comprehensive assessment of the reliability of the neural network on the entire frame is created. In the case of edge harvesting (wheat, barley, oats, etc.), the system determines the geometry of the beveled/unshaded boundary based on its analysis and returns a reliability estimate. If the boundary geometry between these regions is expected, then the reliability index is high. If unpaved areas are left, reliability is low. This gives an idea of ​ ​ how steadily the neural network works at the harvest site in robot mode. The reliability score ranges from 0 to 100, and a level above sixty is considered a reliable metric. Cognitive Feedback allows you to automatically identify problematic scenes - scenes with low neural network reliability. They are transmitted to engineers, after which the analysis of these locations is carried out and, if necessary, the process of updating the neural network is launched.

The system also allows you to collect analytics that are extremely necessary for analyzing problem areas, developing and improving the system. The monitoring subsystem built into the agropilot allows you to time create real-time reports on the movement of the combine, its speed and geolocation. Data on the progress of operations from the aircraft cars are transmitted according to the built-in -. In GSMto the modem the process of cleaning fields, telemetry also records information about the mode in which the agropilot worked.

Finally, the system will allow you to control situations when the agropylote was not involved with a high level of reliability of the artificial intelligence system. In other words, if the machine operator deliberately turned off the autopilot and transmits this data to the owner of the farm.

According to experts, Cognitive Feedback will increase cleaning efficiency by 20-25% due to a clear understanding by the machine operator of the level of reliability of the AI and its adequate response to the situation on the field, elimination of problems, through the use of analytics, as well as control over the machine operator and minimizing the human factor.