CARLA Simulator Training of unmanned vehicles

Researchers from Intel, Toyota and the Spanish Center of computer vision developed the simulator of the urban environment intended for training of control algorithms by unmanned vehicles in 2017. It allows to imitate operation of different sensors and to obtain data in real time. The code of the simulator is published^[1] under the free license therefore developers can adapt it for the needs. Also researchers studied efficiency of algorithms of different types using the simulator. Development was provided at the Conference on training of robots in Google headquarters, article is published on arXiv.org^[2].

Many large car makers develop unmanned vehicles. As for independent management they should consider a huge number of factors, behavior algorithms generally do not register in these or those situations "manually" programmers, and form in learning process. Real testing at which the car moves on roads together with the engineer ready in case of the wrong actions of an algorithm is for this purpose used to intercept management.

Researchers under the leadership of Vladlen Koltun from the Center of Computer Vision in Barcelona created the simulator with the open code under the name CARLA. In it the urban environment with buildings, pedestrians, cars and other objects and also the changing weather is imitated. For rendering in the simulator the Unreal Engine 4 engine, free for non-commercial use, is used. Developers of algorithms for unmanned vehicles can connect to the simulator the algorithms through special API. At the moment in the simulator several sensors are available: the normal camera, the camera of depth and the segmenting camera classifying objects. API is also available to connection of third-party sensors.

Developers also estimated the different approaches used when training such algorithms using the system. They compared the approach called by the modular pipeline in which for processing of different input data, for example, visual acceptability, planning and management, different subsystems answer. Such approach with some differences in implementation is used in the majority of the existing management systems of unmanned vehicles. Also researchers estimated two types of terminal (end-to-end) of deep learning at which a system receives "crudest", i.e. not marked data. For assessment of such approach researchers selected the neuronets trained using simulation training at which the network tries to imitate behavior of the person, and a reinforcement with training at which she receives assessment of the actions.

They found out that different approaches in most cases had approximately identical efficiency, and their results differed less, than for 10 percent. But also researchers found out that in case weather differed from that on which the algorithm trained, modular approach had noticeable advantage, and in case of the new city differing from training on the contrary lost to other algorithms. Researchers also compared among themselves two methods of terminal deep learning, and found out that the algorithms trained coped with a reinforcement much worse in all tasks.

CARLA is an open-source simulator for autonomous driving research. CARLA has been developed from the ground up to support development, training, and validation of autonomous urban driving systems. In addition to open-source code and protocols, CARLA provides open digital assets (urban layouts, buildings, vehicles) that were created for this purpose and can be used freely. The simulation platform supports flexible specification of sensor suites and environmental conditions.

Robotics

Notes