News

Deep learning and sensor fusion in scattering media

Unmanned and semiautonomous vehicles need a full sensor suite which, at least, needs cameras, LiDaR and radar to cover its perception requisites. While the hardware side of is nowadays well-known at CD6, where LiDaR and hardware cameras are integrated, a full solution needs software enhanced perception for e.g. pedestrian or threat detection at long distances. Within this project, we expect to start a working line at CD6 around software-based aspects of perception and sensor fusion of imagers and high density point clouds. Furthermore, the project will investigate the use of a camera+LiDaR+radar solution for detection in scattering media, such as fog or smoke, which is a problem of obvious relevance and which can be tackled through software using approaches based on deep learning.

Autonomous driving cars (ADC) is a constant progressing research field, and a fully automated car released to the public is getting closer than ever. According to some official predictions, most cars are expected to be fully autonomous by the year 2035. In this scenario, one of the safety factors in the ADC is the detection of pedestrians in different weather conditions and different circumstances. For instance, a proper recognition and detection of pedestrians in the vicinity of the ADC in order to avoid potential road accidents.

Therefore, the objective of the present research work is to investigate, analyze, propose and develop sensor fusion algorithms that allow to integrate real time image processing and Light Detection And Ranging (LiDaR) data respectively. Further on, the proposed algorithms must be able to detect pedestrians on foggy and rainy environments. To this end, the classification of potential pedestrians in foggy and rainy regions can be processed by using the state-of-the-art of deep learning algorithms.

This project has received funding from the European Union's Horizon 2020 research and innovation program under the Marie Sklodowska-Curie grant agreement No. 712949 (TECNIOspring PLUS), as well as from the Agency for Business Competitiveness of the Generalitat de Catalunya.