News

Deep learning-sensor fusion architecture for pedestrian and object detection in scattering media

Unmanned and semiautonomous vehicles need a full sensor suite which, at least, needs cameras, LIDAR (Light Detection And Ranging) and radar to cover its perception requisites. While the hardware side is nowadays well-known at CD6, where LIDAR and hardware cameras are integrated, a full solution needs software enhanced perception for e.g. pedestrian or threat detection at long distances. This project will help to boost a working line at CD6 around software-based aspects of perception and sensor fusion of imagers and high density point clouds.

The project will investigate the use of a camera+LIDAR solution for detection in scattering media, such as fog or smoke, which is a problem of obvious relevance which can be tackled through software using approaches based on deep learning. The objective of the present research work is to investigate, analyze, propose and develop sensor fusion algorithms that allow to integrate real time image processing from different imaging sensors, radar and LIDAR data respectively. Such an approach with multiple sensors is expected to cover the different failure modes of the sensors and thus provide a solid working environment in all weather conditions.

This project has received funding from the European Union's Horizon 2020 research and innovation program under the Marie Skłodowska-Curie grant agreement No 712949 (TECNIOspring PLUS), as well as from the Agency for the Competitiveness of the Government of Catalonia