News

MISTED: Multimodal imaging for vision through turbid media

Light propagation in turbid and complex media has been a traditional source of interest in photonics research, due to the promise of seeing previously undetectable features of scenes. The recent industrial trend towards automated vehicles and advanced robotics in industry 4. 0 applications is boosting an increased interest in the topic, as long as such vehicles need to be performant in all type of conditions, especially in critical applications. This may be achieved combining different imaging modes with complementary failure modes and advanced software algorithms performing in real time.  MISTED proposes to improve the performance of the systems already existent, and to explore novel solutions, in four complementary areas: 

• Polarization modelling: Improvements in the existing model of polarized light propagation based in Mie theory to media with higher complexity, which will include distributions of scattering particles of different sizes and the inclusion of irregular shapes. This will enable to extend the current model from fog to media such as smoke, aerosols, smog or sand, much less regular than fog and with large applied interest. 

• Polarization imaging: The project has the double purpose of supporting the validation of the new polarization models previously created and the development of novel imaging modes for improved detection through fog, using the different real-time polarimetric imaging modes available at CD6 and developing a Mueller matrix imager and a polarization modulation imager.

• New imaging modes: The extension to new imaging modes of data fusion algorithms, which will be extended from the fusion of LIDAR and polarimetric imaging to other sensors, including a thermal camera (which takes advantage of temperature differences), a logarithmic camera (which withstands high intensity changes in illumination, such as those due to backscatering) and a radar (sensing with radiofrequency waves) for improved reliability in detection through turbid media. Additionally,  the use of artificial intelligence algorithms for improved perception and tracking of objects in turbid media using the available data fusion strategy, with applications on detection of small point clouds using fused information, will be also explored.

• Underwater applications: Further, a novel perception unit conceived for underwater applications, and thus with a LIDAR working in the visible, will be designed, built and tested in the lab and a pool for testing underwater vehicles.

MISTED project tackles a complex technical challenge to solve a relevant practical problem with implications in several societal fields. Vision through turbid media, the ultimate goal of the project, has the potential to improve a number of practical situations, from automated driving in bad weather to precision underwater inspection tasks or vision through smoke in fire extinction works. Several other applications can also be thought of by combining multiple imaging modes with data fusion and artificial intelligence algorithms.


The research proposed in this project has the potential to create new patents and spin-off companies exploiting in particular polarimetric imaging and the algorithmics of processing and detection in turbid media. 

Grant PID2020-119484RB-I00 funded by MCIN/AEI/10.13039/501100011033 and by the European Union NextGenerationEU/PRTR.