Multimodal sensor based on lidar for underwater environmental sensing
In the MARINEROS project, an underwater multimodal imager combining RGB and polarimetric cameras with, an underwater lidar built ad hoc, and a multibeam sonar will be designed, built and set in operation, and applied to different use cases. The project intends to act as an enabler of applications related to unmanned or remotely operated underwater vehicles, much in the same way that multimodal sensors have become an enabler for automotive and other ground-based vehicles. For this, the project builds on the experience of the IP and the research group on multimodal and lidar hardware development, and teams up with a number of experts and external collaborators in image processing and in the underwater imaging and vehicle integration worlds.
The project challenge pivots mostly around the development of the underwater lidar, a type of development which is not easily available due to the complexities of working in underwater media, where water absorption, turbulence, turbidity and changes in refraction index make lidar imaging significantly more challenging than in the atmosphere. Data fusion using different types of sensors will enable to complement the failure modes of the different imaging strategies and provide a robust underwater sensing suite.
The developed multimodal sensor will be a customized hardware unit that will be used to develop algorithms for enhanced underwater imaging via different software developments, oriented to significantly improve underwater imaging. In particular, thanks to the use of the lidar and of our experience in data fusion of imaging modes, we intend to obtain:
a) enhanced underwater colour reconstruction at all distances;
b) enhanced imaging through scattering media via active polarimetric imaging;
c) enhanced mapping using visual SLAM algorithms;
d) feature detection using deep neural networks. The challenge will extend to the underwater world the existing know-how in multimodal imaging in scattering media at CD6 developed in past projects of the Plan Nacional de I+D.
The project will develop hardware and software, and perform a number of underwater tests in external underwater installations taking advantage of the infrastructure available at the Submarine Robotics Group at Universitat de Girona (VICOROB). For this, the multimodal imager will undergo a number of simple testing installations in underwater vehicles for pool tests, which will end in a final demo in a real environment, after a full integration procedure of the sensor in an underwater vehicle. Professor J.Forest of VICOROB will act as an external collaborator to the project supporting it from his experience in underwater imaging, 3D mapping, and vehicle integration.
The multimodal imager will be demonstrated in different underwater use cases searching for potential applied publications and for technology transfer of the results, a field in which the IP is an expert. Initial targeted applications involved mapping of the seabed, inspection and maintenance of underwater infrastructure, and applications in marine zoology. In all cases, the use of an underwater multimodal imager paves the way for fully autonomous, remotely operated vehicles, with all the continuous monitoring capabilities of underwater applications this may bring in the underwater world.
Grant PID2023-1524270B-I00 funded by MCIU /AEI /10.13039/501100011033 / FEDER, UE.
The project challenge pivots mostly around the development of the underwater lidar, a type of development which is not easily available due to the complexities of working in underwater media, where water absorption, turbulence, turbidity and changes in refraction index make lidar imaging significantly more challenging than in the atmosphere. Data fusion using different types of sensors will enable to complement the failure modes of the different imaging strategies and provide a robust underwater sensing suite.
The developed multimodal sensor will be a customized hardware unit that will be used to develop algorithms for enhanced underwater imaging via different software developments, oriented to significantly improve underwater imaging. In particular, thanks to the use of the lidar and of our experience in data fusion of imaging modes, we intend to obtain:
a) enhanced underwater colour reconstruction at all distances;
b) enhanced imaging through scattering media via active polarimetric imaging;
c) enhanced mapping using visual SLAM algorithms;
d) feature detection using deep neural networks. The challenge will extend to the underwater world the existing know-how in multimodal imaging in scattering media at CD6 developed in past projects of the Plan Nacional de I+D.
The project will develop hardware and software, and perform a number of underwater tests in external underwater installations taking advantage of the infrastructure available at the Submarine Robotics Group at Universitat de Girona (VICOROB). For this, the multimodal imager will undergo a number of simple testing installations in underwater vehicles for pool tests, which will end in a final demo in a real environment, after a full integration procedure of the sensor in an underwater vehicle. Professor J.Forest of VICOROB will act as an external collaborator to the project supporting it from his experience in underwater imaging, 3D mapping, and vehicle integration.
The multimodal imager will be demonstrated in different underwater use cases searching for potential applied publications and for technology transfer of the results, a field in which the IP is an expert. Initial targeted applications involved mapping of the seabed, inspection and maintenance of underwater infrastructure, and applications in marine zoology. In all cases, the use of an underwater multimodal imager paves the way for fully autonomous, remotely operated vehicles, with all the continuous monitoring capabilities of underwater applications this may bring in the underwater world.
Grant PID2023-1524270B-I00 funded by MCIU /AEI /10.13039/501100011033 / FEDER, UE.