Autonomous Thermal Vision Robotic System for Victims Recognition in Search and Rescue Missions

Technological breakthroughs in recent years have led to a revolution in fields such as Machine Vision and Search and Rescue Robotics (SAR), thanks to the application and development of new and improved neural networks to vision models together with modern optical sensors that incorporate thermal cam...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	Christyan Cruz Ulloa, Guillermo Prieto Sánchez, Antonio Barrientos, Jaime Del Cerro
Formato:	article
Lenguaje:	EN
Publicado:	MDPI AG 2021
Materias:	robotic systems thermal images convolutional neural networks computer vision search and rescue robots ROS Chemical technology TP1-1185
Acceso en línea:	https://doaj.org/article/faa81c51f8644c728f9aa6019237b4f2
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:faa81c51f8644c728f9aa6019237b4f2
record_format	dspace
spelling	oai:doaj.org-article:faa81c51f8644c728f9aa6019237b4f22021-11-11T19:17:26ZAutonomous Thermal Vision Robotic System for Victims Recognition in Search and Rescue Missions10.3390/s212173461424-8220https://doaj.org/article/faa81c51f8644c728f9aa6019237b4f22021-11-01T00:00:00Zhttps://www.mdpi.com/1424-8220/21/21/7346https://doaj.org/toc/1424-8220Technological breakthroughs in recent years have led to a revolution in fields such as Machine Vision and Search and Rescue Robotics (SAR), thanks to the application and development of new and improved neural networks to vision models together with modern optical sensors that incorporate thermal cameras, capable of capturing data in post-disaster environments (PDE) with rustic conditions (low luminosity, suspended particles, obstructive materials). Due to the high risk posed by PDE because of the potential collapse of structures, electrical hazards, gas leakage, etc., primary intervention tasks such as victim identification are carried out by robotic teams, provided with specific sensors such as thermal, RGB cameras, and laser. The application of Convolutional Neural Networks (CNN) to computer vision is a breakthrough for detection algorithms. Conventional methods for victim identification in these environments use RGB image processing or trained dogs, but detection with RGB images is inefficient in the absence of light or presence of debris; on the other hand, developments with thermal images are limited to the field of surveillance. This paper’s main contribution focuses on implementing a novel automatic method based on thermal image processing and CNN for victim identification in PDE, using a Robotic System that uses a quadruped robot for data capture and transmission to the central station. The robot’s automatic data processing and control have been carried out through Robot Operating System (ROS). Several tests have been carried out in different environments to validate the proposed method, recreating PDE with varying conditions of light, from which the datasets have been generated for the training of three neural network models (Fast R-CNN, SSD, and YOLO). The method’s efficiency has been tested against another method based on CNN and RGB images for the same task showing greater effectiveness in PDE main results show that the proposed method has an efficiency greater than 90%.Christyan Cruz UlloaGuillermo Prieto SánchezAntonio BarrientosJaime Del CerroMDPI AGarticlerobotic systemsthermal imagesconvolutional neural networkscomputer visionsearch and rescue robotsROSChemical technologyTP1-1185ENSensors, Vol 21, Iss 7346, p 7346 (2021)
institution	DOAJ
collection	DOAJ
language	EN
topic	robotic systems thermal images convolutional neural networks computer vision search and rescue robots ROS Chemical technology TP1-1185
spellingShingle	robotic systems thermal images convolutional neural networks computer vision search and rescue robots ROS Chemical technology TP1-1185 Christyan Cruz Ulloa Guillermo Prieto Sánchez Antonio Barrientos Jaime Del Cerro Autonomous Thermal Vision Robotic System for Victims Recognition in Search and Rescue Missions
description	Technological breakthroughs in recent years have led to a revolution in fields such as Machine Vision and Search and Rescue Robotics (SAR), thanks to the application and development of new and improved neural networks to vision models together with modern optical sensors that incorporate thermal cameras, capable of capturing data in post-disaster environments (PDE) with rustic conditions (low luminosity, suspended particles, obstructive materials). Due to the high risk posed by PDE because of the potential collapse of structures, electrical hazards, gas leakage, etc., primary intervention tasks such as victim identification are carried out by robotic teams, provided with specific sensors such as thermal, RGB cameras, and laser. The application of Convolutional Neural Networks (CNN) to computer vision is a breakthrough for detection algorithms. Conventional methods for victim identification in these environments use RGB image processing or trained dogs, but detection with RGB images is inefficient in the absence of light or presence of debris; on the other hand, developments with thermal images are limited to the field of surveillance. This paper’s main contribution focuses on implementing a novel automatic method based on thermal image processing and CNN for victim identification in PDE, using a Robotic System that uses a quadruped robot for data capture and transmission to the central station. The robot’s automatic data processing and control have been carried out through Robot Operating System (ROS). Several tests have been carried out in different environments to validate the proposed method, recreating PDE with varying conditions of light, from which the datasets have been generated for the training of three neural network models (Fast R-CNN, SSD, and YOLO). The method’s efficiency has been tested against another method based on CNN and RGB images for the same task showing greater effectiveness in PDE main results show that the proposed method has an efficiency greater than 90%.
format	article
author	Christyan Cruz Ulloa Guillermo Prieto Sánchez Antonio Barrientos Jaime Del Cerro
author_facet	Christyan Cruz Ulloa Guillermo Prieto Sánchez Antonio Barrientos Jaime Del Cerro
author_sort	Christyan Cruz Ulloa
title	Autonomous Thermal Vision Robotic System for Victims Recognition in Search and Rescue Missions
title_short	Autonomous Thermal Vision Robotic System for Victims Recognition in Search and Rescue Missions
title_full	Autonomous Thermal Vision Robotic System for Victims Recognition in Search and Rescue Missions
title_fullStr	Autonomous Thermal Vision Robotic System for Victims Recognition in Search and Rescue Missions
title_full_unstemmed	Autonomous Thermal Vision Robotic System for Victims Recognition in Search and Rescue Missions
title_sort	autonomous thermal vision robotic system for victims recognition in search and rescue missions
publisher	MDPI AG
publishDate	2021
url	https://doaj.org/article/faa81c51f8644c728f9aa6019237b4f2
work_keys_str_mv	AT christyancruzulloa autonomousthermalvisionroboticsystemforvictimsrecognitioninsearchandrescuemissions AT guillermoprietosanchez autonomousthermalvisionroboticsystemforvictimsrecognitioninsearchandrescuemissions AT antoniobarrientos autonomousthermalvisionroboticsystemforvictimsrecognitioninsearchandrescuemissions AT jaimedelcerro autonomousthermalvisionroboticsystemforvictimsrecognitioninsearchandrescuemissions
_version_	1718431622623133696

Autonomous Thermal Vision Robotic System for Victims Recognition in Search and Rescue Missions

Ejemplares similares