AUV Obstacle Avoidance Planning Based on Deep Reinforcement Learning

In a complex underwater environment, finding a viable, collision-free path for an autonomous underwater vehicle (AUV) is a challenging task. The purpose of this paper is to establish a safe, real-time, and robust method of collision avoidance that improves the autonomy of AUVs. We propose a method b...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	Jianya Yuan, Hongjian Wang, Honghan Zhang, Changjian Lin, Dan Yu, Chengfeng Li
Formato:	article
Lenguaje:	EN
Publicado:	MDPI AG 2021
Materias:	autonomous underwater vehicle (AUV) collision avoidance planning deep reinforcement learning (DRL) double-DQN (D-DQN) Naval architecture. Shipbuilding. Marine engineering VM1-989 Oceanography GC1-1581
Acceso en línea:	https://doaj.org/article/aa4f8fea66634cbb8579ebfc57708835
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:aa4f8fea66634cbb8579ebfc57708835
record_format	dspace
spelling	oai:doaj.org-article:aa4f8fea66634cbb8579ebfc577088352021-11-25T18:03:49ZAUV Obstacle Avoidance Planning Based on Deep Reinforcement Learning10.3390/jmse91111662077-1312https://doaj.org/article/aa4f8fea66634cbb8579ebfc577088352021-10-01T00:00:00Zhttps://www.mdpi.com/2077-1312/9/11/1166https://doaj.org/toc/2077-1312In a complex underwater environment, finding a viable, collision-free path for an autonomous underwater vehicle (AUV) is a challenging task. The purpose of this paper is to establish a safe, real-time, and robust method of collision avoidance that improves the autonomy of AUVs. We propose a method based on active sonar, which utilizes a deep reinforcement learning algorithm to learn the processed sonar information to navigate the AUV in an uncertain environment. We compare the performance of double deep Q-network algorithms with that of a genetic algorithm and deep learning. We propose a line-of-sight guidance method to mitigate abrupt changes in the yaw direction and smooth the heading changes when the AUV switches trajectory. The different experimental results show that the double deep Q-network algorithms ensure excellent collision avoidance performance. The effectiveness of the algorithm proposed in this paper was verified in three environments: random static, mixed static, and complex dynamic. The results show that the proposed algorithm has significant advantages over other algorithms in terms of success rate, collision avoidance performance, and generalization ability. The double deep Q-network algorithm proposed in this paper is superior to the genetic algorithm and deep learning in terms of the running time, total path, performance in avoiding collisions with moving obstacles, and planning time for each step. After the algorithm is trained in a simulated environment, it can still perform online learning according to the information of the environment after deployment and adjust the weight of the network in real-time. These results demonstrate that the proposed approach has significant potential for practical applications.Jianya YuanHongjian WangHonghan ZhangChangjian LinDan YuChengfeng LiMDPI AGarticleautonomous underwater vehicle (AUV)collision avoidance planningdeep reinforcement learning (DRL)double-DQN (D-DQN)Naval architecture. Shipbuilding. Marine engineeringVM1-989OceanographyGC1-1581ENJournal of Marine Science and Engineering, Vol 9, Iss 1166, p 1166 (2021)
institution	DOAJ
collection	DOAJ
language	EN
topic	autonomous underwater vehicle (AUV) collision avoidance planning deep reinforcement learning (DRL) double-DQN (D-DQN) Naval architecture. Shipbuilding. Marine engineering VM1-989 Oceanography GC1-1581
spellingShingle	autonomous underwater vehicle (AUV) collision avoidance planning deep reinforcement learning (DRL) double-DQN (D-DQN) Naval architecture. Shipbuilding. Marine engineering VM1-989 Oceanography GC1-1581 Jianya Yuan Hongjian Wang Honghan Zhang Changjian Lin Dan Yu Chengfeng Li AUV Obstacle Avoidance Planning Based on Deep Reinforcement Learning
description	In a complex underwater environment, finding a viable, collision-free path for an autonomous underwater vehicle (AUV) is a challenging task. The purpose of this paper is to establish a safe, real-time, and robust method of collision avoidance that improves the autonomy of AUVs. We propose a method based on active sonar, which utilizes a deep reinforcement learning algorithm to learn the processed sonar information to navigate the AUV in an uncertain environment. We compare the performance of double deep Q-network algorithms with that of a genetic algorithm and deep learning. We propose a line-of-sight guidance method to mitigate abrupt changes in the yaw direction and smooth the heading changes when the AUV switches trajectory. The different experimental results show that the double deep Q-network algorithms ensure excellent collision avoidance performance. The effectiveness of the algorithm proposed in this paper was verified in three environments: random static, mixed static, and complex dynamic. The results show that the proposed algorithm has significant advantages over other algorithms in terms of success rate, collision avoidance performance, and generalization ability. The double deep Q-network algorithm proposed in this paper is superior to the genetic algorithm and deep learning in terms of the running time, total path, performance in avoiding collisions with moving obstacles, and planning time for each step. After the algorithm is trained in a simulated environment, it can still perform online learning according to the information of the environment after deployment and adjust the weight of the network in real-time. These results demonstrate that the proposed approach has significant potential for practical applications.
format	article
author	Jianya Yuan Hongjian Wang Honghan Zhang Changjian Lin Dan Yu Chengfeng Li
author_facet	Jianya Yuan Hongjian Wang Honghan Zhang Changjian Lin Dan Yu Chengfeng Li
author_sort	Jianya Yuan
title	AUV Obstacle Avoidance Planning Based on Deep Reinforcement Learning
title_short	AUV Obstacle Avoidance Planning Based on Deep Reinforcement Learning
title_full	AUV Obstacle Avoidance Planning Based on Deep Reinforcement Learning
title_fullStr	AUV Obstacle Avoidance Planning Based on Deep Reinforcement Learning
title_full_unstemmed	AUV Obstacle Avoidance Planning Based on Deep Reinforcement Learning
title_sort	auv obstacle avoidance planning based on deep reinforcement learning
publisher	MDPI AG
publishDate	2021
url	https://doaj.org/article/aa4f8fea66634cbb8579ebfc57708835
work_keys_str_mv	AT jianyayuan auvobstacleavoidanceplanningbasedondeepreinforcementlearning AT hongjianwang auvobstacleavoidanceplanningbasedondeepreinforcementlearning AT honghanzhang auvobstacleavoidanceplanningbasedondeepreinforcementlearning AT changjianlin auvobstacleavoidanceplanningbasedondeepreinforcementlearning AT danyu auvobstacleavoidanceplanningbasedondeepreinforcementlearning AT chengfengli auvobstacleavoidanceplanningbasedondeepreinforcementlearning
_version_	1718411699738902528

AUV Obstacle Avoidance Planning Based on Deep Reinforcement Learning

Ejemplares similares