Tactical Decision-Making for Autonomous Driving Using Dueling Double Deep Q Network With Double Attention

Decision-making is still a significant challenge to realize fully autonomous driving. Using deep reinforcement learning (DRL) to solve autonomous driving decision-making problems is a recent trend. A common method is to encode surrounding vehicles in a grid to describe the state space to help DRL ne...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	Shuwei Zhang, Yutian Wu, Harutoshi Ogai, Hiroshi Inujima, Shigeyuki Tateno
Formato:	article
Lenguaje:	EN
Publicado:	IEEE 2021
Materias:	Autonomous driving decision-making dueling double deep Q network spatial attention channel attention Electrical engineering. Electronics. Nuclear engineering TK1-9971
Acceso en línea:	https://doaj.org/article/f187f625b58e4ec29438dc584887e503
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:f187f625b58e4ec29438dc584887e503
record_format	dspace
spelling	oai:doaj.org-article:f187f625b58e4ec29438dc584887e5032021-11-18T00:00:55ZTactical Decision-Making for Autonomous Driving Using Dueling Double Deep Q Network With Double Attention2169-353610.1109/ACCESS.2021.3127105https://doaj.org/article/f187f625b58e4ec29438dc584887e5032021-01-01T00:00:00Zhttps://ieeexplore.ieee.org/document/9609990/https://doaj.org/toc/2169-3536Decision-making is still a significant challenge to realize fully autonomous driving. Using deep reinforcement learning (DRL) to solve autonomous driving decision-making problems is a recent trend. A common method is to encode surrounding vehicles in a grid to describe the state space to help DRL network extract the scene features. However, in the process of human driving, surrounding vehicles at different positions have different contributions to decision-making. Meanwhile, the network is difficult to fully extract useful features in a sparse state, which can also lead to catastrophic actions. Therefore, this work proposes a spatial attention module to calculate different weights to represent different contributions to decision-making results. And a channel attention module is proposed to fully extract useful features in sparse state features. These two attention modules are integrated into dueling double deep Q network, named D3QN-DA, as a high-level decision-maker of a hierarchical hierarchical control structure based decision-making system. To improve agent performance, an emergency safe checker is introduced in this system. And the agent is trained and test with a designed reward function from safety and efficiency in simulation. The experimental results show that the proposed method increases the safety rate by 54%, and the average explore distance by 30%, which is safer and more intelligent for the decision-making process of automatic driving.Shuwei ZhangYutian WuHarutoshi OgaiHiroshi InujimaShigeyuki TatenoIEEEarticleAutonomous drivingdecision-makingdueling double deep Q networkspatial attentionchannel attentionElectrical engineering. Electronics. Nuclear engineeringTK1-9971ENIEEE Access, Vol 9, Pp 151983-151992 (2021)
institution	DOAJ
collection	DOAJ
language	EN
topic	Autonomous driving decision-making dueling double deep Q network spatial attention channel attention Electrical engineering. Electronics. Nuclear engineering TK1-9971
spellingShingle	Autonomous driving decision-making dueling double deep Q network spatial attention channel attention Electrical engineering. Electronics. Nuclear engineering TK1-9971 Shuwei Zhang Yutian Wu Harutoshi Ogai Hiroshi Inujima Shigeyuki Tateno Tactical Decision-Making for Autonomous Driving Using Dueling Double Deep Q Network With Double Attention
description	Decision-making is still a significant challenge to realize fully autonomous driving. Using deep reinforcement learning (DRL) to solve autonomous driving decision-making problems is a recent trend. A common method is to encode surrounding vehicles in a grid to describe the state space to help DRL network extract the scene features. However, in the process of human driving, surrounding vehicles at different positions have different contributions to decision-making. Meanwhile, the network is difficult to fully extract useful features in a sparse state, which can also lead to catastrophic actions. Therefore, this work proposes a spatial attention module to calculate different weights to represent different contributions to decision-making results. And a channel attention module is proposed to fully extract useful features in sparse state features. These two attention modules are integrated into dueling double deep Q network, named D3QN-DA, as a high-level decision-maker of a hierarchical hierarchical control structure based decision-making system. To improve agent performance, an emergency safe checker is introduced in this system. And the agent is trained and test with a designed reward function from safety and efficiency in simulation. The experimental results show that the proposed method increases the safety rate by 54%, and the average explore distance by 30%, which is safer and more intelligent for the decision-making process of automatic driving.
format	article
author	Shuwei Zhang Yutian Wu Harutoshi Ogai Hiroshi Inujima Shigeyuki Tateno
author_facet	Shuwei Zhang Yutian Wu Harutoshi Ogai Hiroshi Inujima Shigeyuki Tateno
author_sort	Shuwei Zhang
title	Tactical Decision-Making for Autonomous Driving Using Dueling Double Deep Q Network With Double Attention
title_short	Tactical Decision-Making for Autonomous Driving Using Dueling Double Deep Q Network With Double Attention
title_full	Tactical Decision-Making for Autonomous Driving Using Dueling Double Deep Q Network With Double Attention
title_fullStr	Tactical Decision-Making for Autonomous Driving Using Dueling Double Deep Q Network With Double Attention
title_full_unstemmed	Tactical Decision-Making for Autonomous Driving Using Dueling Double Deep Q Network With Double Attention
title_sort	tactical decision-making for autonomous driving using dueling double deep q network with double attention
publisher	IEEE
publishDate	2021
url	https://doaj.org/article/f187f625b58e4ec29438dc584887e503
work_keys_str_mv	AT shuweizhang tacticaldecisionmakingforautonomousdrivingusingduelingdoubledeepqnetworkwithdoubleattention AT yutianwu tacticaldecisionmakingforautonomousdrivingusingduelingdoubledeepqnetworkwithdoubleattention AT harutoshiogai tacticaldecisionmakingforautonomousdrivingusingduelingdoubledeepqnetworkwithdoubleattention AT hiroshiinujima tacticaldecisionmakingforautonomousdrivingusingduelingdoubledeepqnetworkwithdoubleattention AT shigeyukitateno tacticaldecisionmakingforautonomousdrivingusingduelingdoubledeepqnetworkwithdoubleattention
_version_	1718425260251938816

Tactical Decision-Making for Autonomous Driving Using Dueling Double Deep Q Network With Double Attention

Ejemplares similares