NAEM: Noisy Attention Exploration Module for Deep Reinforcement Learning

Recently, deep reinforcement learning (RL) has been a hot topic due to its high capability in solving complex decision-making tasks. Although deep RL has achieved remarkable results in many fields, efficient exploration remains one of its most challenging issues. Conventional exploration heuristic m...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	Zhenwen Cai, Feifei Lee, Chunyan Hu, Koji Kotani, Qiu Chen
Formato:	article
Lenguaje:	EN
Publicado:	IEEE 2021
Materias:	Attention CNNs deep RL exploration Electrical engineering. Electronics. Nuclear engineering TK1-9971
Acceso en línea:	https://doaj.org/article/2f304b4eae664799bacde718e8aa78b3
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:2f304b4eae664799bacde718e8aa78b3
record_format	dspace
spelling	oai:doaj.org-article:2f304b4eae664799bacde718e8aa78b32021-11-25T00:00:41ZNAEM: Noisy Attention Exploration Module for Deep Reinforcement Learning2169-353610.1109/ACCESS.2021.3128558https://doaj.org/article/2f304b4eae664799bacde718e8aa78b32021-01-01T00:00:00Zhttps://ieeexplore.ieee.org/document/9615161/https://doaj.org/toc/2169-3536Recently, deep reinforcement learning (RL) has been a hot topic due to its high capability in solving complex decision-making tasks. Although deep RL has achieved remarkable results in many fields, efficient exploration remains one of its most challenging issues. Conventional exploration heuristic methods, such as random exploration, have proven to be not suitable for complex RL tasks along with high-dimensional states and large action spaces. In this paper, we propose a novel lightweight and general neural network module for effective global exploration, called the Noisy Attention Exploration Module (NAEM), of which the key insight is to introduce parametric and learnable Gaussian noise into the attention mechanism for global exploration. NAEM is a general structure based on the Convolutional Block Attention Module (CBAM), which retains the ability of attention to enhance feature representation for any CNNs. In order to evaluate our module, we embed it into both value-based and actor-critic RL algorithms to test their performance improvement over related agents. The experimental results show that both of the modified agents achieve a performance improvement of more than 130% on most Atari games when compared with their original versions. In addition, for the NoisyNet agents, the training time can be reduced by about 30% through using NAEM as an alternative exploration.Zhenwen CaiFeifei LeeChunyan HuKoji KotaniQiu ChenIEEEarticleAttentionCNNsdeep RLexplorationElectrical engineering. Electronics. Nuclear engineeringTK1-9971ENIEEE Access, Vol 9, Pp 154600-154611 (2021)
institution	DOAJ
collection	DOAJ
language	EN
topic	Attention CNNs deep RL exploration Electrical engineering. Electronics. Nuclear engineering TK1-9971
spellingShingle	Attention CNNs deep RL exploration Electrical engineering. Electronics. Nuclear engineering TK1-9971 Zhenwen Cai Feifei Lee Chunyan Hu Koji Kotani Qiu Chen NAEM: Noisy Attention Exploration Module for Deep Reinforcement Learning
description	Recently, deep reinforcement learning (RL) has been a hot topic due to its high capability in solving complex decision-making tasks. Although deep RL has achieved remarkable results in many fields, efficient exploration remains one of its most challenging issues. Conventional exploration heuristic methods, such as random exploration, have proven to be not suitable for complex RL tasks along with high-dimensional states and large action spaces. In this paper, we propose a novel lightweight and general neural network module for effective global exploration, called the Noisy Attention Exploration Module (NAEM), of which the key insight is to introduce parametric and learnable Gaussian noise into the attention mechanism for global exploration. NAEM is a general structure based on the Convolutional Block Attention Module (CBAM), which retains the ability of attention to enhance feature representation for any CNNs. In order to evaluate our module, we embed it into both value-based and actor-critic RL algorithms to test their performance improvement over related agents. The experimental results show that both of the modified agents achieve a performance improvement of more than 130% on most Atari games when compared with their original versions. In addition, for the NoisyNet agents, the training time can be reduced by about 30% through using NAEM as an alternative exploration.
format	article
author	Zhenwen Cai Feifei Lee Chunyan Hu Koji Kotani Qiu Chen
author_facet	Zhenwen Cai Feifei Lee Chunyan Hu Koji Kotani Qiu Chen
author_sort	Zhenwen Cai
title	NAEM: Noisy Attention Exploration Module for Deep Reinforcement Learning
title_short	NAEM: Noisy Attention Exploration Module for Deep Reinforcement Learning
title_full	NAEM: Noisy Attention Exploration Module for Deep Reinforcement Learning
title_fullStr	NAEM: Noisy Attention Exploration Module for Deep Reinforcement Learning
title_full_unstemmed	NAEM: Noisy Attention Exploration Module for Deep Reinforcement Learning
title_sort	naem: noisy attention exploration module for deep reinforcement learning
publisher	IEEE
publishDate	2021
url	https://doaj.org/article/2f304b4eae664799bacde718e8aa78b3
work_keys_str_mv	AT zhenwencai naemnoisyattentionexplorationmodulefordeepreinforcementlearning AT feifeilee naemnoisyattentionexplorationmodulefordeepreinforcementlearning AT chunyanhu naemnoisyattentionexplorationmodulefordeepreinforcementlearning AT kojikotani naemnoisyattentionexplorationmodulefordeepreinforcementlearning AT qiuchen naemnoisyattentionexplorationmodulefordeepreinforcementlearning
_version_	1718414687200083968

NAEM: Noisy Attention Exploration Module for Deep Reinforcement Learning

Ejemplares similares