Hierarchical Active Tracking Control for UAVs via Deep Reinforcement Learning

Active tracking control is essential for UAVs to perform autonomous operations in GPS-denied environments. In the active tracking task, UAVs take high-dimensional raw images as input and execute motor actions to actively follow the dynamic target. Most research focuses on three-stage methods, which...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	Wenlong Zhao, Zhijun Meng, Kaipeng Wang, Jiahui Zhang, Shaoze Lu
Formato:	article
Lenguaje:	EN
Publicado:	MDPI AG 2021
Materias:	unmanned aerial vehicle deep reinforcement learning visual active tracking Technology T Engineering (General). Civil engineering (General) TA1-2040 Biology (General) QH301-705.5 Physics QC1-999 Chemistry QD1-999
Acceso en línea:	https://doaj.org/article/5ead3f0d10e6402295ea277f5f0a9cf4
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:5ead3f0d10e6402295ea277f5f0a9cf4
record_format	dspace
spelling	oai:doaj.org-article:5ead3f0d10e6402295ea277f5f0a9cf42021-11-25T16:32:32ZHierarchical Active Tracking Control for UAVs via Deep Reinforcement Learning10.3390/app1122105952076-3417https://doaj.org/article/5ead3f0d10e6402295ea277f5f0a9cf42021-11-01T00:00:00Zhttps://www.mdpi.com/2076-3417/11/22/10595https://doaj.org/toc/2076-3417Active tracking control is essential for UAVs to perform autonomous operations in GPS-denied environments. In the active tracking task, UAVs take high-dimensional raw images as input and execute motor actions to actively follow the dynamic target. Most research focuses on three-stage methods, which entail perception first, followed by high-level decision-making based on extracted spatial information of the dynamic target, and then UAV movement control, using a low-level dynamic controller. Perception methods based on deep neural networks are powerful but require considerable effort for manual ground truth labeling. Instead, we unify the perception and decision-making stages using a high-level controller and then leverage deep reinforcement learning to learn the mapping from raw images to the high-level action commands in the V-REP-based environment, where simulation data are infinite and inexpensive. This end-to-end method also has the advantages of a small parameter size and reduced effort requirements for parameter turning in the decision-making stage. The high-level controller, which has a novel architecture, explicitly encodes the spatial and temporal features of the dynamic target. Auxiliary segmentation and motion-in-depth losses are introduced to generate denser training signals for the high-level controller’s fast and stable training. The high-level controller and a conventional low-level PID controller constitute our hierarchical active tracking control framework for the UAVs’ active tracking task. Simulation experiments show that our controller trained with several augmentation techniques sufficiently generalizes dynamic targets with random appearances and velocities, and achieves significantly better performance, compared with three-stage methods.Wenlong ZhaoZhijun MengKaipeng WangJiahui ZhangShaoze LuMDPI AGarticleunmanned aerial vehicledeep reinforcement learningvisual active trackingTechnologyTEngineering (General). Civil engineering (General)TA1-2040Biology (General)QH301-705.5PhysicsQC1-999ChemistryQD1-999ENApplied Sciences, Vol 11, Iss 10595, p 10595 (2021)
institution	DOAJ
collection	DOAJ
language	EN
topic	unmanned aerial vehicle deep reinforcement learning visual active tracking Technology T Engineering (General). Civil engineering (General) TA1-2040 Biology (General) QH301-705.5 Physics QC1-999 Chemistry QD1-999
spellingShingle	unmanned aerial vehicle deep reinforcement learning visual active tracking Technology T Engineering (General). Civil engineering (General) TA1-2040 Biology (General) QH301-705.5 Physics QC1-999 Chemistry QD1-999 Wenlong Zhao Zhijun Meng Kaipeng Wang Jiahui Zhang Shaoze Lu Hierarchical Active Tracking Control for UAVs via Deep Reinforcement Learning
description	Active tracking control is essential for UAVs to perform autonomous operations in GPS-denied environments. In the active tracking task, UAVs take high-dimensional raw images as input and execute motor actions to actively follow the dynamic target. Most research focuses on three-stage methods, which entail perception first, followed by high-level decision-making based on extracted spatial information of the dynamic target, and then UAV movement control, using a low-level dynamic controller. Perception methods based on deep neural networks are powerful but require considerable effort for manual ground truth labeling. Instead, we unify the perception and decision-making stages using a high-level controller and then leverage deep reinforcement learning to learn the mapping from raw images to the high-level action commands in the V-REP-based environment, where simulation data are infinite and inexpensive. This end-to-end method also has the advantages of a small parameter size and reduced effort requirements for parameter turning in the decision-making stage. The high-level controller, which has a novel architecture, explicitly encodes the spatial and temporal features of the dynamic target. Auxiliary segmentation and motion-in-depth losses are introduced to generate denser training signals for the high-level controller’s fast and stable training. The high-level controller and a conventional low-level PID controller constitute our hierarchical active tracking control framework for the UAVs’ active tracking task. Simulation experiments show that our controller trained with several augmentation techniques sufficiently generalizes dynamic targets with random appearances and velocities, and achieves significantly better performance, compared with three-stage methods.
format	article
author	Wenlong Zhao Zhijun Meng Kaipeng Wang Jiahui Zhang Shaoze Lu
author_facet	Wenlong Zhao Zhijun Meng Kaipeng Wang Jiahui Zhang Shaoze Lu
author_sort	Wenlong Zhao
title	Hierarchical Active Tracking Control for UAVs via Deep Reinforcement Learning
title_short	Hierarchical Active Tracking Control for UAVs via Deep Reinforcement Learning
title_full	Hierarchical Active Tracking Control for UAVs via Deep Reinforcement Learning
title_fullStr	Hierarchical Active Tracking Control for UAVs via Deep Reinforcement Learning
title_full_unstemmed	Hierarchical Active Tracking Control for UAVs via Deep Reinforcement Learning
title_sort	hierarchical active tracking control for uavs via deep reinforcement learning
publisher	MDPI AG
publishDate	2021
url	https://doaj.org/article/5ead3f0d10e6402295ea277f5f0a9cf4
work_keys_str_mv	AT wenlongzhao hierarchicalactivetrackingcontrolforuavsviadeepreinforcementlearning AT zhijunmeng hierarchicalactivetrackingcontrolforuavsviadeepreinforcementlearning AT kaipengwang hierarchicalactivetrackingcontrolforuavsviadeepreinforcementlearning AT jiahuizhang hierarchicalactivetrackingcontrolforuavsviadeepreinforcementlearning AT shaozelu hierarchicalactivetrackingcontrolforuavsviadeepreinforcementlearning
_version_	1718413137589305344

Hierarchical Active Tracking Control for UAVs via Deep Reinforcement Learning

Ejemplares similares