Augmented Reality Assisted Assembly Training Oriented Dynamic Gesture Recognition and Prediction

Augmented reality assisted assembly training (ARAAT) is an effective and affordable technique for labor training in the automobile and electronic industry. In general, most tasks of ARAAT are conducted by real-time hand operations. In this paper, we propose an algorithm of dynamic gesture recognitio...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	Jiaqi Dong, Zeyang Xia, Qunfei Zhao
Formato:	article
Lenguaje:	EN
Publicado:	MDPI AG 2021
Materias:	augmented reality assisted assembly training human-machine interaction gesture recognition and prediction Technology T Engineering (General). Civil engineering (General) TA1-2040 Biology (General) QH301-705.5 Physics QC1-999 Chemistry QD1-999
Acceso en línea:	https://doaj.org/article/c69b23c7ea9943d0bd3b6c0ea624b113
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:c69b23c7ea9943d0bd3b6c0ea624b113
record_format	dspace
spelling	oai:doaj.org-article:c69b23c7ea9943d0bd3b6c0ea624b1132021-11-11T14:56:51ZAugmented Reality Assisted Assembly Training Oriented Dynamic Gesture Recognition and Prediction10.3390/app112197892076-3417https://doaj.org/article/c69b23c7ea9943d0bd3b6c0ea624b1132021-10-01T00:00:00Zhttps://www.mdpi.com/2076-3417/11/21/9789https://doaj.org/toc/2076-3417Augmented reality assisted assembly training (ARAAT) is an effective and affordable technique for labor training in the automobile and electronic industry. In general, most tasks of ARAAT are conducted by real-time hand operations. In this paper, we propose an algorithm of dynamic gesture recognition and prediction that aims to evaluate the standard and achievement of the hand operations for a given task in ARAAT. We consider that the given task can be decomposed into a series of hand operations and furthermore each hand operation into several continuous actions. Then, each action is related with a standard gesture based on the practical assembly task such that the standard and achievement of the actions included in the operations can be identified and predicted by the sequences of gestures instead of the performance throughout the whole task. Based on the practical industrial assembly, we specified five typical tasks, three typical operations, and six standard actions. We used Zernike moments combined histogram of oriented gradient and linear interpolation motion trajectories to represent 2D static and 3D dynamic features of standard gestures, respectively, and chose the directional pulse-coupled neural network as the classifier to recognize the gestures. In addition, we defined an action unit to reduce the dimensions of features and computational cost. During gesture recognition, we optimized the gesture boundaries iteratively by calculating the score probability density distribution to reduce interferences of invalid gestures and improve precision. The proposed algorithm was evaluated on four datasets and proved to increase recognition accuracy and reduce the computational cost from the experimental results.Jiaqi DongZeyang XiaQunfei ZhaoMDPI AGarticleaugmented reality assisted assembly traininghuman-machine interactiongesture recognition and predictionTechnologyTEngineering (General). Civil engineering (General)TA1-2040Biology (General)QH301-705.5PhysicsQC1-999ChemistryQD1-999ENApplied Sciences, Vol 11, Iss 9789, p 9789 (2021)
institution	DOAJ
collection	DOAJ
language	EN
topic	augmented reality assisted assembly training human-machine interaction gesture recognition and prediction Technology T Engineering (General). Civil engineering (General) TA1-2040 Biology (General) QH301-705.5 Physics QC1-999 Chemistry QD1-999
spellingShingle	augmented reality assisted assembly training human-machine interaction gesture recognition and prediction Technology T Engineering (General). Civil engineering (General) TA1-2040 Biology (General) QH301-705.5 Physics QC1-999 Chemistry QD1-999 Jiaqi Dong Zeyang Xia Qunfei Zhao Augmented Reality Assisted Assembly Training Oriented Dynamic Gesture Recognition and Prediction
description	Augmented reality assisted assembly training (ARAAT) is an effective and affordable technique for labor training in the automobile and electronic industry. In general, most tasks of ARAAT are conducted by real-time hand operations. In this paper, we propose an algorithm of dynamic gesture recognition and prediction that aims to evaluate the standard and achievement of the hand operations for a given task in ARAAT. We consider that the given task can be decomposed into a series of hand operations and furthermore each hand operation into several continuous actions. Then, each action is related with a standard gesture based on the practical assembly task such that the standard and achievement of the actions included in the operations can be identified and predicted by the sequences of gestures instead of the performance throughout the whole task. Based on the practical industrial assembly, we specified five typical tasks, three typical operations, and six standard actions. We used Zernike moments combined histogram of oriented gradient and linear interpolation motion trajectories to represent 2D static and 3D dynamic features of standard gestures, respectively, and chose the directional pulse-coupled neural network as the classifier to recognize the gestures. In addition, we defined an action unit to reduce the dimensions of features and computational cost. During gesture recognition, we optimized the gesture boundaries iteratively by calculating the score probability density distribution to reduce interferences of invalid gestures and improve precision. The proposed algorithm was evaluated on four datasets and proved to increase recognition accuracy and reduce the computational cost from the experimental results.
format	article
author	Jiaqi Dong Zeyang Xia Qunfei Zhao
author_facet	Jiaqi Dong Zeyang Xia Qunfei Zhao
author_sort	Jiaqi Dong
title	Augmented Reality Assisted Assembly Training Oriented Dynamic Gesture Recognition and Prediction
title_short	Augmented Reality Assisted Assembly Training Oriented Dynamic Gesture Recognition and Prediction
title_full	Augmented Reality Assisted Assembly Training Oriented Dynamic Gesture Recognition and Prediction
title_fullStr	Augmented Reality Assisted Assembly Training Oriented Dynamic Gesture Recognition and Prediction
title_full_unstemmed	Augmented Reality Assisted Assembly Training Oriented Dynamic Gesture Recognition and Prediction
title_sort	augmented reality assisted assembly training oriented dynamic gesture recognition and prediction
publisher	MDPI AG
publishDate	2021
url	https://doaj.org/article/c69b23c7ea9943d0bd3b6c0ea624b113
work_keys_str_mv	AT jiaqidong augmentedrealityassistedassemblytrainingorienteddynamicgesturerecognitionandprediction AT zeyangxia augmentedrealityassistedassemblytrainingorienteddynamicgesturerecognitionandprediction AT qunfeizhao augmentedrealityassistedassemblytrainingorienteddynamicgesturerecognitionandprediction
_version_	1718437926903218176

Augmented Reality Assisted Assembly Training Oriented Dynamic Gesture Recognition and Prediction

Ejemplares similares