Attentive 3D-Ghost Module for Dynamic Hand Gesture Recognition with Positive Knowledge Transfer

Hand gesture recognition is a challenging topic in the field of computer vision. Multimodal hand gesture recognition based on RGB-D is with higher accuracy than that of only RGB or depth. It is not difficult to conclude that the gain originates from the complementary information existing in the two...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	Jinghua Li, Runze Liu, Dehui Kong, Shaofan Wang, Lichun Wang, Baocai Yin, Ronghua Gao
Formato:	article
Lenguaje:	EN
Publicado:	Hindawi Limited 2021
Materias:	Computer applications to medicine. Medical informatics R858-859.7 Neurosciences. Biological psychiatry. Neuropsychiatry RC321-571
Acceso en línea:	https://doaj.org/article/244f8aefa1654aa6b6d090756b502e42
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:244f8aefa1654aa6b6d090756b502e42
record_format	dspace
spelling	oai:doaj.org-article:244f8aefa1654aa6b6d090756b502e422021-11-29T00:56:59ZAttentive 3D-Ghost Module for Dynamic Hand Gesture Recognition with Positive Knowledge Transfer1687-527310.1155/2021/5044916https://doaj.org/article/244f8aefa1654aa6b6d090756b502e422021-01-01T00:00:00Zhttp://dx.doi.org/10.1155/2021/5044916https://doaj.org/toc/1687-5273Hand gesture recognition is a challenging topic in the field of computer vision. Multimodal hand gesture recognition based on RGB-D is with higher accuracy than that of only RGB or depth. It is not difficult to conclude that the gain originates from the complementary information existing in the two modalities. However, in reality, multimodal data are not always easy to acquire simultaneously, while unimodal RGB or depth hand gesture data are more general. Therefore, one hand gesture system is expected, in which only unimordal RGB or Depth data is supported for testing, while multimodal RGB-D data is available for training so as to attain the complementary information. Fortunately, a kind of method via multimodal training and unimodal testing has been proposed. However, unimodal feature representation and cross-modality transfer still need to be further improved. To this end, this paper proposes a new 3D-Ghost and Spatial Attention Inflated 3D ConvNet (3DGSAI) to extract high-quality features for each modality. The baseline of 3DGSAI network is Inflated 3D ConvNet (I3D), and two main improvements are proposed. One is 3D-Ghost module, and the other is the spatial attention mechanism. The 3D-Ghost module can extract richer features for hand gesture representation, and the spatial attention mechanism makes the network pay more attention to hand region. This paper also proposes an adaptive parameter for positive knowledge transfer, which ensures that the transfer always occurs from the strong modality network to the weak one. Extensive experiments on SKIG, VIVA, and NVGesture datasets demonstrate that our method is competitive with the state of the art. Especially, the performance of our method reaches 97.87% on the SKIG dataset using only RGB, which is the current best result.Jinghua LiRunze LiuDehui KongShaofan WangLichun WangBaocai YinRonghua GaoHindawi LimitedarticleComputer applications to medicine. Medical informaticsR858-859.7Neurosciences. Biological psychiatry. NeuropsychiatryRC321-571ENComputational Intelligence and Neuroscience, Vol 2021 (2021)
institution	DOAJ
collection	DOAJ
language	EN
topic	Computer applications to medicine. Medical informatics R858-859.7 Neurosciences. Biological psychiatry. Neuropsychiatry RC321-571
spellingShingle	Computer applications to medicine. Medical informatics R858-859.7 Neurosciences. Biological psychiatry. Neuropsychiatry RC321-571 Jinghua Li Runze Liu Dehui Kong Shaofan Wang Lichun Wang Baocai Yin Ronghua Gao Attentive 3D-Ghost Module for Dynamic Hand Gesture Recognition with Positive Knowledge Transfer
description	Hand gesture recognition is a challenging topic in the field of computer vision. Multimodal hand gesture recognition based on RGB-D is with higher accuracy than that of only RGB or depth. It is not difficult to conclude that the gain originates from the complementary information existing in the two modalities. However, in reality, multimodal data are not always easy to acquire simultaneously, while unimodal RGB or depth hand gesture data are more general. Therefore, one hand gesture system is expected, in which only unimordal RGB or Depth data is supported for testing, while multimodal RGB-D data is available for training so as to attain the complementary information. Fortunately, a kind of method via multimodal training and unimodal testing has been proposed. However, unimodal feature representation and cross-modality transfer still need to be further improved. To this end, this paper proposes a new 3D-Ghost and Spatial Attention Inflated 3D ConvNet (3DGSAI) to extract high-quality features for each modality. The baseline of 3DGSAI network is Inflated 3D ConvNet (I3D), and two main improvements are proposed. One is 3D-Ghost module, and the other is the spatial attention mechanism. The 3D-Ghost module can extract richer features for hand gesture representation, and the spatial attention mechanism makes the network pay more attention to hand region. This paper also proposes an adaptive parameter for positive knowledge transfer, which ensures that the transfer always occurs from the strong modality network to the weak one. Extensive experiments on SKIG, VIVA, and NVGesture datasets demonstrate that our method is competitive with the state of the art. Especially, the performance of our method reaches 97.87% on the SKIG dataset using only RGB, which is the current best result.
format	article
author	Jinghua Li Runze Liu Dehui Kong Shaofan Wang Lichun Wang Baocai Yin Ronghua Gao
author_facet	Jinghua Li Runze Liu Dehui Kong Shaofan Wang Lichun Wang Baocai Yin Ronghua Gao
author_sort	Jinghua Li
title	Attentive 3D-Ghost Module for Dynamic Hand Gesture Recognition with Positive Knowledge Transfer
title_short	Attentive 3D-Ghost Module for Dynamic Hand Gesture Recognition with Positive Knowledge Transfer
title_full	Attentive 3D-Ghost Module for Dynamic Hand Gesture Recognition with Positive Knowledge Transfer
title_fullStr	Attentive 3D-Ghost Module for Dynamic Hand Gesture Recognition with Positive Knowledge Transfer
title_full_unstemmed	Attentive 3D-Ghost Module for Dynamic Hand Gesture Recognition with Positive Knowledge Transfer
title_sort	attentive 3d-ghost module for dynamic hand gesture recognition with positive knowledge transfer
publisher	Hindawi Limited
publishDate	2021
url	https://doaj.org/article/244f8aefa1654aa6b6d090756b502e42
work_keys_str_mv	AT jinghuali attentive3dghostmodulefordynamichandgesturerecognitionwithpositiveknowledgetransfer AT runzeliu attentive3dghostmodulefordynamichandgesturerecognitionwithpositiveknowledgetransfer AT dehuikong attentive3dghostmodulefordynamichandgesturerecognitionwithpositiveknowledgetransfer AT shaofanwang attentive3dghostmodulefordynamichandgesturerecognitionwithpositiveknowledgetransfer AT lichunwang attentive3dghostmodulefordynamichandgesturerecognitionwithpositiveknowledgetransfer AT baocaiyin attentive3dghostmodulefordynamichandgesturerecognitionwithpositiveknowledgetransfer AT ronghuagao attentive3dghostmodulefordynamichandgesturerecognitionwithpositiveknowledgetransfer
_version_	1718407633763827712

Attentive 3D-Ghost Module for Dynamic Hand Gesture Recognition with Positive Knowledge Transfer

Ejemplares similares