Task-Aware Dual Prototypical Network for Few-Shot Human-Object Interaction Recognition

Recognizing human-object interaction (HOI) is an important research topic in computer vision. With the great success of deep learning in image classification, the HOI recognition task has also made great progress. However, the problems of instance imbalance and combinatorial explosion still remain t...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autor principal:	AN Ping, JI Zhong, LIU Xiyao
Formato:	article
Lenguaje:	ZH
Publicado:	Journal of Computer Engineering and Applications Beijing Co., Ltd., Science Press 2021
Materias:	computer vision image classification human-object interaction (hoi) few-shot learning (fsl) atten-tion mechanism Electronic computers. Computer science QA75.5-76.95
Acceso en línea:	https://doaj.org/article/d266a5ee01ae4fd09ee09857772bc5c5
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:d266a5ee01ae4fd09ee09857772bc5c5
record_format	dspace
spelling	oai:doaj.org-article:d266a5ee01ae4fd09ee09857772bc5c52021-11-10T08:25:37ZTask-Aware Dual Prototypical Network for Few-Shot Human-Object Interaction Recognition10.3778/j.issn.1673-9418.20080271673-9418https://doaj.org/article/d266a5ee01ae4fd09ee09857772bc5c52021-11-01T00:00:00Zhttp://fcst.ceaj.org/CN/abstract/abstract2957.shtmlhttps://doaj.org/toc/1673-9418Recognizing human-object interaction (HOI) is an important research topic in computer vision. With the great success of deep learning in image classification, the HOI recognition task has also made great progress. However, the problems of instance imbalance and combinatorial explosion still remain the key challenges, which restrict the performance of HOI recognition methods. Therefore, this paper formulates HOI recognition in a few-shot scene to tackle the above problems and proposes a novel task-aware dual prototypical network (TDP-Net) to address few-shot HOI task. Specifically, it first assigns semantic-aware task representations for different tasks as their prior knowledge, subsequently generates attention weights by semantic graph attention module (SGA-Module). It effectively weights the importance on different regions of the visual features, adaptively for different task conditions, which realizes to reason for novel tasks. In addition, it designs a dual prototypes module (DP-Module) to generate both action class prototypes and object class prototypes, which classifies the verb and noun labels respectively. The complex visual relationships between actions and objects can be effectively separated by constructing class prototypes for actions and objects. Meanwhile, owing to the similarity among the related interactions, the knowledge is transferred to the new interactions by reorganizing the action and object prototypes. The experimental results show that the average accuracies of this model outperform the baseline by 3.2 percentage points and 15.7 percentage points on two exper-imental settings, which verifies its effectiveness on the few-shot HOI task.AN Ping, JI Zhong, LIU XiyaoJournal of Computer Engineering and Applications Beijing Co., Ltd., Science Pressarticlecomputer visionimage classificationhuman-object interaction (hoi)few-shot learning (fsl)atten-tion mechanismElectronic computers. Computer scienceQA75.5-76.95ZHJisuanji kexue yu tansuo, Vol 15, Iss 11, Pp 2184-2192 (2021)
institution	DOAJ
collection	DOAJ
language	ZH
topic	computer vision image classification human-object interaction (hoi) few-shot learning (fsl) atten-tion mechanism Electronic computers. Computer science QA75.5-76.95
spellingShingle	computer vision image classification human-object interaction (hoi) few-shot learning (fsl) atten-tion mechanism Electronic computers. Computer science QA75.5-76.95 AN Ping, JI Zhong, LIU Xiyao Task-Aware Dual Prototypical Network for Few-Shot Human-Object Interaction Recognition
description	Recognizing human-object interaction (HOI) is an important research topic in computer vision. With the great success of deep learning in image classification, the HOI recognition task has also made great progress. However, the problems of instance imbalance and combinatorial explosion still remain the key challenges, which restrict the performance of HOI recognition methods. Therefore, this paper formulates HOI recognition in a few-shot scene to tackle the above problems and proposes a novel task-aware dual prototypical network (TDP-Net) to address few-shot HOI task. Specifically, it first assigns semantic-aware task representations for different tasks as their prior knowledge, subsequently generates attention weights by semantic graph attention module (SGA-Module). It effectively weights the importance on different regions of the visual features, adaptively for different task conditions, which realizes to reason for novel tasks. In addition, it designs a dual prototypes module (DP-Module) to generate both action class prototypes and object class prototypes, which classifies the verb and noun labels respectively. The complex visual relationships between actions and objects can be effectively separated by constructing class prototypes for actions and objects. Meanwhile, owing to the similarity among the related interactions, the knowledge is transferred to the new interactions by reorganizing the action and object prototypes. The experimental results show that the average accuracies of this model outperform the baseline by 3.2 percentage points and 15.7 percentage points on two exper-imental settings, which verifies its effectiveness on the few-shot HOI task.
format	article
author	AN Ping, JI Zhong, LIU Xiyao
author_facet	AN Ping, JI Zhong, LIU Xiyao
author_sort	AN Ping, JI Zhong, LIU Xiyao
title	Task-Aware Dual Prototypical Network for Few-Shot Human-Object Interaction Recognition
title_short	Task-Aware Dual Prototypical Network for Few-Shot Human-Object Interaction Recognition
title_full	Task-Aware Dual Prototypical Network for Few-Shot Human-Object Interaction Recognition
title_fullStr	Task-Aware Dual Prototypical Network for Few-Shot Human-Object Interaction Recognition
title_full_unstemmed	Task-Aware Dual Prototypical Network for Few-Shot Human-Object Interaction Recognition
title_sort	task-aware dual prototypical network for few-shot human-object interaction recognition
publisher	Journal of Computer Engineering and Applications Beijing Co., Ltd., Science Press
publishDate	2021
url	https://doaj.org/article/d266a5ee01ae4fd09ee09857772bc5c5
work_keys_str_mv	AT anpingjizhongliuxiyao taskawaredualprototypicalnetworkforfewshothumanobjectinteractionrecognition
_version_	1718440419634708480

Task-Aware Dual Prototypical Network for Few-Shot Human-Object Interaction Recognition

Ejemplares similares