Continuous Human Activity Recognition through Parallelism LSTM with Multi-Frequency Spectrograms

According to the real-living environment, radar-based human activity recognition (HAR) is dedicated to recognizing and classifying a sequence of activities rather than individual activities, thereby drawing more attention in practical applications of security surveillance, health care and human–comp...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	Congzhang Ding, Yong Jia, Guolong Cui, Chuan Chen, Xiaoling Zhong, Yong Guo
Formato:	article
Lenguaje:	EN
Publicado:	MDPI AG 2021
Materias:	continuous human activity recognition parallelism LSTM SFCW radar multi-frequency spectrogram Science Q
Acceso en línea:	https://doaj.org/article/8cfaa2c44fdd4ec3bf637f91d8ea5d7c
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:8cfaa2c44fdd4ec3bf637f91d8ea5d7c
record_format	dspace
spelling	oai:doaj.org-article:8cfaa2c44fdd4ec3bf637f91d8ea5d7c2021-11-11T18:52:05ZContinuous Human Activity Recognition through Parallelism LSTM with Multi-Frequency Spectrograms10.3390/rs132142642072-4292https://doaj.org/article/8cfaa2c44fdd4ec3bf637f91d8ea5d7c2021-10-01T00:00:00Zhttps://www.mdpi.com/2072-4292/13/21/4264https://doaj.org/toc/2072-4292According to the real-living environment, radar-based human activity recognition (HAR) is dedicated to recognizing and classifying a sequence of activities rather than individual activities, thereby drawing more attention in practical applications of security surveillance, health care and human–computer interactions. This paper proposes a parallelism long short-term memory (LSTM) framework with the input of multi-frequency spectrograms to implement continuous HAR. Specifically, frequency-division short-time Fourier transformation (STFT) is performed on the data stream of continuous activities collected by a stepped-frequency continuous-wave (SFCW) radar, generating spectrograms of multiple frequencies which introduce different scattering properties and frequency resolutions. In the designed parallelism LSTM framework, multiple parallel LSTM sub-networks are trained separately to extract different temporal features from the spectrogram of each frequency and produce corresponding classification probabilities. At the decision level, the probabilities of activity classification from these sub-networks are fused by addition as the recognition output. To validate the proposed method, an experimental data set is collected by using an SFCW radar to monitor 11 participants who continuously perform six activities in sequence with three different transitions and random durations. The validation results demonstrate that the average accuracies of the designed parallelism unidirectional LSTM (Uni-LSTM) and bidirectional LSTM (Bi-LSTM) based on five frequency spectrograms are 85.41% and 96.15%, respectively, outperforming traditional Uni-LSTM and Bi-LSTM networks with only a single-frequency spectrogram by 5.35% and 6.33% at least. Additionally, the recognition accuracy of the parallelism LSTM network reveals an upward trend as the number of multi-frequency spectrograms (namely the number of LSTM subnetworks) increases, and tends to be stable when the number reaches 4.Congzhang DingYong JiaGuolong CuiChuan ChenXiaoling ZhongYong GuoMDPI AGarticlecontinuous human activity recognitionparallelism LSTMSFCW radarmulti-frequency spectrogramScienceQENRemote Sensing, Vol 13, Iss 4264, p 4264 (2021)
institution	DOAJ
collection	DOAJ
language	EN
topic	continuous human activity recognition parallelism LSTM SFCW radar multi-frequency spectrogram Science Q
spellingShingle	continuous human activity recognition parallelism LSTM SFCW radar multi-frequency spectrogram Science Q Congzhang Ding Yong Jia Guolong Cui Chuan Chen Xiaoling Zhong Yong Guo Continuous Human Activity Recognition through Parallelism LSTM with Multi-Frequency Spectrograms
description	According to the real-living environment, radar-based human activity recognition (HAR) is dedicated to recognizing and classifying a sequence of activities rather than individual activities, thereby drawing more attention in practical applications of security surveillance, health care and human–computer interactions. This paper proposes a parallelism long short-term memory (LSTM) framework with the input of multi-frequency spectrograms to implement continuous HAR. Specifically, frequency-division short-time Fourier transformation (STFT) is performed on the data stream of continuous activities collected by a stepped-frequency continuous-wave (SFCW) radar, generating spectrograms of multiple frequencies which introduce different scattering properties and frequency resolutions. In the designed parallelism LSTM framework, multiple parallel LSTM sub-networks are trained separately to extract different temporal features from the spectrogram of each frequency and produce corresponding classification probabilities. At the decision level, the probabilities of activity classification from these sub-networks are fused by addition as the recognition output. To validate the proposed method, an experimental data set is collected by using an SFCW radar to monitor 11 participants who continuously perform six activities in sequence with three different transitions and random durations. The validation results demonstrate that the average accuracies of the designed parallelism unidirectional LSTM (Uni-LSTM) and bidirectional LSTM (Bi-LSTM) based on five frequency spectrograms are 85.41% and 96.15%, respectively, outperforming traditional Uni-LSTM and Bi-LSTM networks with only a single-frequency spectrogram by 5.35% and 6.33% at least. Additionally, the recognition accuracy of the parallelism LSTM network reveals an upward trend as the number of multi-frequency spectrograms (namely the number of LSTM subnetworks) increases, and tends to be stable when the number reaches 4.
format	article
author	Congzhang Ding Yong Jia Guolong Cui Chuan Chen Xiaoling Zhong Yong Guo
author_facet	Congzhang Ding Yong Jia Guolong Cui Chuan Chen Xiaoling Zhong Yong Guo
author_sort	Congzhang Ding
title	Continuous Human Activity Recognition through Parallelism LSTM with Multi-Frequency Spectrograms
title_short	Continuous Human Activity Recognition through Parallelism LSTM with Multi-Frequency Spectrograms
title_full	Continuous Human Activity Recognition through Parallelism LSTM with Multi-Frequency Spectrograms
title_fullStr	Continuous Human Activity Recognition through Parallelism LSTM with Multi-Frequency Spectrograms
title_full_unstemmed	Continuous Human Activity Recognition through Parallelism LSTM with Multi-Frequency Spectrograms
title_sort	continuous human activity recognition through parallelism lstm with multi-frequency spectrograms
publisher	MDPI AG
publishDate	2021
url	https://doaj.org/article/8cfaa2c44fdd4ec3bf637f91d8ea5d7c
work_keys_str_mv	AT congzhangding continuoushumanactivityrecognitionthroughparallelismlstmwithmultifrequencyspectrograms AT yongjia continuoushumanactivityrecognitionthroughparallelismlstmwithmultifrequencyspectrograms AT guolongcui continuoushumanactivityrecognitionthroughparallelismlstmwithmultifrequencyspectrograms AT chuanchen continuoushumanactivityrecognitionthroughparallelismlstmwithmultifrequencyspectrograms AT xiaolingzhong continuoushumanactivityrecognitionthroughparallelismlstmwithmultifrequencyspectrograms AT yongguo continuoushumanactivityrecognitionthroughparallelismlstmwithmultifrequencyspectrograms
_version_	1718431713454981120

Continuous Human Activity Recognition through Parallelism LSTM with Multi-Frequency Spectrograms

Ejemplares similares