Multilabel convolution neural network for facial expression recognition and ordinal intensity estimation

Facial Expression Recognition (FER) has gained considerable attention in affective computing due to its vast area of applications. Diverse approaches and methods have been considered for a robust FER in the field, but only a few works considered the intensity of emotion embedded in the expression. E...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	Olufisayo Ekundayo, Serestina Viriri
Formato:	article
Lenguaje:	EN
Publicado:	PeerJ Inc. 2021
Materias:	Binary cross-entropy Facial expression recognition Island loss Multilabel Ordinal intensity estimation Electronic computers. Computer science QA75.5-76.95
Acceso en línea:	https://doaj.org/article/fc749ed5308345939a17863f58239d1a
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:fc749ed5308345939a17863f58239d1a
record_format	dspace
spelling	oai:doaj.org-article:fc749ed5308345939a17863f58239d1a2021-12-01T15:05:05ZMultilabel convolution neural network for facial expression recognition and ordinal intensity estimation10.7717/peerj-cs.7362376-5992https://doaj.org/article/fc749ed5308345939a17863f58239d1a2021-11-01T00:00:00Zhttps://peerj.com/articles/cs-736.pdfhttps://peerj.com/articles/cs-736/https://doaj.org/toc/2376-5992Facial Expression Recognition (FER) has gained considerable attention in affective computing due to its vast area of applications. Diverse approaches and methods have been considered for a robust FER in the field, but only a few works considered the intensity of emotion embedded in the expression. Even the available studies on expression intensity estimation successfully assigned a nominal/regression value or classified emotion in a range of intervals. Most of the available works on facial expression intensity estimation successfully present only the emotion intensity estimation. At the same time, others proposed methods that predict emotion and its intensity in different channels. These multiclass approaches and extensions do not conform to man heuristic manner of recognising emotion and its intensity estimation. This work presents a Multilabel Convolution Neural Network (ML-CNN)-based model, which could simultaneously recognise emotion and provide ordinal metrics as the intensity estimation of the emotion. The proposed ML-CNN is enhanced with the aggregation of Binary Cross-Entropy (BCE) loss and Island Loss (IL) functions to minimise intraclass and interclass variations. Also, ML-CNN model is pre-trained with Visual Geometric Group (VGG-16) to control overfitting. In the experiments conducted on Binghampton University 3D Facial Expression (BU-3DFE) and Cohn Kanade extension (CK+) datasets, we evaluate ML-CNN’s performance based on accuracy and loss. We also carried out a comparative study of our model with some popularly used multilabel algorithms using standard multilabel metrics. ML-CNN model simultaneously predicts emotion and intensity estimation using ordinal metrics. The model also shows appreciable and superior performance over four standard multilabel algorithms: Chain Classifier (CC), distinct Random K label set (RAKEL), Multilabel K Nearest Neighbour (MLKNN) and Multilabel ARAM (MLARAM).Olufisayo EkundayoSerestina ViririPeerJ Inc.articleBinary cross-entropyFacial expression recognitionIsland lossMultilabelOrdinal intensity estimationElectronic computers. Computer scienceQA75.5-76.95ENPeerJ Computer Science, Vol 7, p e736 (2021)
institution	DOAJ
collection	DOAJ
language	EN
topic	Binary cross-entropy Facial expression recognition Island loss Multilabel Ordinal intensity estimation Electronic computers. Computer science QA75.5-76.95
spellingShingle	Binary cross-entropy Facial expression recognition Island loss Multilabel Ordinal intensity estimation Electronic computers. Computer science QA75.5-76.95 Olufisayo Ekundayo Serestina Viriri Multilabel convolution neural network for facial expression recognition and ordinal intensity estimation
description	Facial Expression Recognition (FER) has gained considerable attention in affective computing due to its vast area of applications. Diverse approaches and methods have been considered for a robust FER in the field, but only a few works considered the intensity of emotion embedded in the expression. Even the available studies on expression intensity estimation successfully assigned a nominal/regression value or classified emotion in a range of intervals. Most of the available works on facial expression intensity estimation successfully present only the emotion intensity estimation. At the same time, others proposed methods that predict emotion and its intensity in different channels. These multiclass approaches and extensions do not conform to man heuristic manner of recognising emotion and its intensity estimation. This work presents a Multilabel Convolution Neural Network (ML-CNN)-based model, which could simultaneously recognise emotion and provide ordinal metrics as the intensity estimation of the emotion. The proposed ML-CNN is enhanced with the aggregation of Binary Cross-Entropy (BCE) loss and Island Loss (IL) functions to minimise intraclass and interclass variations. Also, ML-CNN model is pre-trained with Visual Geometric Group (VGG-16) to control overfitting. In the experiments conducted on Binghampton University 3D Facial Expression (BU-3DFE) and Cohn Kanade extension (CK+) datasets, we evaluate ML-CNN’s performance based on accuracy and loss. We also carried out a comparative study of our model with some popularly used multilabel algorithms using standard multilabel metrics. ML-CNN model simultaneously predicts emotion and intensity estimation using ordinal metrics. The model also shows appreciable and superior performance over four standard multilabel algorithms: Chain Classifier (CC), distinct Random K label set (RAKEL), Multilabel K Nearest Neighbour (MLKNN) and Multilabel ARAM (MLARAM).
format	article
author	Olufisayo Ekundayo Serestina Viriri
author_facet	Olufisayo Ekundayo Serestina Viriri
author_sort	Olufisayo Ekundayo
title	Multilabel convolution neural network for facial expression recognition and ordinal intensity estimation
title_short	Multilabel convolution neural network for facial expression recognition and ordinal intensity estimation
title_full	Multilabel convolution neural network for facial expression recognition and ordinal intensity estimation
title_fullStr	Multilabel convolution neural network for facial expression recognition and ordinal intensity estimation
title_full_unstemmed	Multilabel convolution neural network for facial expression recognition and ordinal intensity estimation
title_sort	multilabel convolution neural network for facial expression recognition and ordinal intensity estimation
publisher	PeerJ Inc.
publishDate	2021
url	https://doaj.org/article/fc749ed5308345939a17863f58239d1a
work_keys_str_mv	AT olufisayoekundayo multilabelconvolutionneuralnetworkforfacialexpressionrecognitionandordinalintensityestimation AT serestinaviriri multilabelconvolutionneuralnetworkforfacialexpressionrecognitionandordinalintensityestimation
_version_	1718404909335838720

Multilabel convolution neural network for facial expression recognition and ordinal intensity estimation

Ejemplares similares