ResMem-Net: memory based deep CNN for image memorability estimation

Image memorability is a very hard problem in image processing due to its subjective nature. But due to the introduction of Deep Learning and the large availability of data and GPUs, great strides have been made in predicting the memorability of an image. In this paper, we propose a novel deep learni...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	Arockia Praveen, Abdulfattah Noorwali, Duraimurugan Samiayya, Mohammad Zubair Khan, Durai Raj Vincent P M, Ali Kashif Bashir, Vinoth Alagupandi
Formato:	article
Lenguaje:	EN
Publicado:	PeerJ Inc. 2021
Materias:	Deep Learning Image Memorability Visual Emotions Saliency Object Interestingness Electronic computers. Computer science QA75.5-76.95
Acceso en línea:	https://doaj.org/article/de7ec13f51074916a588ceccf0033b8e
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:de7ec13f51074916a588ceccf0033b8e
record_format	dspace
spelling	oai:doaj.org-article:de7ec13f51074916a588ceccf0033b8e2021-11-07T15:05:16ZResMem-Net: memory based deep CNN for image memorability estimation10.7717/peerj-cs.7672376-5992https://doaj.org/article/de7ec13f51074916a588ceccf0033b8e2021-11-01T00:00:00Zhttps://peerj.com/articles/cs-767.pdfhttps://peerj.com/articles/cs-767/https://doaj.org/toc/2376-5992Image memorability is a very hard problem in image processing due to its subjective nature. But due to the introduction of Deep Learning and the large availability of data and GPUs, great strides have been made in predicting the memorability of an image. In this paper, we propose a novel deep learning architecture called ResMem-Net that is a hybrid of LSTM and CNN that uses information from the hidden layers of the CNN to compute the memorability score of an image. The intermediate layers are important for predicting the output because they contain information about the intrinsic properties of the image. The proposed architecture automatically learns visual emotions and saliency, shown by the heatmaps generated using the GradRAM technique. We have also used the heatmaps and results to analyze and answer one of the most important questions in image memorability: “What makes an image memorable?”. The model is trained and evaluated using the publicly available Large-scale Image Memorability dataset (LaMem) from MIT. The results show that the model achieves a rank correlation of 0.679 and a mean squared error of 0.011, which is better than the current state-of-the-art models and is close to human consistency (p = 0.68). The proposed architecture also has a significantly low number of parameters compared to the state-of-the-art architecture, making it memory efficient and suitable for production.Arockia PraveenAbdulfattah NoorwaliDuraimurugan SamiayyaMohammad Zubair KhanDurai Raj Vincent P MAli Kashif BashirVinoth AlagupandiPeerJ Inc.articleDeep LearningImage MemorabilityVisual EmotionsSaliencyObject InterestingnessElectronic computers. Computer scienceQA75.5-76.95ENPeerJ Computer Science, Vol 7, p e767 (2021)
institution	DOAJ
collection	DOAJ
language	EN
topic	Deep Learning Image Memorability Visual Emotions Saliency Object Interestingness Electronic computers. Computer science QA75.5-76.95
spellingShingle	Deep Learning Image Memorability Visual Emotions Saliency Object Interestingness Electronic computers. Computer science QA75.5-76.95 Arockia Praveen Abdulfattah Noorwali Duraimurugan Samiayya Mohammad Zubair Khan Durai Raj Vincent P M Ali Kashif Bashir Vinoth Alagupandi ResMem-Net: memory based deep CNN for image memorability estimation
description	Image memorability is a very hard problem in image processing due to its subjective nature. But due to the introduction of Deep Learning and the large availability of data and GPUs, great strides have been made in predicting the memorability of an image. In this paper, we propose a novel deep learning architecture called ResMem-Net that is a hybrid of LSTM and CNN that uses information from the hidden layers of the CNN to compute the memorability score of an image. The intermediate layers are important for predicting the output because they contain information about the intrinsic properties of the image. The proposed architecture automatically learns visual emotions and saliency, shown by the heatmaps generated using the GradRAM technique. We have also used the heatmaps and results to analyze and answer one of the most important questions in image memorability: “What makes an image memorable?”. The model is trained and evaluated using the publicly available Large-scale Image Memorability dataset (LaMem) from MIT. The results show that the model achieves a rank correlation of 0.679 and a mean squared error of 0.011, which is better than the current state-of-the-art models and is close to human consistency (p = 0.68). The proposed architecture also has a significantly low number of parameters compared to the state-of-the-art architecture, making it memory efficient and suitable for production.
format	article
author	Arockia Praveen Abdulfattah Noorwali Duraimurugan Samiayya Mohammad Zubair Khan Durai Raj Vincent P M Ali Kashif Bashir Vinoth Alagupandi
author_facet	Arockia Praveen Abdulfattah Noorwali Duraimurugan Samiayya Mohammad Zubair Khan Durai Raj Vincent P M Ali Kashif Bashir Vinoth Alagupandi
author_sort	Arockia Praveen
title	ResMem-Net: memory based deep CNN for image memorability estimation
title_short	ResMem-Net: memory based deep CNN for image memorability estimation
title_full	ResMem-Net: memory based deep CNN for image memorability estimation
title_fullStr	ResMem-Net: memory based deep CNN for image memorability estimation
title_full_unstemmed	ResMem-Net: memory based deep CNN for image memorability estimation
title_sort	resmem-net: memory based deep cnn for image memorability estimation
publisher	PeerJ Inc.
publishDate	2021
url	https://doaj.org/article/de7ec13f51074916a588ceccf0033b8e
work_keys_str_mv	AT arockiapraveen resmemnetmemorybaseddeepcnnforimagememorabilityestimation AT abdulfattahnoorwali resmemnetmemorybaseddeepcnnforimagememorabilityestimation AT duraimurugansamiayya resmemnetmemorybaseddeepcnnforimagememorabilityestimation AT mohammadzubairkhan resmemnetmemorybaseddeepcnnforimagememorabilityestimation AT durairajvincentpm resmemnetmemorybaseddeepcnnforimagememorabilityestimation AT alikashifbashir resmemnetmemorybaseddeepcnnforimagememorabilityestimation AT vinothalagupandi resmemnetmemorybaseddeepcnnforimagememorabilityestimation
_version_	1718443319301767168

ResMem-Net: memory based deep CNN for image memorability estimation

Ejemplares similares