Attention_FPNet: Two-Branch Remote Sensing Image Pansharpening Network Based on Attention Feature Fusion

Inspired by the impressive achievements of convolutional neural networks in various computer vision tasks and the effective role of attention mechanisms, this article proposes a two-branch fusion network based on attention feature fusion (AFF) called Attention_FPNet to solve the pansharpening proble...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	Xiwu Zhong, Yurong Qian, Hui Liu, Long Chen, Yaling Wan, Liang Gao, Jing Qian, Jun Liu
Formato:	article
Lenguaje:	EN
Publicado:	IEEE 2021
Materias:	Attention feature fusion (AFF) convolutional neural network (CNN) image fusion pansharpening remote sensing Ocean engineering TC1501-1800 Geophysics. Cosmic physics QC801-809
Acceso en línea:	https://doaj.org/article/480514be8fcb4040a37dbe611661879a
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:480514be8fcb4040a37dbe611661879a
record_format	dspace
spelling	oai:doaj.org-article:480514be8fcb4040a37dbe611661879a2021-12-04T00:00:07ZAttention_FPNet: Two-Branch Remote Sensing Image Pansharpening Network Based on Attention Feature Fusion2151-153510.1109/JSTARS.2021.3126645https://doaj.org/article/480514be8fcb4040a37dbe611661879a2021-01-01T00:00:00Zhttps://ieeexplore.ieee.org/document/9609690/https://doaj.org/toc/2151-1535Inspired by the impressive achievements of convolutional neural networks in various computer vision tasks and the effective role of attention mechanisms, this article proposes a two-branch fusion network based on attention feature fusion (AFF) called Attention_FPNet to solve the pansharpening problem. We reconstruct the spatial information of the image in the high-pass filter domain and fully consider the spatial information in the multispectral (MS) and panchromatic (PAN) images. At the same time, the input PAN image and the upsampled MS image are directly transmitted to the reconstructed image through a long skip connection. The spectral information of the PAN and MS images is considered to improve the spectral resolution of the fused image. It also supplements the loss of spatial information that may be caused by network deepening. Moreover, an AFF method is used to replace the existing simple channel concatenation method commonly used in pansharpening, which fully considers the relationship between different feature maps and improves the fusion quality. Through experiments on image datasets acquired by the Pleiades, SPOT-6 and Gaofen-2 satellites, the results show that this method can effectively fuse PAN and MS images and generate a fused image and outperforms existing methods.Xiwu ZhongYurong QianHui LiuLong ChenYaling WanLiang GaoJing QianJun LiuIEEEarticleAttention feature fusion (AFF)convolutional neural network (CNN)image fusionpansharpeningremote sensingOcean engineeringTC1501-1800Geophysics. Cosmic physicsQC801-809ENIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, Vol 14, Pp 11879-11891 (2021)
institution	DOAJ
collection	DOAJ
language	EN
topic	Attention feature fusion (AFF) convolutional neural network (CNN) image fusion pansharpening remote sensing Ocean engineering TC1501-1800 Geophysics. Cosmic physics QC801-809
spellingShingle	Attention feature fusion (AFF) convolutional neural network (CNN) image fusion pansharpening remote sensing Ocean engineering TC1501-1800 Geophysics. Cosmic physics QC801-809 Xiwu Zhong Yurong Qian Hui Liu Long Chen Yaling Wan Liang Gao Jing Qian Jun Liu Attention_FPNet: Two-Branch Remote Sensing Image Pansharpening Network Based on Attention Feature Fusion
description	Inspired by the impressive achievements of convolutional neural networks in various computer vision tasks and the effective role of attention mechanisms, this article proposes a two-branch fusion network based on attention feature fusion (AFF) called Attention_FPNet to solve the pansharpening problem. We reconstruct the spatial information of the image in the high-pass filter domain and fully consider the spatial information in the multispectral (MS) and panchromatic (PAN) images. At the same time, the input PAN image and the upsampled MS image are directly transmitted to the reconstructed image through a long skip connection. The spectral information of the PAN and MS images is considered to improve the spectral resolution of the fused image. It also supplements the loss of spatial information that may be caused by network deepening. Moreover, an AFF method is used to replace the existing simple channel concatenation method commonly used in pansharpening, which fully considers the relationship between different feature maps and improves the fusion quality. Through experiments on image datasets acquired by the Pleiades, SPOT-6 and Gaofen-2 satellites, the results show that this method can effectively fuse PAN and MS images and generate a fused image and outperforms existing methods.
format	article
author	Xiwu Zhong Yurong Qian Hui Liu Long Chen Yaling Wan Liang Gao Jing Qian Jun Liu
author_facet	Xiwu Zhong Yurong Qian Hui Liu Long Chen Yaling Wan Liang Gao Jing Qian Jun Liu
author_sort	Xiwu Zhong
title	Attention_FPNet: Two-Branch Remote Sensing Image Pansharpening Network Based on Attention Feature Fusion
title_short	Attention_FPNet: Two-Branch Remote Sensing Image Pansharpening Network Based on Attention Feature Fusion
title_full	Attention_FPNet: Two-Branch Remote Sensing Image Pansharpening Network Based on Attention Feature Fusion
title_fullStr	Attention_FPNet: Two-Branch Remote Sensing Image Pansharpening Network Based on Attention Feature Fusion
title_full_unstemmed	Attention_FPNet: Two-Branch Remote Sensing Image Pansharpening Network Based on Attention Feature Fusion
title_sort	attention_fpnet: two-branch remote sensing image pansharpening network based on attention feature fusion
publisher	IEEE
publishDate	2021
url	https://doaj.org/article/480514be8fcb4040a37dbe611661879a
work_keys_str_mv	AT xiwuzhong attentionfpnettwobranchremotesensingimagepansharpeningnetworkbasedonattentionfeaturefusion AT yurongqian attentionfpnettwobranchremotesensingimagepansharpeningnetworkbasedonattentionfeaturefusion AT huiliu attentionfpnettwobranchremotesensingimagepansharpeningnetworkbasedonattentionfeaturefusion AT longchen attentionfpnettwobranchremotesensingimagepansharpeningnetworkbasedonattentionfeaturefusion AT yalingwan attentionfpnettwobranchremotesensingimagepansharpeningnetworkbasedonattentionfeaturefusion AT lianggao attentionfpnettwobranchremotesensingimagepansharpeningnetworkbasedonattentionfeaturefusion AT jingqian attentionfpnettwobranchremotesensingimagepansharpeningnetworkbasedonattentionfeaturefusion AT junliu attentionfpnettwobranchremotesensingimagepansharpeningnetworkbasedonattentionfeaturefusion
_version_	1718373027209543680

Attention_FPNet: Two-Branch Remote Sensing Image Pansharpening Network Based on Attention Feature Fusion

Ejemplares similares