Integrating EfficientNet into an HAFNet Structure for Building Mapping in High-Resolution Optical Earth Observation Data

Automated extraction of buildings from Earth observation (EO) data is important for various applications, including updating of maps, risk assessment, urban planning, and policy-making. Combining data from different sensors, such as high-resolution multispectral images (HRI) and light detection and...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	Luca Ferrari, Fabio Dell’Acqua, Peng Zhang, Peijun Du
Formato:	article
Lenguaje:	EN
Publicado:	MDPI AG 2021
Materias:	attention mechanism building mapping data fusion EfficientNet HAFNet high-resolution imagery (HRI) Science Q
Acceso en línea:	https://doaj.org/article/22c98dc1331f4819a38fcd996dbbbce5
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:22c98dc1331f4819a38fcd996dbbbce5
record_format	dspace
spelling	oai:doaj.org-article:22c98dc1331f4819a38fcd996dbbbce52021-11-11T18:54:41ZIntegrating EfficientNet into an HAFNet Structure for Building Mapping in High-Resolution Optical Earth Observation Data10.3390/rs132143612072-4292https://doaj.org/article/22c98dc1331f4819a38fcd996dbbbce52021-10-01T00:00:00Zhttps://www.mdpi.com/2072-4292/13/21/4361https://doaj.org/toc/2072-4292Automated extraction of buildings from Earth observation (EO) data is important for various applications, including updating of maps, risk assessment, urban planning, and policy-making. Combining data from different sensors, such as high-resolution multispectral images (HRI) and light detection and ranging (LiDAR) data, has shown great potential in building extraction. Deep learning (DL) is increasingly used in multi-modal data fusion and urban object extraction. However, DL-based multi-modal fusion networks may under-perform due to insufficient learning of “joint features” from multiple sources and oversimplified approaches to fusing multi-modal features. Recently, a hybrid attention-aware fusion network (HAFNet) has been proposed for building extraction from a dataset, including co-located Very-High-Resolution (VHR) optical images and light detection and ranging (LiDAR) joint data. The system reported good performances thanks to the adaptivity of the attention mechanism to the features of the information content of the three streams but suffered from model over-parametrization, which inevitably leads to long training times and heavy computational load. In this paper, the authors propose a restructuring of the scheme, which involved replacing VGG-16-like encoders with the recently proposed EfficientNet, whose advantages counteract exactly the issues found with the HAFNet scheme. The novel configuration was tested on multiple benchmark datasets, reporting great improvements in terms of processing times, and also in terms of accuracy. The new scheme, called HAFNetE (HAFNet with EfficientNet integration), appears indeed capable of achieving good results with less parameters, translating into better computational efficiency. Based on these findings, we can conclude that, given the current advancements in single-thread schemes, the classical multi-thread HAFNet scheme could be effectively transformed by the HAFNetE scheme by replacing VGG-16 with EfficientNet blocks on each single thread. The remarkable reduction achieved in computational requirements moves the system one step closer to on-board implementation in a possible, future “urban mapping” satellite constellation.Luca FerrariFabio Dell’AcquaPeng ZhangPeijun DuMDPI AGarticleattention mechanismbuilding mappingdata fusionEfficientNetHAFNethigh-resolution imagery (HRI)ScienceQENRemote Sensing, Vol 13, Iss 4361, p 4361 (2021)
institution	DOAJ
collection	DOAJ
language	EN
topic	attention mechanism building mapping data fusion EfficientNet HAFNet high-resolution imagery (HRI) Science Q
spellingShingle	attention mechanism building mapping data fusion EfficientNet HAFNet high-resolution imagery (HRI) Science Q Luca Ferrari Fabio Dell’Acqua Peng Zhang Peijun Du Integrating EfficientNet into an HAFNet Structure for Building Mapping in High-Resolution Optical Earth Observation Data
description	Automated extraction of buildings from Earth observation (EO) data is important for various applications, including updating of maps, risk assessment, urban planning, and policy-making. Combining data from different sensors, such as high-resolution multispectral images (HRI) and light detection and ranging (LiDAR) data, has shown great potential in building extraction. Deep learning (DL) is increasingly used in multi-modal data fusion and urban object extraction. However, DL-based multi-modal fusion networks may under-perform due to insufficient learning of “joint features” from multiple sources and oversimplified approaches to fusing multi-modal features. Recently, a hybrid attention-aware fusion network (HAFNet) has been proposed for building extraction from a dataset, including co-located Very-High-Resolution (VHR) optical images and light detection and ranging (LiDAR) joint data. The system reported good performances thanks to the adaptivity of the attention mechanism to the features of the information content of the three streams but suffered from model over-parametrization, which inevitably leads to long training times and heavy computational load. In this paper, the authors propose a restructuring of the scheme, which involved replacing VGG-16-like encoders with the recently proposed EfficientNet, whose advantages counteract exactly the issues found with the HAFNet scheme. The novel configuration was tested on multiple benchmark datasets, reporting great improvements in terms of processing times, and also in terms of accuracy. The new scheme, called HAFNetE (HAFNet with EfficientNet integration), appears indeed capable of achieving good results with less parameters, translating into better computational efficiency. Based on these findings, we can conclude that, given the current advancements in single-thread schemes, the classical multi-thread HAFNet scheme could be effectively transformed by the HAFNetE scheme by replacing VGG-16 with EfficientNet blocks on each single thread. The remarkable reduction achieved in computational requirements moves the system one step closer to on-board implementation in a possible, future “urban mapping” satellite constellation.
format	article
author	Luca Ferrari Fabio Dell’Acqua Peng Zhang Peijun Du
author_facet	Luca Ferrari Fabio Dell’Acqua Peng Zhang Peijun Du
author_sort	Luca Ferrari
title	Integrating EfficientNet into an HAFNet Structure for Building Mapping in High-Resolution Optical Earth Observation Data
title_short	Integrating EfficientNet into an HAFNet Structure for Building Mapping in High-Resolution Optical Earth Observation Data
title_full	Integrating EfficientNet into an HAFNet Structure for Building Mapping in High-Resolution Optical Earth Observation Data
title_fullStr	Integrating EfficientNet into an HAFNet Structure for Building Mapping in High-Resolution Optical Earth Observation Data
title_full_unstemmed	Integrating EfficientNet into an HAFNet Structure for Building Mapping in High-Resolution Optical Earth Observation Data
title_sort	integrating efficientnet into an hafnet structure for building mapping in high-resolution optical earth observation data
publisher	MDPI AG
publishDate	2021
url	https://doaj.org/article/22c98dc1331f4819a38fcd996dbbbce5
work_keys_str_mv	AT lucaferrari integratingefficientnetintoanhafnetstructureforbuildingmappinginhighresolutionopticalearthobservationdata AT fabiodellacqua integratingefficientnetintoanhafnetstructureforbuildingmappinginhighresolutionopticalearthobservationdata AT pengzhang integratingefficientnetintoanhafnetstructureforbuildingmappinginhighresolutionopticalearthobservationdata AT peijundu integratingefficientnetintoanhafnetstructureforbuildingmappinginhighresolutionopticalearthobservationdata
_version_	1718431631739453440

Integrating EfficientNet into an HAFNet Structure for Building Mapping in High-Resolution Optical Earth Observation Data

Ejemplares similares