SAT-Hadoop-Processor: A Distributed Remote Sensing Big Data Processing Software for Earth Observation Applications

Nowadays, several environmental applications take advantage of remote sensing techniques. A considerable volume of this remote sensing data occurs in near real-time. Such data are diverse and are provided with high velocity and variety, their pre-processing requires large computing capacities, and a...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	Badr-Eddine Boudriki Semlali, Felix Freitag
Formato:	article
Lenguaje:	EN
Publicado:	MDPI AG 2021
Materias:	remote sensing big data data pre-processing parallel programming cloud computing Technology T Engineering (General). Civil engineering (General) TA1-2040 Biology (General) QH301-705.5 Physics QC1-999 Chemistry QD1-999
Acceso en línea:	https://doaj.org/article/750da33f9ddd47498516136ccc32b45e
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:750da33f9ddd47498516136ccc32b45e
record_format	dspace
spelling	oai:doaj.org-article:750da33f9ddd47498516136ccc32b45e2021-11-25T16:33:18ZSAT-Hadoop-Processor: A Distributed Remote Sensing Big Data Processing Software for Earth Observation Applications10.3390/app1122106102076-3417https://doaj.org/article/750da33f9ddd47498516136ccc32b45e2021-11-01T00:00:00Zhttps://www.mdpi.com/2076-3417/11/22/10610https://doaj.org/toc/2076-3417Nowadays, several environmental applications take advantage of remote sensing techniques. A considerable volume of this remote sensing data occurs in near real-time. Such data are diverse and are provided with high velocity and variety, their pre-processing requires large computing capacities, and a fast execution time is critical. This paper proposes a new distributed software for remote sensing data pre-processing and ingestion using cloud computing technology, specifically OpenStack. The developed software discarded 86% of the unneeded daily files and removed around 20% of the erroneous and inaccurate datasets. The parallel processing optimized the total execution time by 90%. Finally, the software efficiently processed and integrated data into the Hadoop storage system, notably the HDFS, HBase, and Hive.Badr-Eddine Boudriki SemlaliFelix FreitagMDPI AGarticleremote sensing big datadata pre-processingparallel programmingcloud computingTechnologyTEngineering (General). Civil engineering (General)TA1-2040Biology (General)QH301-705.5PhysicsQC1-999ChemistryQD1-999ENApplied Sciences, Vol 11, Iss 10610, p 10610 (2021)
institution	DOAJ
collection	DOAJ
language	EN
topic	remote sensing big data data pre-processing parallel programming cloud computing Technology T Engineering (General). Civil engineering (General) TA1-2040 Biology (General) QH301-705.5 Physics QC1-999 Chemistry QD1-999
spellingShingle	remote sensing big data data pre-processing parallel programming cloud computing Technology T Engineering (General). Civil engineering (General) TA1-2040 Biology (General) QH301-705.5 Physics QC1-999 Chemistry QD1-999 Badr-Eddine Boudriki Semlali Felix Freitag SAT-Hadoop-Processor: A Distributed Remote Sensing Big Data Processing Software for Earth Observation Applications
description	Nowadays, several environmental applications take advantage of remote sensing techniques. A considerable volume of this remote sensing data occurs in near real-time. Such data are diverse and are provided with high velocity and variety, their pre-processing requires large computing capacities, and a fast execution time is critical. This paper proposes a new distributed software for remote sensing data pre-processing and ingestion using cloud computing technology, specifically OpenStack. The developed software discarded 86% of the unneeded daily files and removed around 20% of the erroneous and inaccurate datasets. The parallel processing optimized the total execution time by 90%. Finally, the software efficiently processed and integrated data into the Hadoop storage system, notably the HDFS, HBase, and Hive.
format	article
author	Badr-Eddine Boudriki Semlali Felix Freitag
author_facet	Badr-Eddine Boudriki Semlali Felix Freitag
author_sort	Badr-Eddine Boudriki Semlali
title	SAT-Hadoop-Processor: A Distributed Remote Sensing Big Data Processing Software for Earth Observation Applications
title_short	SAT-Hadoop-Processor: A Distributed Remote Sensing Big Data Processing Software for Earth Observation Applications
title_full	SAT-Hadoop-Processor: A Distributed Remote Sensing Big Data Processing Software for Earth Observation Applications
title_fullStr	SAT-Hadoop-Processor: A Distributed Remote Sensing Big Data Processing Software for Earth Observation Applications
title_full_unstemmed	SAT-Hadoop-Processor: A Distributed Remote Sensing Big Data Processing Software for Earth Observation Applications
title_sort	sat-hadoop-processor: a distributed remote sensing big data processing software for earth observation applications
publisher	MDPI AG
publishDate	2021
url	https://doaj.org/article/750da33f9ddd47498516136ccc32b45e
work_keys_str_mv	AT badreddineboudrikisemlali sathadoopprocessoradistributedremotesensingbigdataprocessingsoftwareforearthobservationapplications AT felixfreitag sathadoopprocessoradistributedremotesensingbigdataprocessingsoftwareforearthobservationapplications
_version_	1718413144044339200

SAT-Hadoop-Processor: A Distributed Remote Sensing Big Data Processing Software for Earth Observation Applications

Ejemplares similares