Fully Automated Pose Estimation of Historical Images in the Context of 4D Geographic Information Systems Utilizing Machine Learning Methods

The idea of virtual time machines in digital environments like hand-held virtual reality or four-dimensional (4D) geographic information systems requires an accurate positioning and orientation of urban historical images. The browsing of large repositories to retrieve historical images and their sub...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	Ferdinand Maiwald, Christoph Lehmann, Taras Lazariv
Formato:	article
Lenguaje:	EN
Publicado:	MDPI AG 2021
Materias:	historical images pose estimation photogrammetry 4D-GIS cultural heritage automation Geography (General) G1-922
Acceso en línea:	https://doaj.org/article/de0b8f1249964a02a5e6497ba39f7ab6
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:de0b8f1249964a02a5e6497ba39f7ab6
record_format	dspace
spelling	oai:doaj.org-article:de0b8f1249964a02a5e6497ba39f7ab62021-11-25T17:52:57ZFully Automated Pose Estimation of Historical Images in the Context of 4D Geographic Information Systems Utilizing Machine Learning Methods10.3390/ijgi101107482220-9964https://doaj.org/article/de0b8f1249964a02a5e6497ba39f7ab62021-11-01T00:00:00Zhttps://www.mdpi.com/2220-9964/10/11/748https://doaj.org/toc/2220-9964The idea of virtual time machines in digital environments like hand-held virtual reality or four-dimensional (4D) geographic information systems requires an accurate positioning and orientation of urban historical images. The browsing of large repositories to retrieve historical images and their subsequent precise pose estimation is still a manual and time-consuming process in the field of Cultural Heritage. This contribution presents an end-to-end pipeline from finding relevant images with utilization of content-based image retrieval to photogrammetric pose estimation of large historical terrestrial image datasets. Image retrieval as well as pose estimation are challenging tasks and are subjects of current research. Thereby, research has a strong focus on contemporary images but the methods are not considered for a use on historical image material. The first part of the pipeline comprises the precise selection of many relevant historical images based on a few example images (so called query images) by using content-based image retrieval. Therefore, two different retrieval approaches based on convolutional neural networks (CNN) are tested, evaluated, and compared with conventional metadata search in repositories. Results show that image retrieval approaches outperform the metadata search and are a valuable strategy for finding images of interest. The second part of the pipeline uses techniques of photogrammetry to derive the camera position and orientation of the historical images identified by the image retrieval. Multiple feature matching methods are used on four different datasets, the scene is reconstructed in the Structure-from-Motion software COLMAP, and all experiments are evaluated on a newly generated historical benchmark dataset. A large number of oriented images, as well as low error measures for most of the datasets, show that the workflow can be successfully applied. Finally, the combination of a CNN-based image retrieval and the feature matching methods SuperGlue and DISK show very promising results to realize a fully automated workflow. Such an automated workflow of selection and pose estimation of historical terrestrial images enables the creation of large-scale 4D models.Ferdinand MaiwaldChristoph LehmannTaras LazarivMDPI AGarticlehistorical imagespose estimationphotogrammetry4D-GIScultural heritageautomationGeography (General)G1-922ENISPRS International Journal of Geo-Information, Vol 10, Iss 748, p 748 (2021)
institution	DOAJ
collection	DOAJ
language	EN
topic	historical images pose estimation photogrammetry 4D-GIS cultural heritage automation Geography (General) G1-922
spellingShingle	historical images pose estimation photogrammetry 4D-GIS cultural heritage automation Geography (General) G1-922 Ferdinand Maiwald Christoph Lehmann Taras Lazariv Fully Automated Pose Estimation of Historical Images in the Context of 4D Geographic Information Systems Utilizing Machine Learning Methods
description	The idea of virtual time machines in digital environments like hand-held virtual reality or four-dimensional (4D) geographic information systems requires an accurate positioning and orientation of urban historical images. The browsing of large repositories to retrieve historical images and their subsequent precise pose estimation is still a manual and time-consuming process in the field of Cultural Heritage. This contribution presents an end-to-end pipeline from finding relevant images with utilization of content-based image retrieval to photogrammetric pose estimation of large historical terrestrial image datasets. Image retrieval as well as pose estimation are challenging tasks and are subjects of current research. Thereby, research has a strong focus on contemporary images but the methods are not considered for a use on historical image material. The first part of the pipeline comprises the precise selection of many relevant historical images based on a few example images (so called query images) by using content-based image retrieval. Therefore, two different retrieval approaches based on convolutional neural networks (CNN) are tested, evaluated, and compared with conventional metadata search in repositories. Results show that image retrieval approaches outperform the metadata search and are a valuable strategy for finding images of interest. The second part of the pipeline uses techniques of photogrammetry to derive the camera position and orientation of the historical images identified by the image retrieval. Multiple feature matching methods are used on four different datasets, the scene is reconstructed in the Structure-from-Motion software COLMAP, and all experiments are evaluated on a newly generated historical benchmark dataset. A large number of oriented images, as well as low error measures for most of the datasets, show that the workflow can be successfully applied. Finally, the combination of a CNN-based image retrieval and the feature matching methods SuperGlue and DISK show very promising results to realize a fully automated workflow. Such an automated workflow of selection and pose estimation of historical terrestrial images enables the creation of large-scale 4D models.
format	article
author	Ferdinand Maiwald Christoph Lehmann Taras Lazariv
author_facet	Ferdinand Maiwald Christoph Lehmann Taras Lazariv
author_sort	Ferdinand Maiwald
title	Fully Automated Pose Estimation of Historical Images in the Context of 4D Geographic Information Systems Utilizing Machine Learning Methods
title_short	Fully Automated Pose Estimation of Historical Images in the Context of 4D Geographic Information Systems Utilizing Machine Learning Methods
title_full	Fully Automated Pose Estimation of Historical Images in the Context of 4D Geographic Information Systems Utilizing Machine Learning Methods
title_fullStr	Fully Automated Pose Estimation of Historical Images in the Context of 4D Geographic Information Systems Utilizing Machine Learning Methods
title_full_unstemmed	Fully Automated Pose Estimation of Historical Images in the Context of 4D Geographic Information Systems Utilizing Machine Learning Methods
title_sort	fully automated pose estimation of historical images in the context of 4d geographic information systems utilizing machine learning methods
publisher	MDPI AG
publishDate	2021
url	https://doaj.org/article/de0b8f1249964a02a5e6497ba39f7ab6
work_keys_str_mv	AT ferdinandmaiwald fullyautomatedposeestimationofhistoricalimagesinthecontextof4dgeographicinformationsystemsutilizingmachinelearningmethods AT christophlehmann fullyautomatedposeestimationofhistoricalimagesinthecontextof4dgeographicinformationsystemsutilizingmachinelearningmethods AT taraslazariv fullyautomatedposeestimationofhistoricalimagesinthecontextof4dgeographicinformationsystemsutilizingmachinelearningmethods
_version_	1718411892987265024

Fully Automated Pose Estimation of Historical Images in the Context of 4D Geographic Information Systems Utilizing Machine Learning Methods

Ejemplares similares