Large-scale interactive retrieval in art collections using multi-style feature aggregation.

Finding objects and motifs across artworks is of great importance for art history as it helps to understand individual works and analyze relations between them. The advent of digitization has produced extensive digital art collections with many research opportunities. However, manual approaches are...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	Nikolai Ufer, Max Simon, Sabine Lang, Björn Ommer
Formato:	article
Lenguaje:	EN
Publicado:	Public Library of Science (PLoS) 2021
Materias:	Medicine R Science Q
Acceso en línea:	https://doaj.org/article/77678e5009544d57ab91982c87c64a2b
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:77678e5009544d57ab91982c87c64a2b
record_format	dspace
spelling	oai:doaj.org-article:77678e5009544d57ab91982c87c64a2b2021-12-02T20:16:07ZLarge-scale interactive retrieval in art collections using multi-style feature aggregation.1932-620310.1371/journal.pone.0259718https://doaj.org/article/77678e5009544d57ab91982c87c64a2b2021-01-01T00:00:00Zhttps://doi.org/10.1371/journal.pone.0259718https://doaj.org/toc/1932-6203Finding objects and motifs across artworks is of great importance for art history as it helps to understand individual works and analyze relations between them. The advent of digitization has produced extensive digital art collections with many research opportunities. However, manual approaches are inadequate to handle this amount of data, and it requires appropriate computer-based methods to analyze them. This article presents a visual search algorithm and user interface to support art historians to find objects and motifs in extensive datasets. Artistic image collections are subject to significant domain shifts induced by large variations in styles, artistic media, and materials. This poses new challenges to most computer vision models which are trained on photographs. To alleviate this problem, we introduce a multi-style feature aggregation that projects images into the same distribution, leading to more accurate and style-invariant search results. Our retrieval system is based on a voting procedure combined with fast nearest-neighbor search and enables finding and localizing motifs within an extensive image collection in seconds. The presented approach significantly improves the state-of-the-art in terms of accuracy and search time on various datasets and applies to large and inhomogeneous collections. In addition to the search algorithm, we introduce a user interface that allows art historians to apply our algorithm in practice. The interface enables users to search for single regions, multiple regions regarding different connection types and holds an interactive feedback system to improve retrieval results further. With our methodological contribution and easy-to-use user interface, this work manifests further progress towards a computer-based analysis of visual art.Nikolai UferMax SimonSabine LangBjörn OmmerPublic Library of Science (PLoS)articleMedicineRScienceQENPLoS ONE, Vol 16, Iss 11, p e0259718 (2021)
institution	DOAJ
collection	DOAJ
language	EN
topic	Medicine R Science Q
spellingShingle	Medicine R Science Q Nikolai Ufer Max Simon Sabine Lang Björn Ommer Large-scale interactive retrieval in art collections using multi-style feature aggregation.
description	Finding objects and motifs across artworks is of great importance for art history as it helps to understand individual works and analyze relations between them. The advent of digitization has produced extensive digital art collections with many research opportunities. However, manual approaches are inadequate to handle this amount of data, and it requires appropriate computer-based methods to analyze them. This article presents a visual search algorithm and user interface to support art historians to find objects and motifs in extensive datasets. Artistic image collections are subject to significant domain shifts induced by large variations in styles, artistic media, and materials. This poses new challenges to most computer vision models which are trained on photographs. To alleviate this problem, we introduce a multi-style feature aggregation that projects images into the same distribution, leading to more accurate and style-invariant search results. Our retrieval system is based on a voting procedure combined with fast nearest-neighbor search and enables finding and localizing motifs within an extensive image collection in seconds. The presented approach significantly improves the state-of-the-art in terms of accuracy and search time on various datasets and applies to large and inhomogeneous collections. In addition to the search algorithm, we introduce a user interface that allows art historians to apply our algorithm in practice. The interface enables users to search for single regions, multiple regions regarding different connection types and holds an interactive feedback system to improve retrieval results further. With our methodological contribution and easy-to-use user interface, this work manifests further progress towards a computer-based analysis of visual art.
format	article
author	Nikolai Ufer Max Simon Sabine Lang Björn Ommer
author_facet	Nikolai Ufer Max Simon Sabine Lang Björn Ommer
author_sort	Nikolai Ufer
title	Large-scale interactive retrieval in art collections using multi-style feature aggregation.
title_short	Large-scale interactive retrieval in art collections using multi-style feature aggregation.
title_full	Large-scale interactive retrieval in art collections using multi-style feature aggregation.
title_fullStr	Large-scale interactive retrieval in art collections using multi-style feature aggregation.
title_full_unstemmed	Large-scale interactive retrieval in art collections using multi-style feature aggregation.
title_sort	large-scale interactive retrieval in art collections using multi-style feature aggregation.
publisher	Public Library of Science (PLoS)
publishDate	2021
url	https://doaj.org/article/77678e5009544d57ab91982c87c64a2b
work_keys_str_mv	AT nikolaiufer largescaleinteractiveretrievalinartcollectionsusingmultistylefeatureaggregation AT maxsimon largescaleinteractiveretrievalinartcollectionsusingmultistylefeatureaggregation AT sabinelang largescaleinteractiveretrievalinartcollectionsusingmultistylefeatureaggregation AT bjornommer largescaleinteractiveretrievalinartcollectionsusingmultistylefeatureaggregation
_version_	1718374534636109824

Large-scale interactive retrieval in art collections using multi-style feature aggregation.

Ejemplares similares