Materials In Paintings (MIP): An interdisciplinary dataset for perception, art history, and computer vision.

In this paper, we capture and explore the painterly depictions of materials to enable the study of depiction and perception of materials through the artists' eye. We annotated a dataset of 19k paintings with 200k+ bounding boxes from which polygon segments were automatically extracted. Each bou...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	Mitchell J P Van Zuijlen, Hubert Lin, Kavita Bala, Sylvia C Pont, Maarten W A Wijntjes
Formato:	article
Lenguaje:	EN
Publicado:	Public Library of Science (PLoS) 2021
Materias:	Medicine R Science Q
Acceso en línea:	https://doaj.org/article/36cc1bab6114437c9192a03ea017f735
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:36cc1bab6114437c9192a03ea017f735
record_format	dspace
spelling	oai:doaj.org-article:36cc1bab6114437c9192a03ea017f7352021-12-02T20:14:54ZMaterials In Paintings (MIP): An interdisciplinary dataset for perception, art history, and computer vision.1932-620310.1371/journal.pone.0255109https://doaj.org/article/36cc1bab6114437c9192a03ea017f7352021-01-01T00:00:00Zhttps://doi.org/10.1371/journal.pone.0255109https://doaj.org/toc/1932-6203In this paper, we capture and explore the painterly depictions of materials to enable the study of depiction and perception of materials through the artists' eye. We annotated a dataset of 19k paintings with 200k+ bounding boxes from which polygon segments were automatically extracted. Each bounding box was assigned a coarse material label (e.g., fabric) and half was also assigned a fine-grained label (e.g., velvety, silky). The dataset in its entirety is available for browsing and downloading at materialsinpaintings.tudelft.nl. We demonstrate the cross-disciplinary utility of our dataset by presenting novel findings across human perception, art history and, computer vision. Our experiments include a demonstration of how painters create convincing depictions using a stylized approach. We further provide an analysis of the spatial and probabilistic distributions of materials depicted in paintings, in which we for example show that strong patterns exists for material presence and location. Furthermore, we demonstrate how paintings could be used to build more robust computer vision classifiers by learning a more perceptually relevant feature representation. Additionally, we demonstrate that training classifiers on paintings could be used to uncover hidden perceptual cues by visualizing the features used by the classifiers. We conclude that our dataset of painterly material depictions is a rich source for gaining insights into the depiction and perception of materials across multiple disciplines and hope that the release of this dataset will drive multidisciplinary research.Mitchell J P Van ZuijlenHubert LinKavita BalaSylvia C PontMaarten W A WijntjesPublic Library of Science (PLoS)articleMedicineRScienceQENPLoS ONE, Vol 16, Iss 8, p e0255109 (2021)
institution	DOAJ
collection	DOAJ
language	EN
topic	Medicine R Science Q
spellingShingle	Medicine R Science Q Mitchell J P Van Zuijlen Hubert Lin Kavita Bala Sylvia C Pont Maarten W A Wijntjes Materials In Paintings (MIP): An interdisciplinary dataset for perception, art history, and computer vision.
description	In this paper, we capture and explore the painterly depictions of materials to enable the study of depiction and perception of materials through the artists' eye. We annotated a dataset of 19k paintings with 200k+ bounding boxes from which polygon segments were automatically extracted. Each bounding box was assigned a coarse material label (e.g., fabric) and half was also assigned a fine-grained label (e.g., velvety, silky). The dataset in its entirety is available for browsing and downloading at materialsinpaintings.tudelft.nl. We demonstrate the cross-disciplinary utility of our dataset by presenting novel findings across human perception, art history and, computer vision. Our experiments include a demonstration of how painters create convincing depictions using a stylized approach. We further provide an analysis of the spatial and probabilistic distributions of materials depicted in paintings, in which we for example show that strong patterns exists for material presence and location. Furthermore, we demonstrate how paintings could be used to build more robust computer vision classifiers by learning a more perceptually relevant feature representation. Additionally, we demonstrate that training classifiers on paintings could be used to uncover hidden perceptual cues by visualizing the features used by the classifiers. We conclude that our dataset of painterly material depictions is a rich source for gaining insights into the depiction and perception of materials across multiple disciplines and hope that the release of this dataset will drive multidisciplinary research.
format	article
author	Mitchell J P Van Zuijlen Hubert Lin Kavita Bala Sylvia C Pont Maarten W A Wijntjes
author_facet	Mitchell J P Van Zuijlen Hubert Lin Kavita Bala Sylvia C Pont Maarten W A Wijntjes
author_sort	Mitchell J P Van Zuijlen
title	Materials In Paintings (MIP): An interdisciplinary dataset for perception, art history, and computer vision.
title_short	Materials In Paintings (MIP): An interdisciplinary dataset for perception, art history, and computer vision.
title_full	Materials In Paintings (MIP): An interdisciplinary dataset for perception, art history, and computer vision.
title_fullStr	Materials In Paintings (MIP): An interdisciplinary dataset for perception, art history, and computer vision.
title_full_unstemmed	Materials In Paintings (MIP): An interdisciplinary dataset for perception, art history, and computer vision.
title_sort	materials in paintings (mip): an interdisciplinary dataset for perception, art history, and computer vision.
publisher	Public Library of Science (PLoS)
publishDate	2021
url	https://doaj.org/article/36cc1bab6114437c9192a03ea017f735
work_keys_str_mv	AT mitchelljpvanzuijlen materialsinpaintingsmipaninterdisciplinarydatasetforperceptionarthistoryandcomputervision AT hubertlin materialsinpaintingsmipaninterdisciplinarydatasetforperceptionarthistoryandcomputervision AT kavitabala materialsinpaintingsmipaninterdisciplinarydatasetforperceptionarthistoryandcomputervision AT sylviacpont materialsinpaintingsmipaninterdisciplinarydatasetforperceptionarthistoryandcomputervision AT maartenwawijntjes materialsinpaintingsmipaninterdisciplinarydatasetforperceptionarthistoryandcomputervision
_version_	1718374637265485824

Materials In Paintings (MIP): An interdisciplinary dataset for perception, art history, and computer vision.

Ejemplares similares