Learning from crowds in digital pathology using scalable variational Gaussian processes

Abstract The volume of labeled data is often the primary determinant of success in developing machine learning algorithms. This has increased interest in methods for leveraging crowds to scale data labeling efforts, and methods to learn from noisy crowd-sourced labels. The need to scale labeling is...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	Miguel López-Pérez, Mohamed Amgad, Pablo Morales-Álvarez, Pablo Ruiz, Lee A. D. Cooper, Rafael Molina, Aggelos K. Katsaggelos
Formato:	article
Lenguaje:	EN
Publicado:	Nature Portfolio 2021
Materias:	Medicine R Science Q
Acceso en línea:	https://doaj.org/article/6f4708f23d7141a982a40bf1745ac009
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:6f4708f23d7141a982a40bf1745ac009
record_format	dspace
spelling	oai:doaj.org-article:6f4708f23d7141a982a40bf1745ac0092021-12-02T18:24:55ZLearning from crowds in digital pathology using scalable variational Gaussian processes10.1038/s41598-021-90821-32045-2322https://doaj.org/article/6f4708f23d7141a982a40bf1745ac0092021-06-01T00:00:00Zhttps://doi.org/10.1038/s41598-021-90821-3https://doaj.org/toc/2045-2322Abstract The volume of labeled data is often the primary determinant of success in developing machine learning algorithms. This has increased interest in methods for leveraging crowds to scale data labeling efforts, and methods to learn from noisy crowd-sourced labels. The need to scale labeling is acute but particularly challenging in medical applications like pathology, due to the expertise required to generate quality labels and the limited availability of qualified experts. In this paper we investigate the application of Scalable Variational Gaussian Processes for Crowdsourcing (SVGPCR) in digital pathology. We compare SVGPCR with other crowdsourcing methods using a large multi-rater dataset where pathologists, pathology residents, and medical students annotated tissue regions breast cancer. Our study shows that SVGPCR is competitive with equivalent methods trained using gold-standard pathologist generated labels, and that SVGPCR meets or exceeds the performance of other crowdsourcing methods based on deep learning. We also show how SVGPCR can effectively learn the class-conditional reliabilities of individual annotators and demonstrate that Gaussian-process classifiers have comparable performance to similar deep learning methods. These results suggest that SVGPCR can meaningfully engage non-experts in pathology labeling tasks, and that the class-conditional reliabilities estimated by SVGPCR may assist in matching annotators to tasks where they perform well.Miguel López-PérezMohamed AmgadPablo Morales-ÁlvarezPablo RuizLee A. D. CooperRafael MolinaAggelos K. KatsaggelosNature PortfolioarticleMedicineRScienceQENScientific Reports, Vol 11, Iss 1, Pp 1-9 (2021)
institution	DOAJ
collection	DOAJ
language	EN
topic	Medicine R Science Q
spellingShingle	Medicine R Science Q Miguel López-Pérez Mohamed Amgad Pablo Morales-Álvarez Pablo Ruiz Lee A. D. Cooper Rafael Molina Aggelos K. Katsaggelos Learning from crowds in digital pathology using scalable variational Gaussian processes
description	Abstract The volume of labeled data is often the primary determinant of success in developing machine learning algorithms. This has increased interest in methods for leveraging crowds to scale data labeling efforts, and methods to learn from noisy crowd-sourced labels. The need to scale labeling is acute but particularly challenging in medical applications like pathology, due to the expertise required to generate quality labels and the limited availability of qualified experts. In this paper we investigate the application of Scalable Variational Gaussian Processes for Crowdsourcing (SVGPCR) in digital pathology. We compare SVGPCR with other crowdsourcing methods using a large multi-rater dataset where pathologists, pathology residents, and medical students annotated tissue regions breast cancer. Our study shows that SVGPCR is competitive with equivalent methods trained using gold-standard pathologist generated labels, and that SVGPCR meets or exceeds the performance of other crowdsourcing methods based on deep learning. We also show how SVGPCR can effectively learn the class-conditional reliabilities of individual annotators and demonstrate that Gaussian-process classifiers have comparable performance to similar deep learning methods. These results suggest that SVGPCR can meaningfully engage non-experts in pathology labeling tasks, and that the class-conditional reliabilities estimated by SVGPCR may assist in matching annotators to tasks where they perform well.
format	article
author	Miguel López-Pérez Mohamed Amgad Pablo Morales-Álvarez Pablo Ruiz Lee A. D. Cooper Rafael Molina Aggelos K. Katsaggelos
author_facet	Miguel López-Pérez Mohamed Amgad Pablo Morales-Álvarez Pablo Ruiz Lee A. D. Cooper Rafael Molina Aggelos K. Katsaggelos
author_sort	Miguel López-Pérez
title	Learning from crowds in digital pathology using scalable variational Gaussian processes
title_short	Learning from crowds in digital pathology using scalable variational Gaussian processes
title_full	Learning from crowds in digital pathology using scalable variational Gaussian processes
title_fullStr	Learning from crowds in digital pathology using scalable variational Gaussian processes
title_full_unstemmed	Learning from crowds in digital pathology using scalable variational Gaussian processes
title_sort	learning from crowds in digital pathology using scalable variational gaussian processes
publisher	Nature Portfolio
publishDate	2021
url	https://doaj.org/article/6f4708f23d7141a982a40bf1745ac009
work_keys_str_mv	AT miguellopezperez learningfromcrowdsindigitalpathologyusingscalablevariationalgaussianprocesses AT mohamedamgad learningfromcrowdsindigitalpathologyusingscalablevariationalgaussianprocesses AT pablomoralesalvarez learningfromcrowdsindigitalpathologyusingscalablevariationalgaussianprocesses AT pabloruiz learningfromcrowdsindigitalpathologyusingscalablevariationalgaussianprocesses AT leeadcooper learningfromcrowdsindigitalpathologyusingscalablevariationalgaussianprocesses AT rafaelmolina learningfromcrowdsindigitalpathologyusingscalablevariationalgaussianprocesses AT aggeloskkatsaggelos learningfromcrowdsindigitalpathologyusingscalablevariationalgaussianprocesses
_version_	1718378102445309952

Learning from crowds in digital pathology using scalable variational Gaussian processes

Ejemplares similares