Man versus machine? Self-reports versus algorithmic measurement of publications.

This paper uses newly available data from Web of Science on publications matched to researchers in Survey of Doctorate Recipients to compare the quality of scientific publication data collected by surveys versus algorithmic approaches. We illustrate the different types of measurement errors in self-...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	Xuan Jiang, Wan-Ying Chang, Bruce A Weinberg
Formato:	article
Lenguaje:	EN
Publicado:	Public Library of Science (PLoS) 2021
Materias:	Medicine R Science Q
Acceso en línea:	https://doaj.org/article/d4acbfd806434169bc4c8595485c61c5
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:d4acbfd806434169bc4c8595485c61c5
record_format	dspace
spelling	oai:doaj.org-article:d4acbfd806434169bc4c8595485c61c52021-12-02T20:06:07ZMan versus machine? Self-reports versus algorithmic measurement of publications.1932-620310.1371/journal.pone.0257309https://doaj.org/article/d4acbfd806434169bc4c8595485c61c52021-01-01T00:00:00Zhttps://doi.org/10.1371/journal.pone.0257309https://doaj.org/toc/1932-6203This paper uses newly available data from Web of Science on publications matched to researchers in Survey of Doctorate Recipients to compare the quality of scientific publication data collected by surveys versus algorithmic approaches. We illustrate the different types of measurement errors in self-reported and machine-generated data by estimating how publication measures from the two approaches are related to career outcomes (e.g., salaries and faculty rankings). We find that the potential biases in the self-reports are smaller relative to the algorithmic data. Moreover, the errors in the two approaches are quite intuitive: the measurement errors in algorithmic data are mainly due to the accuracy of matching, which primarily depends on the frequency of names and the data that was available to make matches, while the noise in self reports increases over the career as researchers' publication records become more complex, harder to recall, and less immediately relevant for career progress. At a methodological level, we show how the approaches can be evaluated using accepted statistical methods without gold standard data. We also provide guidance on how to use the new linked data.Xuan JiangWan-Ying ChangBruce A WeinbergPublic Library of Science (PLoS)articleMedicineRScienceQENPLoS ONE, Vol 16, Iss 9, p e0257309 (2021)
institution	DOAJ
collection	DOAJ
language	EN
topic	Medicine R Science Q
spellingShingle	Medicine R Science Q Xuan Jiang Wan-Ying Chang Bruce A Weinberg Man versus machine? Self-reports versus algorithmic measurement of publications.
description	This paper uses newly available data from Web of Science on publications matched to researchers in Survey of Doctorate Recipients to compare the quality of scientific publication data collected by surveys versus algorithmic approaches. We illustrate the different types of measurement errors in self-reported and machine-generated data by estimating how publication measures from the two approaches are related to career outcomes (e.g., salaries and faculty rankings). We find that the potential biases in the self-reports are smaller relative to the algorithmic data. Moreover, the errors in the two approaches are quite intuitive: the measurement errors in algorithmic data are mainly due to the accuracy of matching, which primarily depends on the frequency of names and the data that was available to make matches, while the noise in self reports increases over the career as researchers' publication records become more complex, harder to recall, and less immediately relevant for career progress. At a methodological level, we show how the approaches can be evaluated using accepted statistical methods without gold standard data. We also provide guidance on how to use the new linked data.
format	article
author	Xuan Jiang Wan-Ying Chang Bruce A Weinberg
author_facet	Xuan Jiang Wan-Ying Chang Bruce A Weinberg
author_sort	Xuan Jiang
title	Man versus machine? Self-reports versus algorithmic measurement of publications.
title_short	Man versus machine? Self-reports versus algorithmic measurement of publications.
title_full	Man versus machine? Self-reports versus algorithmic measurement of publications.
title_fullStr	Man versus machine? Self-reports versus algorithmic measurement of publications.
title_full_unstemmed	Man versus machine? Self-reports versus algorithmic measurement of publications.
title_sort	man versus machine? self-reports versus algorithmic measurement of publications.
publisher	Public Library of Science (PLoS)
publishDate	2021
url	https://doaj.org/article/d4acbfd806434169bc4c8595485c61c5
work_keys_str_mv	AT xuanjiang manversusmachineselfreportsversusalgorithmicmeasurementofpublications AT wanyingchang manversusmachineselfreportsversusalgorithmicmeasurementofpublications AT bruceaweinberg manversusmachineselfreportsversusalgorithmicmeasurementofpublications
_version_	1718375453240066048

Man versus machine? Self-reports versus algorithmic measurement of publications.

Ejemplares similares