The assessment of science: the relative merits of post-publication review, the impact factor, and the number of citations.

The assessment of scientific publications is an integral part of the scientific process. Here we investigate three methods of assessing the merit of a scientific paper: subjective post-publication peer review, the number of citations gained by a paper, and the impact factor of the journal in which t...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	Adam Eyre-Walker, Nina Stoletzki
Formato:	article
Lenguaje:	EN
Publicado:	Public Library of Science (PLoS) 2013
Materias:	Biology (General) QH301-705.5
Acceso en línea:	https://doaj.org/article/9a2a9d8a948a417ab9dbd97721f80d3a
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:9a2a9d8a948a417ab9dbd97721f80d3a
record_format	dspace
spelling	oai:doaj.org-article:9a2a9d8a948a417ab9dbd97721f80d3a2021-11-18T05:37:48ZThe assessment of science: the relative merits of post-publication review, the impact factor, and the number of citations.1544-91731545-788510.1371/journal.pbio.1001675https://doaj.org/article/9a2a9d8a948a417ab9dbd97721f80d3a2013-10-01T00:00:00Zhttps://www.ncbi.nlm.nih.gov/pmc/articles/pmid/24115908/?tool=EBIhttps://doaj.org/toc/1544-9173https://doaj.org/toc/1545-7885The assessment of scientific publications is an integral part of the scientific process. Here we investigate three methods of assessing the merit of a scientific paper: subjective post-publication peer review, the number of citations gained by a paper, and the impact factor of the journal in which the article was published. We investigate these methods using two datasets in which subjective post-publication assessments of scientific publications have been made by experts. We find that there are moderate, but statistically significant, correlations between assessor scores, when two assessors have rated the same paper, and between assessor score and the number of citations a paper accrues. However, we show that assessor score depends strongly on the journal in which the paper is published, and that assessors tend to over-rate papers published in journals with high impact factors. If we control for this bias, we find that the correlation between assessor scores and between assessor score and the number of citations is weak, suggesting that scientists have little ability to judge either the intrinsic merit of a paper or its likely impact. We also show that the number of citations a paper receives is an extremely error-prone measure of scientific merit. Finally, we argue that the impact factor is likely to be a poor measure of merit, since it depends on subjective assessment. We conclude that the three measures of scientific merit considered here are poor; in particular subjective assessments are an error-prone, biased, and expensive method by which to assess merit. We argue that the impact factor may be the most satisfactory of the methods we have considered, since it is a form of pre-publication review. However, we emphasise that it is likely to be a very error-prone measure of merit that is qualitative, not quantitative.Adam Eyre-WalkerNina StoletzkiPublic Library of Science (PLoS)articleBiology (General)QH301-705.5ENPLoS Biology, Vol 11, Iss 10, p e1001675 (2013)
institution	DOAJ
collection	DOAJ
language	EN
topic	Biology (General) QH301-705.5
spellingShingle	Biology (General) QH301-705.5 Adam Eyre-Walker Nina Stoletzki The assessment of science: the relative merits of post-publication review, the impact factor, and the number of citations.
description	The assessment of scientific publications is an integral part of the scientific process. Here we investigate three methods of assessing the merit of a scientific paper: subjective post-publication peer review, the number of citations gained by a paper, and the impact factor of the journal in which the article was published. We investigate these methods using two datasets in which subjective post-publication assessments of scientific publications have been made by experts. We find that there are moderate, but statistically significant, correlations between assessor scores, when two assessors have rated the same paper, and between assessor score and the number of citations a paper accrues. However, we show that assessor score depends strongly on the journal in which the paper is published, and that assessors tend to over-rate papers published in journals with high impact factors. If we control for this bias, we find that the correlation between assessor scores and between assessor score and the number of citations is weak, suggesting that scientists have little ability to judge either the intrinsic merit of a paper or its likely impact. We also show that the number of citations a paper receives is an extremely error-prone measure of scientific merit. Finally, we argue that the impact factor is likely to be a poor measure of merit, since it depends on subjective assessment. We conclude that the three measures of scientific merit considered here are poor; in particular subjective assessments are an error-prone, biased, and expensive method by which to assess merit. We argue that the impact factor may be the most satisfactory of the methods we have considered, since it is a form of pre-publication review. However, we emphasise that it is likely to be a very error-prone measure of merit that is qualitative, not quantitative.
format	article
author	Adam Eyre-Walker Nina Stoletzki
author_facet	Adam Eyre-Walker Nina Stoletzki
author_sort	Adam Eyre-Walker
title	The assessment of science: the relative merits of post-publication review, the impact factor, and the number of citations.
title_short	The assessment of science: the relative merits of post-publication review, the impact factor, and the number of citations.
title_full	The assessment of science: the relative merits of post-publication review, the impact factor, and the number of citations.
title_fullStr	The assessment of science: the relative merits of post-publication review, the impact factor, and the number of citations.
title_full_unstemmed	The assessment of science: the relative merits of post-publication review, the impact factor, and the number of citations.
title_sort	assessment of science: the relative merits of post-publication review, the impact factor, and the number of citations.
publisher	Public Library of Science (PLoS)
publishDate	2013
url	https://doaj.org/article/9a2a9d8a948a417ab9dbd97721f80d3a
work_keys_str_mv	AT adameyrewalker theassessmentofsciencetherelativemeritsofpostpublicationreviewtheimpactfactorandthenumberofcitations AT ninastoletzki theassessmentofsciencetherelativemeritsofpostpublicationreviewtheimpactfactorandthenumberofcitations AT adameyrewalker assessmentofsciencetherelativemeritsofpostpublicationreviewtheimpactfactorandthenumberofcitations AT ninastoletzki assessmentofsciencetherelativemeritsofpostpublicationreviewtheimpactfactorandthenumberofcitations
_version_	1718424867019161600

The assessment of science: the relative merits of post-publication review, the impact factor, and the number of citations.

Ejemplares similares