The validation of peer review through research impact measures and the implications for funding strategies.

There is a paucity of data in the literature concerning the validation of the grant application peer review process, which is used to help direct billions of dollars in research funds. Ultimately, this validation will hinge upon empirical data relating the output of funded projects to the prediction...

Descripción completa

Guardado en:
Detalles Bibliográficos
Autores principales: Stephen A Gallo, Afton S Carpenter, David Irwin, Caitlin D McPartland, Joseph Travis, Sofie Reynders, Lisa A Thompson, Scott R Glisson
Formato: article
Lenguaje:EN
Publicado: Public Library of Science (PLoS) 2014
Materias:
R
Q
Acceso en línea:https://doaj.org/article/193683a1e52e45c9ab5bc3d71216d006
Etiquetas: Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
id oai:doaj.org-article:193683a1e52e45c9ab5bc3d71216d006
record_format dspace
spelling oai:doaj.org-article:193683a1e52e45c9ab5bc3d71216d0062021-11-25T06:02:03ZThe validation of peer review through research impact measures and the implications for funding strategies.1932-620310.1371/journal.pone.0106474https://doaj.org/article/193683a1e52e45c9ab5bc3d71216d0062014-01-01T00:00:00Zhttps://www.ncbi.nlm.nih.gov/pmc/articles/pmid/25184367/?tool=EBIhttps://doaj.org/toc/1932-6203There is a paucity of data in the literature concerning the validation of the grant application peer review process, which is used to help direct billions of dollars in research funds. Ultimately, this validation will hinge upon empirical data relating the output of funded projects to the predictions implicit in the overall scientific merit scores from the peer review of submitted applications. In an effort to address this need, the American Institute of Biological Sciences (AIBS) conducted a retrospective analysis of peer review data of 2,063 applications submitted to a particular research program and the bibliometric output of the resultant 227 funded projects over an 8-year period. Peer review scores associated with applications were found to be moderately correlated with the total time-adjusted citation output of funded projects, although a high degree of variability existed in the data. Analysis over time revealed that as average annual scores of all applications (both funded and unfunded) submitted to this program improved with time, the average annual citation output per application increased. Citation impact did not correlate with the amount of funds awarded per application or with the total annual programmatic budget. However, the number of funded applications per year was found to correlate well with total annual citation impact, suggesting that improving funding success rates by reducing the size of awards may be an efficient strategy to optimize the scientific impact of research program portfolios. This strategy must be weighed against the need for a balanced research portfolio and the inherent high costs of some areas of research. The relationship observed between peer review scores and bibliometric output lays the groundwork for establishing a model system for future prospective testing of the validity of peer review formats and procedures.Stephen A GalloAfton S CarpenterDavid IrwinCaitlin D McPartlandJoseph TravisSofie ReyndersLisa A ThompsonScott R GlissonPublic Library of Science (PLoS)articleMedicineRScienceQENPLoS ONE, Vol 9, Iss 9, p e106474 (2014)
institution DOAJ
collection DOAJ
language EN
topic Medicine
R
Science
Q
spellingShingle Medicine
R
Science
Q
Stephen A Gallo
Afton S Carpenter
David Irwin
Caitlin D McPartland
Joseph Travis
Sofie Reynders
Lisa A Thompson
Scott R Glisson
The validation of peer review through research impact measures and the implications for funding strategies.
description There is a paucity of data in the literature concerning the validation of the grant application peer review process, which is used to help direct billions of dollars in research funds. Ultimately, this validation will hinge upon empirical data relating the output of funded projects to the predictions implicit in the overall scientific merit scores from the peer review of submitted applications. In an effort to address this need, the American Institute of Biological Sciences (AIBS) conducted a retrospective analysis of peer review data of 2,063 applications submitted to a particular research program and the bibliometric output of the resultant 227 funded projects over an 8-year period. Peer review scores associated with applications were found to be moderately correlated with the total time-adjusted citation output of funded projects, although a high degree of variability existed in the data. Analysis over time revealed that as average annual scores of all applications (both funded and unfunded) submitted to this program improved with time, the average annual citation output per application increased. Citation impact did not correlate with the amount of funds awarded per application or with the total annual programmatic budget. However, the number of funded applications per year was found to correlate well with total annual citation impact, suggesting that improving funding success rates by reducing the size of awards may be an efficient strategy to optimize the scientific impact of research program portfolios. This strategy must be weighed against the need for a balanced research portfolio and the inherent high costs of some areas of research. The relationship observed between peer review scores and bibliometric output lays the groundwork for establishing a model system for future prospective testing of the validity of peer review formats and procedures.
format article
author Stephen A Gallo
Afton S Carpenter
David Irwin
Caitlin D McPartland
Joseph Travis
Sofie Reynders
Lisa A Thompson
Scott R Glisson
author_facet Stephen A Gallo
Afton S Carpenter
David Irwin
Caitlin D McPartland
Joseph Travis
Sofie Reynders
Lisa A Thompson
Scott R Glisson
author_sort Stephen A Gallo
title The validation of peer review through research impact measures and the implications for funding strategies.
title_short The validation of peer review through research impact measures and the implications for funding strategies.
title_full The validation of peer review through research impact measures and the implications for funding strategies.
title_fullStr The validation of peer review through research impact measures and the implications for funding strategies.
title_full_unstemmed The validation of peer review through research impact measures and the implications for funding strategies.
title_sort validation of peer review through research impact measures and the implications for funding strategies.
publisher Public Library of Science (PLoS)
publishDate 2014
url https://doaj.org/article/193683a1e52e45c9ab5bc3d71216d006
work_keys_str_mv AT stephenagallo thevalidationofpeerreviewthroughresearchimpactmeasuresandtheimplicationsforfundingstrategies
AT aftonscarpenter thevalidationofpeerreviewthroughresearchimpactmeasuresandtheimplicationsforfundingstrategies
AT davidirwin thevalidationofpeerreviewthroughresearchimpactmeasuresandtheimplicationsforfundingstrategies
AT caitlindmcpartland thevalidationofpeerreviewthroughresearchimpactmeasuresandtheimplicationsforfundingstrategies
AT josephtravis thevalidationofpeerreviewthroughresearchimpactmeasuresandtheimplicationsforfundingstrategies
AT sofiereynders thevalidationofpeerreviewthroughresearchimpactmeasuresandtheimplicationsforfundingstrategies
AT lisaathompson thevalidationofpeerreviewthroughresearchimpactmeasuresandtheimplicationsforfundingstrategies
AT scottrglisson thevalidationofpeerreviewthroughresearchimpactmeasuresandtheimplicationsforfundingstrategies
AT stephenagallo validationofpeerreviewthroughresearchimpactmeasuresandtheimplicationsforfundingstrategies
AT aftonscarpenter validationofpeerreviewthroughresearchimpactmeasuresandtheimplicationsforfundingstrategies
AT davidirwin validationofpeerreviewthroughresearchimpactmeasuresandtheimplicationsforfundingstrategies
AT caitlindmcpartland validationofpeerreviewthroughresearchimpactmeasuresandtheimplicationsforfundingstrategies
AT josephtravis validationofpeerreviewthroughresearchimpactmeasuresandtheimplicationsforfundingstrategies
AT sofiereynders validationofpeerreviewthroughresearchimpactmeasuresandtheimplicationsforfundingstrategies
AT lisaathompson validationofpeerreviewthroughresearchimpactmeasuresandtheimplicationsforfundingstrategies
AT scottrglisson validationofpeerreviewthroughresearchimpactmeasuresandtheimplicationsforfundingstrategies
_version_ 1718414265038143488