Evaluation of excess significance bias in animal studies of neurological diseases.

Animal studies generate valuable hypotheses that lead to the conduct of preventive or therapeutic clinical trials. We assessed whether there is evidence for excess statistical significance in results of animal studies on neurological disorders, suggesting biases. We used data from meta-analyses of i...

Descripción completa

Guardado en:
Detalles Bibliográficos
Autores principales: Konstantinos K Tsilidis, Orestis A Panagiotou, Emily S Sena, Eleni Aretouli, Evangelos Evangelou, David W Howells, Rustam Al-Shahi Salman, Malcolm R Macleod, John P A Ioannidis
Formato: article
Lenguaje:EN
Publicado: Public Library of Science (PLoS) 2013
Materias:
Acceso en línea:https://doaj.org/article/bca865a5a5cf43c1bf8387fb018d8c16
Etiquetas: Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
id oai:doaj.org-article:bca865a5a5cf43c1bf8387fb018d8c16
record_format dspace
spelling oai:doaj.org-article:bca865a5a5cf43c1bf8387fb018d8c162021-11-18T05:37:00ZEvaluation of excess significance bias in animal studies of neurological diseases.1544-91731545-788510.1371/journal.pbio.1001609https://doaj.org/article/bca865a5a5cf43c1bf8387fb018d8c162013-07-01T00:00:00Zhttps://www.ncbi.nlm.nih.gov/pmc/articles/pmid/23874156/pdf/?tool=EBIhttps://doaj.org/toc/1544-9173https://doaj.org/toc/1545-7885Animal studies generate valuable hypotheses that lead to the conduct of preventive or therapeutic clinical trials. We assessed whether there is evidence for excess statistical significance in results of animal studies on neurological disorders, suggesting biases. We used data from meta-analyses of interventions deposited in Collaborative Approach to Meta-Analysis and Review of Animal Data in Experimental Studies (CAMARADES). The number of observed studies with statistically significant results (O) was compared with the expected number (E), based on the statistical power of each study under different assumptions for the plausible effect size. We assessed 4,445 datasets synthesized in 160 meta-analyses on Alzheimer disease (n = 2), experimental autoimmune encephalomyelitis (n = 34), focal ischemia (n = 16), intracerebral hemorrhage (n = 61), Parkinson disease (n = 45), and spinal cord injury (n = 2). 112 meta-analyses (70%) found nominally (p≤0.05) statistically significant summary fixed effects. Assuming the effect size in the most precise study to be a plausible effect, 919 out of 4,445 nominally significant results were expected versus 1,719 observed (p<10⁻⁹). Excess significance was present across all neurological disorders, in all subgroups defined by methodological characteristics, and also according to alternative plausible effects. Asymmetry tests also showed evidence of small-study effects in 74 (46%) meta-analyses. Significantly effective interventions with more than 500 animals, and no hints of bias were seen in eight (5%) meta-analyses. Overall, there are too many animal studies with statistically significant results in the literature of neurological disorders. This observation suggests strong biases, with selective analysis and outcome reporting biases being plausible explanations, and provides novel evidence on how these biases might influence the whole research domain of neurological animal literature.Konstantinos K TsilidisOrestis A PanagiotouEmily S SenaEleni AretouliEvangelos EvangelouDavid W HowellsRustam Al-Shahi SalmanMalcolm R MacleodJohn P A IoannidisPublic Library of Science (PLoS)articleBiology (General)QH301-705.5ENPLoS Biology, Vol 11, Iss 7, p e1001609 (2013)
institution DOAJ
collection DOAJ
language EN
topic Biology (General)
QH301-705.5
spellingShingle Biology (General)
QH301-705.5
Konstantinos K Tsilidis
Orestis A Panagiotou
Emily S Sena
Eleni Aretouli
Evangelos Evangelou
David W Howells
Rustam Al-Shahi Salman
Malcolm R Macleod
John P A Ioannidis
Evaluation of excess significance bias in animal studies of neurological diseases.
description Animal studies generate valuable hypotheses that lead to the conduct of preventive or therapeutic clinical trials. We assessed whether there is evidence for excess statistical significance in results of animal studies on neurological disorders, suggesting biases. We used data from meta-analyses of interventions deposited in Collaborative Approach to Meta-Analysis and Review of Animal Data in Experimental Studies (CAMARADES). The number of observed studies with statistically significant results (O) was compared with the expected number (E), based on the statistical power of each study under different assumptions for the plausible effect size. We assessed 4,445 datasets synthesized in 160 meta-analyses on Alzheimer disease (n = 2), experimental autoimmune encephalomyelitis (n = 34), focal ischemia (n = 16), intracerebral hemorrhage (n = 61), Parkinson disease (n = 45), and spinal cord injury (n = 2). 112 meta-analyses (70%) found nominally (p≤0.05) statistically significant summary fixed effects. Assuming the effect size in the most precise study to be a plausible effect, 919 out of 4,445 nominally significant results were expected versus 1,719 observed (p<10⁻⁹). Excess significance was present across all neurological disorders, in all subgroups defined by methodological characteristics, and also according to alternative plausible effects. Asymmetry tests also showed evidence of small-study effects in 74 (46%) meta-analyses. Significantly effective interventions with more than 500 animals, and no hints of bias were seen in eight (5%) meta-analyses. Overall, there are too many animal studies with statistically significant results in the literature of neurological disorders. This observation suggests strong biases, with selective analysis and outcome reporting biases being plausible explanations, and provides novel evidence on how these biases might influence the whole research domain of neurological animal literature.
format article
author Konstantinos K Tsilidis
Orestis A Panagiotou
Emily S Sena
Eleni Aretouli
Evangelos Evangelou
David W Howells
Rustam Al-Shahi Salman
Malcolm R Macleod
John P A Ioannidis
author_facet Konstantinos K Tsilidis
Orestis A Panagiotou
Emily S Sena
Eleni Aretouli
Evangelos Evangelou
David W Howells
Rustam Al-Shahi Salman
Malcolm R Macleod
John P A Ioannidis
author_sort Konstantinos K Tsilidis
title Evaluation of excess significance bias in animal studies of neurological diseases.
title_short Evaluation of excess significance bias in animal studies of neurological diseases.
title_full Evaluation of excess significance bias in animal studies of neurological diseases.
title_fullStr Evaluation of excess significance bias in animal studies of neurological diseases.
title_full_unstemmed Evaluation of excess significance bias in animal studies of neurological diseases.
title_sort evaluation of excess significance bias in animal studies of neurological diseases.
publisher Public Library of Science (PLoS)
publishDate 2013
url https://doaj.org/article/bca865a5a5cf43c1bf8387fb018d8c16
work_keys_str_mv AT konstantinosktsilidis evaluationofexcesssignificancebiasinanimalstudiesofneurologicaldiseases
AT orestisapanagiotou evaluationofexcesssignificancebiasinanimalstudiesofneurologicaldiseases
AT emilyssena evaluationofexcesssignificancebiasinanimalstudiesofneurologicaldiseases
AT eleniaretouli evaluationofexcesssignificancebiasinanimalstudiesofneurologicaldiseases
AT evangelosevangelou evaluationofexcesssignificancebiasinanimalstudiesofneurologicaldiseases
AT davidwhowells evaluationofexcesssignificancebiasinanimalstudiesofneurologicaldiseases
AT rustamalshahisalman evaluationofexcesssignificancebiasinanimalstudiesofneurologicaldiseases
AT malcolmrmacleod evaluationofexcesssignificancebiasinanimalstudiesofneurologicaldiseases
AT johnpaioannidis evaluationofexcesssignificancebiasinanimalstudiesofneurologicaldiseases
_version_ 1718424876391333888