Quantifying reproducibility in computational biology: the case of the tuberculosis drugome.

How easy is it to reproduce the results found in a typical computational biology paper? Either through experience or intuition the reader will already know that the answer is with difficulty or not at all. In this paper we attempt to quantify this difficulty by reproducing a previously published pap...

Descripción completa

Guardado en:
Detalles Bibliográficos
Autores principales: Daniel Garijo, Sarah Kinnings, Li Xie, Lei Xie, Yinliang Zhang, Philip E Bourne, Yolanda Gil
Formato: article
Lenguaje:EN
Publicado: Public Library of Science (PLoS) 2013
Materias:
R
Q
Acceso en línea:https://doaj.org/article/454d1ddf94184285835e1293a52f3b80
Etiquetas: Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
id oai:doaj.org-article:454d1ddf94184285835e1293a52f3b80
record_format dspace
spelling oai:doaj.org-article:454d1ddf94184285835e1293a52f3b802021-11-18T08:44:27ZQuantifying reproducibility in computational biology: the case of the tuberculosis drugome.1932-620310.1371/journal.pone.0080278https://doaj.org/article/454d1ddf94184285835e1293a52f3b802013-01-01T00:00:00Zhttps://www.ncbi.nlm.nih.gov/pmc/articles/pmid/24312207/pdf/?tool=EBIhttps://doaj.org/toc/1932-6203How easy is it to reproduce the results found in a typical computational biology paper? Either through experience or intuition the reader will already know that the answer is with difficulty or not at all. In this paper we attempt to quantify this difficulty by reproducing a previously published paper for different classes of users (ranging from users with little expertise to domain experts) and suggest ways in which the situation might be improved. Quantification is achieved by estimating the time required to reproduce each of the steps in the method described in the original paper and make them part of an explicit workflow that reproduces the original results. Reproducing the method took several months of effort, and required using new versions and new software that posed challenges to reconstructing and validating the results. The quantification leads to "reproducibility maps" that reveal that novice researchers would only be able to reproduce a few of the steps in the method, and that only expert researchers with advance knowledge of the domain would be able to reproduce the method in its entirety. The workflow itself is published as an online resource together with supporting software and data. The paper concludes with a brief discussion of the complexities of requiring reproducibility in terms of cost versus benefit, and a desiderata with our observations and guidelines for improving reproducibility. This has implications not only in reproducing the work of others from published papers, but reproducing work from one's own laboratory.Daniel GarijoSarah KinningsLi XieLei XieYinliang ZhangPhilip E BourneYolanda GilPublic Library of Science (PLoS)articleMedicineRScienceQENPLoS ONE, Vol 8, Iss 11, p e80278 (2013)
institution DOAJ
collection DOAJ
language EN
topic Medicine
R
Science
Q
spellingShingle Medicine
R
Science
Q
Daniel Garijo
Sarah Kinnings
Li Xie
Lei Xie
Yinliang Zhang
Philip E Bourne
Yolanda Gil
Quantifying reproducibility in computational biology: the case of the tuberculosis drugome.
description How easy is it to reproduce the results found in a typical computational biology paper? Either through experience or intuition the reader will already know that the answer is with difficulty or not at all. In this paper we attempt to quantify this difficulty by reproducing a previously published paper for different classes of users (ranging from users with little expertise to domain experts) and suggest ways in which the situation might be improved. Quantification is achieved by estimating the time required to reproduce each of the steps in the method described in the original paper and make them part of an explicit workflow that reproduces the original results. Reproducing the method took several months of effort, and required using new versions and new software that posed challenges to reconstructing and validating the results. The quantification leads to "reproducibility maps" that reveal that novice researchers would only be able to reproduce a few of the steps in the method, and that only expert researchers with advance knowledge of the domain would be able to reproduce the method in its entirety. The workflow itself is published as an online resource together with supporting software and data. The paper concludes with a brief discussion of the complexities of requiring reproducibility in terms of cost versus benefit, and a desiderata with our observations and guidelines for improving reproducibility. This has implications not only in reproducing the work of others from published papers, but reproducing work from one's own laboratory.
format article
author Daniel Garijo
Sarah Kinnings
Li Xie
Lei Xie
Yinliang Zhang
Philip E Bourne
Yolanda Gil
author_facet Daniel Garijo
Sarah Kinnings
Li Xie
Lei Xie
Yinliang Zhang
Philip E Bourne
Yolanda Gil
author_sort Daniel Garijo
title Quantifying reproducibility in computational biology: the case of the tuberculosis drugome.
title_short Quantifying reproducibility in computational biology: the case of the tuberculosis drugome.
title_full Quantifying reproducibility in computational biology: the case of the tuberculosis drugome.
title_fullStr Quantifying reproducibility in computational biology: the case of the tuberculosis drugome.
title_full_unstemmed Quantifying reproducibility in computational biology: the case of the tuberculosis drugome.
title_sort quantifying reproducibility in computational biology: the case of the tuberculosis drugome.
publisher Public Library of Science (PLoS)
publishDate 2013
url https://doaj.org/article/454d1ddf94184285835e1293a52f3b80
work_keys_str_mv AT danielgarijo quantifyingreproducibilityincomputationalbiologythecaseofthetuberculosisdrugome
AT sarahkinnings quantifyingreproducibilityincomputationalbiologythecaseofthetuberculosisdrugome
AT lixie quantifyingreproducibilityincomputationalbiologythecaseofthetuberculosisdrugome
AT leixie quantifyingreproducibilityincomputationalbiologythecaseofthetuberculosisdrugome
AT yinliangzhang quantifyingreproducibilityincomputationalbiologythecaseofthetuberculosisdrugome
AT philipebourne quantifyingreproducibilityincomputationalbiologythecaseofthetuberculosisdrugome
AT yolandagil quantifyingreproducibilityincomputationalbiologythecaseofthetuberculosisdrugome
_version_ 1718421379696558080