Design considerations for workflow management systems use in production genomics research and the clinic

Abstract The changing landscape of genomics research and clinical practice has created a need for computational pipelines capable of efficiently orchestrating complex analysis stages while handling large volumes of data across heterogeneous computational environments. Workflow Management Systems (Wf...

Descripción completa

Guardado en:
Detalles Bibliográficos
Autores principales: Azza E. Ahmed, Joshua M. Allen, Tajesvi Bhat, Prakruthi Burra, Christina E. Fliege, Steven N. Hart, Jacob R. Heldenbrand, Matthew E. Hudson, Dave Deandre Istanto, Michael T. Kalmbach, Gregory D. Kapraun, Katherine I. Kendig, Matthew Charles Kendzior, Eric W. Klee, Nate Mattson, Christian A. Ross, Sami M. Sharif, Ramshankar Venkatakrishnan, Faisal M. Fadlelmola, Liudmila S. Mainzer
Formato: article
Lenguaje:EN
Publicado: Nature Portfolio 2021
Materias:
R
Q
Acceso en línea:https://doaj.org/article/71063f0cca8645c6a434d845100e2c90
Etiquetas: Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
id oai:doaj.org-article:71063f0cca8645c6a434d845100e2c90
record_format dspace
spelling oai:doaj.org-article:71063f0cca8645c6a434d845100e2c902021-11-08T10:55:45ZDesign considerations for workflow management systems use in production genomics research and the clinic10.1038/s41598-021-99288-82045-2322https://doaj.org/article/71063f0cca8645c6a434d845100e2c902021-11-01T00:00:00Zhttps://doi.org/10.1038/s41598-021-99288-8https://doaj.org/toc/2045-2322Abstract The changing landscape of genomics research and clinical practice has created a need for computational pipelines capable of efficiently orchestrating complex analysis stages while handling large volumes of data across heterogeneous computational environments. Workflow Management Systems (WfMSs) are the software components employed to fill this gap. This work provides an approach and systematic evaluation of key features of popular bioinformatics WfMSs in use today: Nextflow, CWL, and WDL and some of their executors, along with Swift/T, a workflow manager commonly used in high-scale physics applications. We employed two use cases: a variant-calling genomic pipeline and a scalability-testing framework, where both were run locally, on an HPC cluster, and in the cloud. This allowed for evaluation of those four WfMSs in terms of language expressiveness, modularity, scalability, robustness, reproducibility, interoperability, ease of development, along with adoption and usage in research labs and healthcare settings. This article is trying to answer, which WfMS should be chosen for a given bioinformatics application regardless of analysis type?. The choice of a given WfMS is a function of both its intrinsic language and engine features. Within bioinformatics, where analysts are a mix of dry and wet lab scientists, the choice is also governed by collaborations and adoption within large consortia and technical support provided by the WfMS team/community. As the community and its needs continue to evolve along with computational infrastructure, WfMSs will also evolve, especially those with permissive licenses that allow commercial use. In much the same way as the dataflow paradigm and containerization are now well understood to be very useful in bioinformatics applications, we will continue to see innovations of tools and utilities for other purposes, like big data technologies, interoperability, and provenance.Azza E. AhmedJoshua M. AllenTajesvi BhatPrakruthi BurraChristina E. FliegeSteven N. HartJacob R. HeldenbrandMatthew E. HudsonDave Deandre IstantoMichael T. KalmbachGregory D. KapraunKatherine I. KendigMatthew Charles KendziorEric W. KleeNate MattsonChristian A. RossSami M. SharifRamshankar VenkatakrishnanFaisal M. FadlelmolaLiudmila S. MainzerNature PortfolioarticleMedicineRScienceQENScientific Reports, Vol 11, Iss 1, Pp 1-18 (2021)
institution DOAJ
collection DOAJ
language EN
topic Medicine
R
Science
Q
spellingShingle Medicine
R
Science
Q
Azza E. Ahmed
Joshua M. Allen
Tajesvi Bhat
Prakruthi Burra
Christina E. Fliege
Steven N. Hart
Jacob R. Heldenbrand
Matthew E. Hudson
Dave Deandre Istanto
Michael T. Kalmbach
Gregory D. Kapraun
Katherine I. Kendig
Matthew Charles Kendzior
Eric W. Klee
Nate Mattson
Christian A. Ross
Sami M. Sharif
Ramshankar Venkatakrishnan
Faisal M. Fadlelmola
Liudmila S. Mainzer
Design considerations for workflow management systems use in production genomics research and the clinic
description Abstract The changing landscape of genomics research and clinical practice has created a need for computational pipelines capable of efficiently orchestrating complex analysis stages while handling large volumes of data across heterogeneous computational environments. Workflow Management Systems (WfMSs) are the software components employed to fill this gap. This work provides an approach and systematic evaluation of key features of popular bioinformatics WfMSs in use today: Nextflow, CWL, and WDL and some of their executors, along with Swift/T, a workflow manager commonly used in high-scale physics applications. We employed two use cases: a variant-calling genomic pipeline and a scalability-testing framework, where both were run locally, on an HPC cluster, and in the cloud. This allowed for evaluation of those four WfMSs in terms of language expressiveness, modularity, scalability, robustness, reproducibility, interoperability, ease of development, along with adoption and usage in research labs and healthcare settings. This article is trying to answer, which WfMS should be chosen for a given bioinformatics application regardless of analysis type?. The choice of a given WfMS is a function of both its intrinsic language and engine features. Within bioinformatics, where analysts are a mix of dry and wet lab scientists, the choice is also governed by collaborations and adoption within large consortia and technical support provided by the WfMS team/community. As the community and its needs continue to evolve along with computational infrastructure, WfMSs will also evolve, especially those with permissive licenses that allow commercial use. In much the same way as the dataflow paradigm and containerization are now well understood to be very useful in bioinformatics applications, we will continue to see innovations of tools and utilities for other purposes, like big data technologies, interoperability, and provenance.
format article
author Azza E. Ahmed
Joshua M. Allen
Tajesvi Bhat
Prakruthi Burra
Christina E. Fliege
Steven N. Hart
Jacob R. Heldenbrand
Matthew E. Hudson
Dave Deandre Istanto
Michael T. Kalmbach
Gregory D. Kapraun
Katherine I. Kendig
Matthew Charles Kendzior
Eric W. Klee
Nate Mattson
Christian A. Ross
Sami M. Sharif
Ramshankar Venkatakrishnan
Faisal M. Fadlelmola
Liudmila S. Mainzer
author_facet Azza E. Ahmed
Joshua M. Allen
Tajesvi Bhat
Prakruthi Burra
Christina E. Fliege
Steven N. Hart
Jacob R. Heldenbrand
Matthew E. Hudson
Dave Deandre Istanto
Michael T. Kalmbach
Gregory D. Kapraun
Katherine I. Kendig
Matthew Charles Kendzior
Eric W. Klee
Nate Mattson
Christian A. Ross
Sami M. Sharif
Ramshankar Venkatakrishnan
Faisal M. Fadlelmola
Liudmila S. Mainzer
author_sort Azza E. Ahmed
title Design considerations for workflow management systems use in production genomics research and the clinic
title_short Design considerations for workflow management systems use in production genomics research and the clinic
title_full Design considerations for workflow management systems use in production genomics research and the clinic
title_fullStr Design considerations for workflow management systems use in production genomics research and the clinic
title_full_unstemmed Design considerations for workflow management systems use in production genomics research and the clinic
title_sort design considerations for workflow management systems use in production genomics research and the clinic
publisher Nature Portfolio
publishDate 2021
url https://doaj.org/article/71063f0cca8645c6a434d845100e2c90
work_keys_str_mv AT azzaeahmed designconsiderationsforworkflowmanagementsystemsuseinproductiongenomicsresearchandtheclinic
AT joshuamallen designconsiderationsforworkflowmanagementsystemsuseinproductiongenomicsresearchandtheclinic
AT tajesvibhat designconsiderationsforworkflowmanagementsystemsuseinproductiongenomicsresearchandtheclinic
AT prakruthiburra designconsiderationsforworkflowmanagementsystemsuseinproductiongenomicsresearchandtheclinic
AT christinaefliege designconsiderationsforworkflowmanagementsystemsuseinproductiongenomicsresearchandtheclinic
AT stevennhart designconsiderationsforworkflowmanagementsystemsuseinproductiongenomicsresearchandtheclinic
AT jacobrheldenbrand designconsiderationsforworkflowmanagementsystemsuseinproductiongenomicsresearchandtheclinic
AT matthewehudson designconsiderationsforworkflowmanagementsystemsuseinproductiongenomicsresearchandtheclinic
AT davedeandreistanto designconsiderationsforworkflowmanagementsystemsuseinproductiongenomicsresearchandtheclinic
AT michaeltkalmbach designconsiderationsforworkflowmanagementsystemsuseinproductiongenomicsresearchandtheclinic
AT gregorydkapraun designconsiderationsforworkflowmanagementsystemsuseinproductiongenomicsresearchandtheclinic
AT katherineikendig designconsiderationsforworkflowmanagementsystemsuseinproductiongenomicsresearchandtheclinic
AT matthewcharleskendzior designconsiderationsforworkflowmanagementsystemsuseinproductiongenomicsresearchandtheclinic
AT ericwklee designconsiderationsforworkflowmanagementsystemsuseinproductiongenomicsresearchandtheclinic
AT natemattson designconsiderationsforworkflowmanagementsystemsuseinproductiongenomicsresearchandtheclinic
AT christianaross designconsiderationsforworkflowmanagementsystemsuseinproductiongenomicsresearchandtheclinic
AT samimsharif designconsiderationsforworkflowmanagementsystemsuseinproductiongenomicsresearchandtheclinic
AT ramshankarvenkatakrishnan designconsiderationsforworkflowmanagementsystemsuseinproductiongenomicsresearchandtheclinic
AT faisalmfadlelmola designconsiderationsforworkflowmanagementsystemsuseinproductiongenomicsresearchandtheclinic
AT liudmilasmainzer designconsiderationsforworkflowmanagementsystemsuseinproductiongenomicsresearchandtheclinic
_version_ 1718442536365719552