A Framework for Effective Application of Machine Learning to Microbiome-Based Classification Problems

ABSTRACT Machine learning (ML) modeling of the human microbiome has the potential to identify microbial biomarkers and aid in the diagnosis of many diseases such as inflammatory bowel disease, diabetes, and colorectal cancer. Progress has been made toward developing ML models that predict health out...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	Begüm D. Topçuoğlu, Nicholas A. Lesniak, Mack T. Ruffin, Jenna Wiens, Patrick D. Schloss
Formato:	article
Lenguaje:	EN
Publicado:	American Society for Microbiology 2020
Materias:	16S rRNA gene colon cancer machine learning microbial ecology microbiome Microbiology QR1-502
Acceso en línea:	https://doaj.org/article/47b3a3d71e7741b48a1bbd42dbb2aeb9
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:47b3a3d71e7741b48a1bbd42dbb2aeb9
record_format	dspace
spelling	oai:doaj.org-article:47b3a3d71e7741b48a1bbd42dbb2aeb92021-11-15T15:56:47ZA Framework for Effective Application of Machine Learning to Microbiome-Based Classification Problems10.1128/mBio.00434-202150-7511https://doaj.org/article/47b3a3d71e7741b48a1bbd42dbb2aeb92020-06-01T00:00:00Zhttps://journals.asm.org/doi/10.1128/mBio.00434-20https://doaj.org/toc/2150-7511ABSTRACT Machine learning (ML) modeling of the human microbiome has the potential to identify microbial biomarkers and aid in the diagnosis of many diseases such as inflammatory bowel disease, diabetes, and colorectal cancer. Progress has been made toward developing ML models that predict health outcomes using bacterial abundances, but inconsistent adoption of training and evaluation methods call the validity of these models into question. Furthermore, there appears to be a preference by many researchers to favor increased model complexity over interpretability. To overcome these challenges, we trained seven models that used fecal 16S rRNA sequence data to predict the presence of colonic screen relevant neoplasias (SRNs) (n = 490 patients, 261 controls and 229 cases). We developed a reusable open-source pipeline to train, validate, and interpret ML models. To show the effect of model selection, we assessed the predictive performance, interpretability, and training time of L2-regularized logistic regression, L1- and L2-regularized support vector machines (SVM) with linear and radial basis function kernels, a decision tree, random forest, and gradient boosted trees (XGBoost). The random forest model performed best at detecting SRNs with an area under the receiver operating characteristic curve (AUROC) of 0.695 (interquartile range [IQR], 0.651 to 0.739) but was slow to train (83.2 h) and not inherently interpretable. Despite its simplicity, L2-regularized logistic regression followed random forest in predictive performance with an AUROC of 0.680 (IQR, 0.625 to 0.735), trained faster (12 min), and was inherently interpretable. Our analysis highlights the importance of choosing an ML approach based on the goal of the study, as the choice will inform expectations of performance and interpretability. IMPORTANCE Diagnosing diseases using machine learning (ML) is rapidly being adopted in microbiome studies. However, the estimated performance associated with these models is likely overoptimistic. Moreover, there is a trend toward using black box models without a discussion of the difficulty of interpreting such models when trying to identify microbial biomarkers of disease. This work represents a step toward developing more-reproducible ML practices in applying ML to microbiome research. We implement a rigorous pipeline and emphasize the importance of selecting ML models that reflect the goal of the study. These concepts are not particular to the study of human health but can also be applied to environmental microbiology studies.Begüm D. TopçuoğluNicholas A. LesniakMack T. RuffinJenna WiensPatrick D. SchlossAmerican Society for Microbiologyarticle16S rRNA genecolon cancermachine learningmicrobial ecologymicrobiomeMicrobiologyQR1-502ENmBio, Vol 11, Iss 3 (2020)
institution	DOAJ
collection	DOAJ
language	EN
topic	16S rRNA gene colon cancer machine learning microbial ecology microbiome Microbiology QR1-502
spellingShingle	16S rRNA gene colon cancer machine learning microbial ecology microbiome Microbiology QR1-502 Begüm D. Topçuoğlu Nicholas A. Lesniak Mack T. Ruffin Jenna Wiens Patrick D. Schloss A Framework for Effective Application of Machine Learning to Microbiome-Based Classification Problems
description	ABSTRACT Machine learning (ML) modeling of the human microbiome has the potential to identify microbial biomarkers and aid in the diagnosis of many diseases such as inflammatory bowel disease, diabetes, and colorectal cancer. Progress has been made toward developing ML models that predict health outcomes using bacterial abundances, but inconsistent adoption of training and evaluation methods call the validity of these models into question. Furthermore, there appears to be a preference by many researchers to favor increased model complexity over interpretability. To overcome these challenges, we trained seven models that used fecal 16S rRNA sequence data to predict the presence of colonic screen relevant neoplasias (SRNs) (n = 490 patients, 261 controls and 229 cases). We developed a reusable open-source pipeline to train, validate, and interpret ML models. To show the effect of model selection, we assessed the predictive performance, interpretability, and training time of L2-regularized logistic regression, L1- and L2-regularized support vector machines (SVM) with linear and radial basis function kernels, a decision tree, random forest, and gradient boosted trees (XGBoost). The random forest model performed best at detecting SRNs with an area under the receiver operating characteristic curve (AUROC) of 0.695 (interquartile range [IQR], 0.651 to 0.739) but was slow to train (83.2 h) and not inherently interpretable. Despite its simplicity, L2-regularized logistic regression followed random forest in predictive performance with an AUROC of 0.680 (IQR, 0.625 to 0.735), trained faster (12 min), and was inherently interpretable. Our analysis highlights the importance of choosing an ML approach based on the goal of the study, as the choice will inform expectations of performance and interpretability. IMPORTANCE Diagnosing diseases using machine learning (ML) is rapidly being adopted in microbiome studies. However, the estimated performance associated with these models is likely overoptimistic. Moreover, there is a trend toward using black box models without a discussion of the difficulty of interpreting such models when trying to identify microbial biomarkers of disease. This work represents a step toward developing more-reproducible ML practices in applying ML to microbiome research. We implement a rigorous pipeline and emphasize the importance of selecting ML models that reflect the goal of the study. These concepts are not particular to the study of human health but can also be applied to environmental microbiology studies.
format	article
author	Begüm D. Topçuoğlu Nicholas A. Lesniak Mack T. Ruffin Jenna Wiens Patrick D. Schloss
author_facet	Begüm D. Topçuoğlu Nicholas A. Lesniak Mack T. Ruffin Jenna Wiens Patrick D. Schloss
author_sort	Begüm D. Topçuoğlu
title	A Framework for Effective Application of Machine Learning to Microbiome-Based Classification Problems
title_short	A Framework for Effective Application of Machine Learning to Microbiome-Based Classification Problems
title_full	A Framework for Effective Application of Machine Learning to Microbiome-Based Classification Problems
title_fullStr	A Framework for Effective Application of Machine Learning to Microbiome-Based Classification Problems
title_full_unstemmed	A Framework for Effective Application of Machine Learning to Microbiome-Based Classification Problems
title_sort	framework for effective application of machine learning to microbiome-based classification problems
publisher	American Society for Microbiology
publishDate	2020
url	https://doaj.org/article/47b3a3d71e7741b48a1bbd42dbb2aeb9
work_keys_str_mv	AT begumdtopcuoglu aframeworkforeffectiveapplicationofmachinelearningtomicrobiomebasedclassificationproblems AT nicholasalesniak aframeworkforeffectiveapplicationofmachinelearningtomicrobiomebasedclassificationproblems AT macktruffin aframeworkforeffectiveapplicationofmachinelearningtomicrobiomebasedclassificationproblems AT jennawiens aframeworkforeffectiveapplicationofmachinelearningtomicrobiomebasedclassificationproblems AT patrickdschloss aframeworkforeffectiveapplicationofmachinelearningtomicrobiomebasedclassificationproblems AT begumdtopcuoglu frameworkforeffectiveapplicationofmachinelearningtomicrobiomebasedclassificationproblems AT nicholasalesniak frameworkforeffectiveapplicationofmachinelearningtomicrobiomebasedclassificationproblems AT macktruffin frameworkforeffectiveapplicationofmachinelearningtomicrobiomebasedclassificationproblems AT jennawiens frameworkforeffectiveapplicationofmachinelearningtomicrobiomebasedclassificationproblems AT patrickdschloss frameworkforeffectiveapplicationofmachinelearningtomicrobiomebasedclassificationproblems
_version_	1718427085450510336

A Framework for Effective Application of Machine Learning to Microbiome-Based Classification Problems

Ejemplares similares