Using explainable machine learning to characterise data drift and detect emergent health risks for emergency department admissions during COVID-19

Abstract A key task of emergency departments is to promptly identify patients who require hospital admission. Early identification ensures patient safety and aids organisational planning. Supervised machine learning algorithms can use data describing historical episodes to make ahead-of-time predict...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	Christopher Duckworth, Francis P. Chmiel, Dan K. Burns, Zlatko D. Zlatev, Neil M. White, Thomas W. V. Daniels, Michael Kiuber, Michael J. Boniface
Formato:	article
Lenguaje:	EN
Publicado:	Nature Portfolio 2021
Materias:	Medicine R Science Q
Acceso en línea:	https://doaj.org/article/404d9950339044a28d5e3a7b9f106d13
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:404d9950339044a28d5e3a7b9f106d13
record_format	dspace
spelling	oai:doaj.org-article:404d9950339044a28d5e3a7b9f106d132021-11-28T12:17:52ZUsing explainable machine learning to characterise data drift and detect emergent health risks for emergency department admissions during COVID-1910.1038/s41598-021-02481-y2045-2322https://doaj.org/article/404d9950339044a28d5e3a7b9f106d132021-11-01T00:00:00Zhttps://doi.org/10.1038/s41598-021-02481-yhttps://doaj.org/toc/2045-2322Abstract A key task of emergency departments is to promptly identify patients who require hospital admission. Early identification ensures patient safety and aids organisational planning. Supervised machine learning algorithms can use data describing historical episodes to make ahead-of-time predictions of clinical outcomes. Despite this, clinical settings are dynamic environments and the underlying data distributions characterising episodes can change with time (data drift), and so can the relationship between episode characteristics and associated clinical outcomes (concept drift). Practically this means deployed algorithms must be monitored to ensure their safety. We demonstrate how explainable machine learning can be used to monitor data drift, using the COVID-19 pandemic as a severe example. We present a machine learning classifier trained using (pre-COVID-19) data, to identify patients at high risk of admission during an emergency department attendance. We then evaluate our model’s performance on attendances occurring pre-pandemic (AUROC of 0.856 with 95%CI [0.852, 0.859]) and during the COVID-19 pandemic (AUROC of 0.826 with 95%CI [0.814, 0.837]). We demonstrate two benefits of explainable machine learning (SHAP) for models deployed in healthcare settings: (1) By tracking the variation in a feature’s SHAP value relative to its global importance, a complimentary measure of data drift is found which highlights the need to retrain a predictive model. (2) By observing the relative changes in feature importance emergent health risks can be identified.Christopher DuckworthFrancis P. ChmielDan K. BurnsZlatko D. ZlatevNeil M. WhiteThomas W. V. DanielsMichael KiuberMichael J. BonifaceNature PortfolioarticleMedicineRScienceQENScientific Reports, Vol 11, Iss 1, Pp 1-10 (2021)
institution	DOAJ
collection	DOAJ
language	EN
topic	Medicine R Science Q
spellingShingle	Medicine R Science Q Christopher Duckworth Francis P. Chmiel Dan K. Burns Zlatko D. Zlatev Neil M. White Thomas W. V. Daniels Michael Kiuber Michael J. Boniface Using explainable machine learning to characterise data drift and detect emergent health risks for emergency department admissions during COVID-19
description	Abstract A key task of emergency departments is to promptly identify patients who require hospital admission. Early identification ensures patient safety and aids organisational planning. Supervised machine learning algorithms can use data describing historical episodes to make ahead-of-time predictions of clinical outcomes. Despite this, clinical settings are dynamic environments and the underlying data distributions characterising episodes can change with time (data drift), and so can the relationship between episode characteristics and associated clinical outcomes (concept drift). Practically this means deployed algorithms must be monitored to ensure their safety. We demonstrate how explainable machine learning can be used to monitor data drift, using the COVID-19 pandemic as a severe example. We present a machine learning classifier trained using (pre-COVID-19) data, to identify patients at high risk of admission during an emergency department attendance. We then evaluate our model’s performance on attendances occurring pre-pandemic (AUROC of 0.856 with 95%CI [0.852, 0.859]) and during the COVID-19 pandemic (AUROC of 0.826 with 95%CI [0.814, 0.837]). We demonstrate two benefits of explainable machine learning (SHAP) for models deployed in healthcare settings: (1) By tracking the variation in a feature’s SHAP value relative to its global importance, a complimentary measure of data drift is found which highlights the need to retrain a predictive model. (2) By observing the relative changes in feature importance emergent health risks can be identified.
format	article
author	Christopher Duckworth Francis P. Chmiel Dan K. Burns Zlatko D. Zlatev Neil M. White Thomas W. V. Daniels Michael Kiuber Michael J. Boniface
author_facet	Christopher Duckworth Francis P. Chmiel Dan K. Burns Zlatko D. Zlatev Neil M. White Thomas W. V. Daniels Michael Kiuber Michael J. Boniface
author_sort	Christopher Duckworth
title	Using explainable machine learning to characterise data drift and detect emergent health risks for emergency department admissions during COVID-19
title_short	Using explainable machine learning to characterise data drift and detect emergent health risks for emergency department admissions during COVID-19
title_full	Using explainable machine learning to characterise data drift and detect emergent health risks for emergency department admissions during COVID-19
title_fullStr	Using explainable machine learning to characterise data drift and detect emergent health risks for emergency department admissions during COVID-19
title_full_unstemmed	Using explainable machine learning to characterise data drift and detect emergent health risks for emergency department admissions during COVID-19
title_sort	using explainable machine learning to characterise data drift and detect emergent health risks for emergency department admissions during covid-19
publisher	Nature Portfolio
publishDate	2021
url	https://doaj.org/article/404d9950339044a28d5e3a7b9f106d13
work_keys_str_mv	AT christopherduckworth usingexplainablemachinelearningtocharacterisedatadriftanddetectemergenthealthrisksforemergencydepartmentadmissionsduringcovid19 AT francispchmiel usingexplainablemachinelearningtocharacterisedatadriftanddetectemergenthealthrisksforemergencydepartmentadmissionsduringcovid19 AT dankburns usingexplainablemachinelearningtocharacterisedatadriftanddetectemergenthealthrisksforemergencydepartmentadmissionsduringcovid19 AT zlatkodzlatev usingexplainablemachinelearningtocharacterisedatadriftanddetectemergenthealthrisksforemergencydepartmentadmissionsduringcovid19 AT neilmwhite usingexplainablemachinelearningtocharacterisedatadriftanddetectemergenthealthrisksforemergencydepartmentadmissionsduringcovid19 AT thomaswvdaniels usingexplainablemachinelearningtocharacterisedatadriftanddetectemergenthealthrisksforemergencydepartmentadmissionsduringcovid19 AT michaelkiuber usingexplainablemachinelearningtocharacterisedatadriftanddetectemergenthealthrisksforemergencydepartmentadmissionsduringcovid19 AT michaeljboniface usingexplainablemachinelearningtocharacterisedatadriftanddetectemergenthealthrisksforemergencydepartmentadmissionsduringcovid19
_version_	1718408071502364672

Using explainable machine learning to characterise data drift and detect emergent health risks for emergency department admissions during COVID-19

Ejemplares similares