The Effect of Outcome Selection on the Performance of Prediction Models in Patients at Risk for Sepsis

Importance:. Risk prediction models for patients with suspected sepsis have been derived on and applied to various outcomes, including readily available outcomes such as hospital mortality and ICU admission as well as longer-term mortality outcomes that may be more important to patients. It is unkno...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	Stephanie P. Taylor, MD, MS, Shih-Hsiung Chou, PhD, Andrew D. McWilliams, MD, MPH, Mark Russo, MD, Alan C. Heffner, MD, Stephanie Murphy, DO, Susan L. Evans, MD, FACS, FCCM, Whitney Rossman, MS, Marc Kowalkowski, PhD, on behalf of Acute Care Outcomes Research Network (ACORN) Investigators, Ryan Brown, Larry Burke, Shih-Hsiung Chou, Kyle Cunningham, Susan L. Evans, Scott Furney, Michael Gibbs, Alan Heffner, Timothy Hetherington, Daniel Howard, Marc Kowalkowski, Scott Lindblom, Andrea McCall, Lewis McCurdy, Andrew McWilliams, Stephanie Murphy, Alfred Papali, Christopher Polk, Whitney Rossman, Michael Runyon, Mark Russo, Melanie Spencer, Brice Taylor, Stephanie Taylor
Formato:	article
Lenguaje:	EN
Publicado:	Wolters Kluwer 2020
Materias:	Medical emergencies. Critical care. Intensive care. First aid RC86-88.9
Acceso en línea:	https://doaj.org/article/f02d5b96c37e49f685dad9fb517fbbe7
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:f02d5b96c37e49f685dad9fb517fbbe7
record_format	dspace
spelling	oai:doaj.org-article:f02d5b96c37e49f685dad9fb517fbbe72021-11-25T07:50:55ZThe Effect of Outcome Selection on the Performance of Prediction Models in Patients at Risk for Sepsis2639-802810.1097/CCE.0000000000000078https://doaj.org/article/f02d5b96c37e49f685dad9fb517fbbe72020-01-01T00:00:00Zhttp://journals.lww.com/10.1097/CCE.0000000000000078https://doaj.org/toc/2639-8028Importance:. Risk prediction models for patients with suspected sepsis have been derived on and applied to various outcomes, including readily available outcomes such as hospital mortality and ICU admission as well as longer-term mortality outcomes that may be more important to patients. It is unknown how selecting different outcomes influences model performance in patients at risk for sepsis. Objectives:. Evaluate the impact of outcome selection on risk model performance and weighting of individual predictor variables. Design, Setting, and Participants:. We retrospectively analyzed adults hospitalized with suspected infection from January 2014 to September 2017 at 12 hospitals. Main Outcomes and Measures:. We used routinely collected clinical data to derive logistic regression models for four outcomes: hospital mortality, composite ICU length of stay greater than 72 hours or hospital mortality, 30-day mortality, and 90-day mortality. We compared the performance of the models using area under the receiver operating characteristic curve and calibration plots. Results:. Among 52,184 admissions, 2,030 (4%) experienced hospital mortality, 6,659 (13%) experienced the composite of hospital mortality or ICU length of stay greater than 72 hours, 3,417 (7%) experienced 30-day mortality, and 5,655 (11%) experienced 90-day mortality. Area under the receiver operating characteristic curves decreased when hospital-based models were applied to predict 30-day (hospital mortality = 0.88–0.85; –0.03, composite ICU length of stay greater than 72 hours or hospital mortality = 0.90–0.81; –0.09) and 90-day mortality (hospital mortality = 0.88–0.81; –0.07, composite ICU length of stay greater than 72 hours or hospital mortality = 0.90–0.76; –0.14; all p < 0.01). Models were well calibrated for derived (root-mean-square error = 5–15) but not alternate outcomes (root-mean-square error = 8–35). Conclusions and Relevance:. Risk models trained to predict readily available hospital-based outcomes in suspected sepsis show poorer discrimination and calibration when applied to 30- and 90-day mortality. Interpretation and application of risk models for patients at risk of sepsis should consider these findings.Stephanie P. Taylor, MD, MSShih-Hsiung Chou, PhDAndrew D. McWilliams, MD, MPHMark Russo, MDAlan C. Heffner, MDStephanie Murphy, DOSusan L. Evans, MD, FACS, FCCMWhitney Rossman, MSMarc Kowalkowski, PhDon behalf of Acute Care Outcomes Research Network (ACORN) InvestigatorsRyan BrownLarry BurkeShih-Hsiung ChouKyle CunninghamSusan L. EvansScott FurneyMichael GibbsAlan HeffnerTimothy HetheringtonDaniel HowardMarc KowalkowskiScott LindblomAndrea McCallLewis McCurdyAndrew McWilliamsStephanie MurphyAlfred PapaliChristopher PolkWhitney RossmanMichael RunyonMark RussoMelanie SpencerBrice TaylorStephanie TaylorWolters KluwerarticleMedical emergencies. Critical care. Intensive care. First aidRC86-88.9ENCritical Care Explorations, Vol 2, Iss 1, p e0078 (2020)
institution	DOAJ
collection	DOAJ
language	EN
topic	Medical emergencies. Critical care. Intensive care. First aid RC86-88.9
spellingShingle	Medical emergencies. Critical care. Intensive care. First aid RC86-88.9 Stephanie P. Taylor, MD, MS Shih-Hsiung Chou, PhD Andrew D. McWilliams, MD, MPH Mark Russo, MD Alan C. Heffner, MD Stephanie Murphy, DO Susan L. Evans, MD, FACS, FCCM Whitney Rossman, MS Marc Kowalkowski, PhD on behalf of Acute Care Outcomes Research Network (ACORN) Investigators Ryan Brown Larry Burke Shih-Hsiung Chou Kyle Cunningham Susan L. Evans Scott Furney Michael Gibbs Alan Heffner Timothy Hetherington Daniel Howard Marc Kowalkowski Scott Lindblom Andrea McCall Lewis McCurdy Andrew McWilliams Stephanie Murphy Alfred Papali Christopher Polk Whitney Rossman Michael Runyon Mark Russo Melanie Spencer Brice Taylor Stephanie Taylor The Effect of Outcome Selection on the Performance of Prediction Models in Patients at Risk for Sepsis
description	Importance:. Risk prediction models for patients with suspected sepsis have been derived on and applied to various outcomes, including readily available outcomes such as hospital mortality and ICU admission as well as longer-term mortality outcomes that may be more important to patients. It is unknown how selecting different outcomes influences model performance in patients at risk for sepsis. Objectives:. Evaluate the impact of outcome selection on risk model performance and weighting of individual predictor variables. Design, Setting, and Participants:. We retrospectively analyzed adults hospitalized with suspected infection from January 2014 to September 2017 at 12 hospitals. Main Outcomes and Measures:. We used routinely collected clinical data to derive logistic regression models for four outcomes: hospital mortality, composite ICU length of stay greater than 72 hours or hospital mortality, 30-day mortality, and 90-day mortality. We compared the performance of the models using area under the receiver operating characteristic curve and calibration plots. Results:. Among 52,184 admissions, 2,030 (4%) experienced hospital mortality, 6,659 (13%) experienced the composite of hospital mortality or ICU length of stay greater than 72 hours, 3,417 (7%) experienced 30-day mortality, and 5,655 (11%) experienced 90-day mortality. Area under the receiver operating characteristic curves decreased when hospital-based models were applied to predict 30-day (hospital mortality = 0.88–0.85; –0.03, composite ICU length of stay greater than 72 hours or hospital mortality = 0.90–0.81; –0.09) and 90-day mortality (hospital mortality = 0.88–0.81; –0.07, composite ICU length of stay greater than 72 hours or hospital mortality = 0.90–0.76; –0.14; all p < 0.01). Models were well calibrated for derived (root-mean-square error = 5–15) but not alternate outcomes (root-mean-square error = 8–35). Conclusions and Relevance:. Risk models trained to predict readily available hospital-based outcomes in suspected sepsis show poorer discrimination and calibration when applied to 30- and 90-day mortality. Interpretation and application of risk models for patients at risk of sepsis should consider these findings.
format	article
author	Stephanie P. Taylor, MD, MS Shih-Hsiung Chou, PhD Andrew D. McWilliams, MD, MPH Mark Russo, MD Alan C. Heffner, MD Stephanie Murphy, DO Susan L. Evans, MD, FACS, FCCM Whitney Rossman, MS Marc Kowalkowski, PhD on behalf of Acute Care Outcomes Research Network (ACORN) Investigators Ryan Brown Larry Burke Shih-Hsiung Chou Kyle Cunningham Susan L. Evans Scott Furney Michael Gibbs Alan Heffner Timothy Hetherington Daniel Howard Marc Kowalkowski Scott Lindblom Andrea McCall Lewis McCurdy Andrew McWilliams Stephanie Murphy Alfred Papali Christopher Polk Whitney Rossman Michael Runyon Mark Russo Melanie Spencer Brice Taylor Stephanie Taylor
author_facet	Stephanie P. Taylor, MD, MS Shih-Hsiung Chou, PhD Andrew D. McWilliams, MD, MPH Mark Russo, MD Alan C. Heffner, MD Stephanie Murphy, DO Susan L. Evans, MD, FACS, FCCM Whitney Rossman, MS Marc Kowalkowski, PhD on behalf of Acute Care Outcomes Research Network (ACORN) Investigators Ryan Brown Larry Burke Shih-Hsiung Chou Kyle Cunningham Susan L. Evans Scott Furney Michael Gibbs Alan Heffner Timothy Hetherington Daniel Howard Marc Kowalkowski Scott Lindblom Andrea McCall Lewis McCurdy Andrew McWilliams Stephanie Murphy Alfred Papali Christopher Polk Whitney Rossman Michael Runyon Mark Russo Melanie Spencer Brice Taylor Stephanie Taylor
author_sort	Stephanie P. Taylor, MD, MS
title	The Effect of Outcome Selection on the Performance of Prediction Models in Patients at Risk for Sepsis
title_short	The Effect of Outcome Selection on the Performance of Prediction Models in Patients at Risk for Sepsis
title_full	The Effect of Outcome Selection on the Performance of Prediction Models in Patients at Risk for Sepsis
title_fullStr	The Effect of Outcome Selection on the Performance of Prediction Models in Patients at Risk for Sepsis
title_full_unstemmed	The Effect of Outcome Selection on the Performance of Prediction Models in Patients at Risk for Sepsis
title_sort	effect of outcome selection on the performance of prediction models in patients at risk for sepsis
publisher	Wolters Kluwer
publishDate	2020
url	https://doaj.org/article/f02d5b96c37e49f685dad9fb517fbbe7
work_keys_str_mv	AT stephanieptaylormdms theeffectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT shihhsiungchouphd theeffectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT andrewdmcwilliamsmdmph theeffectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT markrussomd theeffectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT alancheffnermd theeffectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT stephaniemurphydo theeffectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT susanlevansmdfacsfccm theeffectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT whitneyrossmanms theeffectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT marckowalkowskiphd theeffectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT onbehalfofacutecareoutcomesresearchnetworkacorninvestigators theeffectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT ryanbrown theeffectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT larryburke theeffectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT shihhsiungchou theeffectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT kylecunningham theeffectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT susanlevans theeffectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT scottfurney theeffectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT michaelgibbs theeffectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT alanheffner theeffectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT timothyhetherington theeffectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT danielhoward theeffectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT marckowalkowski theeffectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT scottlindblom theeffectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT andreamccall theeffectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT lewismccurdy theeffectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT andrewmcwilliams theeffectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT stephaniemurphy theeffectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT alfredpapali theeffectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT christopherpolk theeffectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT whitneyrossman theeffectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT michaelrunyon theeffectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT markrusso theeffectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT melaniespencer theeffectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT bricetaylor theeffectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT stephanietaylor theeffectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT stephanieptaylormdms effectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT shihhsiungchouphd effectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT andrewdmcwilliamsmdmph effectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT markrussomd effectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT alancheffnermd effectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT stephaniemurphydo effectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT susanlevansmdfacsfccm effectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT whitneyrossmanms effectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT marckowalkowskiphd effectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT onbehalfofacutecareoutcomesresearchnetworkacorninvestigators effectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT ryanbrown effectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT larryburke effectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT shihhsiungchou effectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT kylecunningham effectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT susanlevans effectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT scottfurney effectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT michaelgibbs effectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT alanheffner effectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT timothyhetherington effectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT danielhoward effectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT marckowalkowski effectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT scottlindblom effectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT andreamccall effectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT lewismccurdy effectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT andrewmcwilliams effectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT stephaniemurphy effectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT alfredpapali effectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT christopherpolk effectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT whitneyrossman effectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT michaelrunyon effectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT markrusso effectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT melaniespencer effectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT bricetaylor effectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis AT stephanietaylor effectofoutcomeselectionontheperformanceofpredictionmodelsinpatientsatriskforsepsis
_version_	1718413655246110720

The Effect of Outcome Selection on the Performance of Prediction Models in Patients at Risk for Sepsis

Ejemplares similares