Effect of deep learning-based assistive technology use on chest radiograph interpretation by emergency department physicians: a prospective interventional simulation-based study

Abstract Background Interpretation of chest radiographs (CRs) by emergency department (ED) physicians is inferior to that by radiologists. Recent studies have investigated the effect of deep learning-based assistive technology on CR interpretation (DLCR), although its relevance to ED physicians rema...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	Ji Hoon Kim, Sang Gil Han, Ara Cho, Hye Jung Shin, Song-Ee Baek
Formato:	article
Lenguaje:	EN
Publicado:	BMC 2021
Materias:	Chest radiograph Emergency department Deep learning-based assistive technology Decision-making Computer applications to medicine. Medical informatics R858-859.7
Acceso en línea:	https://doaj.org/article/708f74a503d14da78185fd2464308dc7
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:708f74a503d14da78185fd2464308dc7
record_format	dspace
spelling	oai:doaj.org-article:708f74a503d14da78185fd2464308dc72021-11-14T12:29:15ZEffect of deep learning-based assistive technology use on chest radiograph interpretation by emergency department physicians: a prospective interventional simulation-based study10.1186/s12911-021-01679-41472-6947https://doaj.org/article/708f74a503d14da78185fd2464308dc72021-11-01T00:00:00Zhttps://doi.org/10.1186/s12911-021-01679-4https://doaj.org/toc/1472-6947Abstract Background Interpretation of chest radiographs (CRs) by emergency department (ED) physicians is inferior to that by radiologists. Recent studies have investigated the effect of deep learning-based assistive technology on CR interpretation (DLCR), although its relevance to ED physicians remains unclear. This study aimed to investigate whether DLCR supports CR interpretation and the clinical decision-making of ED physicians. Methods We conducted a prospective interventional study using a web-based performance assessment system. Study participants were recruited through the official notice targeting board for certified emergency physicians and residents working at the present ED. Of the eight ED physicians who volunteered to participate in the study, seven ED physicians were included, while one participant declared withdrawal during performance assessment. Seven physicians’ CR interpretations and clinical decision-making were assessed based on the clinical data from 388 patients, including detecting the target lesion with DLCR. Participant performance was evaluated by area under the receiver operating characteristic curve (AUROC), sensitivity, specificity, and accuracy analyses; decision-making consistency was measured by kappa statistics. ED physicians with < 24 months of experience were defined as ‘inexperienced’. Results Among the 388 simulated cases, 259 (66.8%) had CR abnormality. Their median value of abnormality score measured by DLCR was 59.3 (31.77, 76.25) compared to a score of 3.35 (1.57, 8.89) for cases of normal CR. There was a difference in performance between ED physicians working with and without DLCR (AUROC: 0.801, P < 0.001). The diagnostic sensitivity and accuracy of CR were higher for all ED physicians working with DLCR than for those working without it. The overall kappa value for decision-making consistency was 0.902 (95% confidence interval [CI] 0.884–0.920); concurrently, the kappa value for the experienced group was 0.956 (95% CI 0.934–0.979), and that for the inexperienced group was 0.862 (95% CI 0.835–0.889). Conclusions This study presents preliminary evidence that ED physicians using DLCR in a clinical setting perform better at CR interpretation than their counterparts who do not use this technology. DLCR use influenced the clinical decision-making of inexperienced physicians more strongly than that of experienced physicians. These findings require prospective validation before DLCR can be recommended for use in routine clinical practice.Ji Hoon KimSang Gil HanAra ChoHye Jung ShinSong-Ee BaekBMCarticleChest radiographEmergency departmentDeep learning-based assistive technologyDecision-makingComputer applications to medicine. Medical informaticsR858-859.7ENBMC Medical Informatics and Decision Making, Vol 21, Iss 1, Pp 1-9 (2021)
institution	DOAJ
collection	DOAJ
language	EN
topic	Chest radiograph Emergency department Deep learning-based assistive technology Decision-making Computer applications to medicine. Medical informatics R858-859.7
spellingShingle	Chest radiograph Emergency department Deep learning-based assistive technology Decision-making Computer applications to medicine. Medical informatics R858-859.7 Ji Hoon Kim Sang Gil Han Ara Cho Hye Jung Shin Song-Ee Baek Effect of deep learning-based assistive technology use on chest radiograph interpretation by emergency department physicians: a prospective interventional simulation-based study
description	Abstract Background Interpretation of chest radiographs (CRs) by emergency department (ED) physicians is inferior to that by radiologists. Recent studies have investigated the effect of deep learning-based assistive technology on CR interpretation (DLCR), although its relevance to ED physicians remains unclear. This study aimed to investigate whether DLCR supports CR interpretation and the clinical decision-making of ED physicians. Methods We conducted a prospective interventional study using a web-based performance assessment system. Study participants were recruited through the official notice targeting board for certified emergency physicians and residents working at the present ED. Of the eight ED physicians who volunteered to participate in the study, seven ED physicians were included, while one participant declared withdrawal during performance assessment. Seven physicians’ CR interpretations and clinical decision-making were assessed based on the clinical data from 388 patients, including detecting the target lesion with DLCR. Participant performance was evaluated by area under the receiver operating characteristic curve (AUROC), sensitivity, specificity, and accuracy analyses; decision-making consistency was measured by kappa statistics. ED physicians with < 24 months of experience were defined as ‘inexperienced’. Results Among the 388 simulated cases, 259 (66.8%) had CR abnormality. Their median value of abnormality score measured by DLCR was 59.3 (31.77, 76.25) compared to a score of 3.35 (1.57, 8.89) for cases of normal CR. There was a difference in performance between ED physicians working with and without DLCR (AUROC: 0.801, P < 0.001). The diagnostic sensitivity and accuracy of CR were higher for all ED physicians working with DLCR than for those working without it. The overall kappa value for decision-making consistency was 0.902 (95% confidence interval [CI] 0.884–0.920); concurrently, the kappa value for the experienced group was 0.956 (95% CI 0.934–0.979), and that for the inexperienced group was 0.862 (95% CI 0.835–0.889). Conclusions This study presents preliminary evidence that ED physicians using DLCR in a clinical setting perform better at CR interpretation than their counterparts who do not use this technology. DLCR use influenced the clinical decision-making of inexperienced physicians more strongly than that of experienced physicians. These findings require prospective validation before DLCR can be recommended for use in routine clinical practice.
format	article
author	Ji Hoon Kim Sang Gil Han Ara Cho Hye Jung Shin Song-Ee Baek
author_facet	Ji Hoon Kim Sang Gil Han Ara Cho Hye Jung Shin Song-Ee Baek
author_sort	Ji Hoon Kim
title	Effect of deep learning-based assistive technology use on chest radiograph interpretation by emergency department physicians: a prospective interventional simulation-based study
title_short	Effect of deep learning-based assistive technology use on chest radiograph interpretation by emergency department physicians: a prospective interventional simulation-based study
title_full	Effect of deep learning-based assistive technology use on chest radiograph interpretation by emergency department physicians: a prospective interventional simulation-based study
title_fullStr	Effect of deep learning-based assistive technology use on chest radiograph interpretation by emergency department physicians: a prospective interventional simulation-based study
title_full_unstemmed	Effect of deep learning-based assistive technology use on chest radiograph interpretation by emergency department physicians: a prospective interventional simulation-based study
title_sort	effect of deep learning-based assistive technology use on chest radiograph interpretation by emergency department physicians: a prospective interventional simulation-based study
publisher	BMC
publishDate	2021
url	https://doaj.org/article/708f74a503d14da78185fd2464308dc7
work_keys_str_mv	AT jihoonkim effectofdeeplearningbasedassistivetechnologyuseonchestradiographinterpretationbyemergencydepartmentphysiciansaprospectiveinterventionalsimulationbasedstudy AT sanggilhan effectofdeeplearningbasedassistivetechnologyuseonchestradiographinterpretationbyemergencydepartmentphysiciansaprospectiveinterventionalsimulationbasedstudy AT aracho effectofdeeplearningbasedassistivetechnologyuseonchestradiographinterpretationbyemergencydepartmentphysiciansaprospectiveinterventionalsimulationbasedstudy AT hyejungshin effectofdeeplearningbasedassistivetechnologyuseonchestradiographinterpretationbyemergencydepartmentphysiciansaprospectiveinterventionalsimulationbasedstudy AT songeebaek effectofdeeplearningbasedassistivetechnologyuseonchestradiographinterpretationbyemergencydepartmentphysiciansaprospectiveinterventionalsimulationbasedstudy
_version_	1718429173734703104

Effect of deep learning-based assistive technology use on chest radiograph interpretation by emergency department physicians: a prospective interventional simulation-based study

Ejemplares similares