Accuracy of deep learning-based computed tomography diagnostic system for COVID-19: A consecutive sampling external validation cohort study

Ali-M3, an artificial intelligence program, analyzes chest computed tomography (CT) and detects the likelihood of coronavirus disease (COVID-19) based on scores ranging from 0 to 1. However, Ali-M3 has not been externally validated. Our aim was to evaluate the accuracy of Ali-M3 for detecting COVID-...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	Tatsuyoshi Ikenoue, Yuki Kataoka, Yoshinori Matsuoka, Junichi Matsumoto, Junji Kumasawa, Kentaro Tochitatni, Hiraku Funakoshi, Tomohiro Hosoda, Aiko Kugimiya, Michinori Shirano, Fumiko Hamabe, Sachiyo Iwata, Shingo Fukuma, Japan COVID-19 AI team
Formato:	article
Lenguaje:	EN
Publicado:	Public Library of Science (PLoS) 2021
Materias:	Medicine R Science Q
Acceso en línea:	https://doaj.org/article/b272f81f741141169857c1ff9495d25d
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:b272f81f741141169857c1ff9495d25d
record_format	dspace
spelling	oai:doaj.org-article:b272f81f741141169857c1ff9495d25d2021-11-11T07:14:41ZAccuracy of deep learning-based computed tomography diagnostic system for COVID-19: A consecutive sampling external validation cohort study1932-6203https://doaj.org/article/b272f81f741141169857c1ff9495d25d2021-01-01T00:00:00Zhttps://www.ncbi.nlm.nih.gov/pmc/articles/PMC8568139/?tool=EBIhttps://doaj.org/toc/1932-6203Ali-M3, an artificial intelligence program, analyzes chest computed tomography (CT) and detects the likelihood of coronavirus disease (COVID-19) based on scores ranging from 0 to 1. However, Ali-M3 has not been externally validated. Our aim was to evaluate the accuracy of Ali-M3 for detecting COVID-19 and discuss its clinical value. We evaluated the external validity of Ali-M3 using sequential Japanese sampling data. In this retrospective cohort study, COVID-19 infection probabilities for 617 symptomatic patients were determined using Ali-M3. In 11 Japanese tertiary care facilities, these patients underwent reverse transcription-polymerase chain reaction (RT-PCR) testing. They also underwent chest CT to confirm a diagnosis of COVID-19. Of the 617 patients, 289 (46.8%) were RT-PCR-positive. The area under the curve (AUC) of Ali-M3 for predicting a COVID-19 diagnosis was 0.797 (95% confidence interval: 0.762‒0.833) and the goodness-of-fit was P = 0.156. With a cut-off probability of a diagnosis of COVID-19 by Ali-M3 set at 0.5, the sensitivity and specificity were 80.6% and 68.3%, respectively. A cut-off of 0.2 yielded a sensitivity and specificity of 89.2% and 43.2%, respectively. Among the 223 patients who required oxygen, the AUC was 0.825. Sensitivity at a cut-off of 0.5% and 0.2% was 88.7% and 97.9%, respectively. Although the sensitivity was lower when the days from symptom onset were fewer, the sensitivity increased for both cut-off values after 5 days. We evaluated Ali-M3 using external validation with symptomatic patient data from Japanese tertiary care facilities. As Ali-M3 showed sufficient sensitivity performance, despite a lower specificity performance, Ali-M3 could be useful in excluding a diagnosis of COVID-19.Tatsuyoshi IkenoueYuki KataokaYoshinori MatsuokaJunichi MatsumotoJunji KumasawaKentaro TochitatniHiraku FunakoshiTomohiro HosodaAiko KugimiyaMichinori ShiranoFumiko HamabeSachiyo IwataShingo FukumaJapan COVID-19 AI teamPublic Library of Science (PLoS)articleMedicineRScienceQENPLoS ONE, Vol 16, Iss 11 (2021)
institution	DOAJ
collection	DOAJ
language	EN
topic	Medicine R Science Q
spellingShingle	Medicine R Science Q Tatsuyoshi Ikenoue Yuki Kataoka Yoshinori Matsuoka Junichi Matsumoto Junji Kumasawa Kentaro Tochitatni Hiraku Funakoshi Tomohiro Hosoda Aiko Kugimiya Michinori Shirano Fumiko Hamabe Sachiyo Iwata Shingo Fukuma Japan COVID-19 AI team Accuracy of deep learning-based computed tomography diagnostic system for COVID-19: A consecutive sampling external validation cohort study
description	Ali-M3, an artificial intelligence program, analyzes chest computed tomography (CT) and detects the likelihood of coronavirus disease (COVID-19) based on scores ranging from 0 to 1. However, Ali-M3 has not been externally validated. Our aim was to evaluate the accuracy of Ali-M3 for detecting COVID-19 and discuss its clinical value. We evaluated the external validity of Ali-M3 using sequential Japanese sampling data. In this retrospective cohort study, COVID-19 infection probabilities for 617 symptomatic patients were determined using Ali-M3. In 11 Japanese tertiary care facilities, these patients underwent reverse transcription-polymerase chain reaction (RT-PCR) testing. They also underwent chest CT to confirm a diagnosis of COVID-19. Of the 617 patients, 289 (46.8%) were RT-PCR-positive. The area under the curve (AUC) of Ali-M3 for predicting a COVID-19 diagnosis was 0.797 (95% confidence interval: 0.762‒0.833) and the goodness-of-fit was P = 0.156. With a cut-off probability of a diagnosis of COVID-19 by Ali-M3 set at 0.5, the sensitivity and specificity were 80.6% and 68.3%, respectively. A cut-off of 0.2 yielded a sensitivity and specificity of 89.2% and 43.2%, respectively. Among the 223 patients who required oxygen, the AUC was 0.825. Sensitivity at a cut-off of 0.5% and 0.2% was 88.7% and 97.9%, respectively. Although the sensitivity was lower when the days from symptom onset were fewer, the sensitivity increased for both cut-off values after 5 days. We evaluated Ali-M3 using external validation with symptomatic patient data from Japanese tertiary care facilities. As Ali-M3 showed sufficient sensitivity performance, despite a lower specificity performance, Ali-M3 could be useful in excluding a diagnosis of COVID-19.
format	article
author	Tatsuyoshi Ikenoue Yuki Kataoka Yoshinori Matsuoka Junichi Matsumoto Junji Kumasawa Kentaro Tochitatni Hiraku Funakoshi Tomohiro Hosoda Aiko Kugimiya Michinori Shirano Fumiko Hamabe Sachiyo Iwata Shingo Fukuma Japan COVID-19 AI team
author_facet	Tatsuyoshi Ikenoue Yuki Kataoka Yoshinori Matsuoka Junichi Matsumoto Junji Kumasawa Kentaro Tochitatni Hiraku Funakoshi Tomohiro Hosoda Aiko Kugimiya Michinori Shirano Fumiko Hamabe Sachiyo Iwata Shingo Fukuma Japan COVID-19 AI team
author_sort	Tatsuyoshi Ikenoue
title	Accuracy of deep learning-based computed tomography diagnostic system for COVID-19: A consecutive sampling external validation cohort study
title_short	Accuracy of deep learning-based computed tomography diagnostic system for COVID-19: A consecutive sampling external validation cohort study
title_full	Accuracy of deep learning-based computed tomography diagnostic system for COVID-19: A consecutive sampling external validation cohort study
title_fullStr	Accuracy of deep learning-based computed tomography diagnostic system for COVID-19: A consecutive sampling external validation cohort study
title_full_unstemmed	Accuracy of deep learning-based computed tomography diagnostic system for COVID-19: A consecutive sampling external validation cohort study
title_sort	accuracy of deep learning-based computed tomography diagnostic system for covid-19: a consecutive sampling external validation cohort study
publisher	Public Library of Science (PLoS)
publishDate	2021
url	https://doaj.org/article/b272f81f741141169857c1ff9495d25d
work_keys_str_mv	AT tatsuyoshiikenoue accuracyofdeeplearningbasedcomputedtomographydiagnosticsystemforcovid19aconsecutivesamplingexternalvalidationcohortstudy AT yukikataoka accuracyofdeeplearningbasedcomputedtomographydiagnosticsystemforcovid19aconsecutivesamplingexternalvalidationcohortstudy AT yoshinorimatsuoka accuracyofdeeplearningbasedcomputedtomographydiagnosticsystemforcovid19aconsecutivesamplingexternalvalidationcohortstudy AT junichimatsumoto accuracyofdeeplearningbasedcomputedtomographydiagnosticsystemforcovid19aconsecutivesamplingexternalvalidationcohortstudy AT junjikumasawa accuracyofdeeplearningbasedcomputedtomographydiagnosticsystemforcovid19aconsecutivesamplingexternalvalidationcohortstudy AT kentarotochitatni accuracyofdeeplearningbasedcomputedtomographydiagnosticsystemforcovid19aconsecutivesamplingexternalvalidationcohortstudy AT hirakufunakoshi accuracyofdeeplearningbasedcomputedtomographydiagnosticsystemforcovid19aconsecutivesamplingexternalvalidationcohortstudy AT tomohirohosoda accuracyofdeeplearningbasedcomputedtomographydiagnosticsystemforcovid19aconsecutivesamplingexternalvalidationcohortstudy AT aikokugimiya accuracyofdeeplearningbasedcomputedtomographydiagnosticsystemforcovid19aconsecutivesamplingexternalvalidationcohortstudy AT michinorishirano accuracyofdeeplearningbasedcomputedtomographydiagnosticsystemforcovid19aconsecutivesamplingexternalvalidationcohortstudy AT fumikohamabe accuracyofdeeplearningbasedcomputedtomographydiagnosticsystemforcovid19aconsecutivesamplingexternalvalidationcohortstudy AT sachiyoiwata accuracyofdeeplearningbasedcomputedtomographydiagnosticsystemforcovid19aconsecutivesamplingexternalvalidationcohortstudy AT shingofukuma accuracyofdeeplearningbasedcomputedtomographydiagnosticsystemforcovid19aconsecutivesamplingexternalvalidationcohortstudy AT japancovid19aiteam accuracyofdeeplearningbasedcomputedtomographydiagnosticsystemforcovid19aconsecutivesamplingexternalvalidationcohortstudy
_version_	1718439430765674496

Accuracy of deep learning-based computed tomography diagnostic system for COVID-19: A consecutive sampling external validation cohort study

Ejemplares similares