Lung nodule detection in chest X-rays using synthetic ground-truth data comparing CNN-based diagnosis to human performance
Abstract We present a method to generate synthetic thorax radiographs with realistic nodules from CT scans, and a perfect ground truth knowledge. We evaluated the detection performance of nine radiologists and two convolutional neural networks in a reader study. Nodules were artificially inserted in...
Guardado en:
| Autores principales: | , , , , , , , , , , , , , , , , |
|---|---|
| Formato: | article |
| Lenguaje: | EN |
| Publicado: |
Nature Portfolio
2021
|
| Materias: | |
| Acceso en línea: | https://doaj.org/article/d2e234157a904202804449b039ffdf57 |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| id |
oai:doaj.org-article:d2e234157a904202804449b039ffdf57 |
|---|---|
| record_format |
dspace |
| spelling |
oai:doaj.org-article:d2e234157a904202804449b039ffdf572021-12-02T18:49:23ZLung nodule detection in chest X-rays using synthetic ground-truth data comparing CNN-based diagnosis to human performance10.1038/s41598-021-94750-z2045-2322https://doaj.org/article/d2e234157a904202804449b039ffdf572021-08-01T00:00:00Zhttps://doi.org/10.1038/s41598-021-94750-zhttps://doaj.org/toc/2045-2322Abstract We present a method to generate synthetic thorax radiographs with realistic nodules from CT scans, and a perfect ground truth knowledge. We evaluated the detection performance of nine radiologists and two convolutional neural networks in a reader study. Nodules were artificially inserted into the lung of a CT volume and synthetic radiographs were obtained by forward-projecting the volume. Hence, our framework allowed for a detailed evaluation of CAD systems’ and radiologists’ performance due to the availability of accurate ground-truth labels for nodules from synthetic data. Radiographs for network training (U-Net and RetinaNet) were generated from 855 CT scans of a public dataset. For the reader study, 201 radiographs were generated from 21 nodule-free CT scans with altering nodule positions, sizes and nodule counts of inserted nodules. Average true positive detections by nine radiologists were 248.8 nodules, 51.7 false positive predicted nodules and 121.2 false negative predicted nodules. The best performing CAD system achieved 268 true positives, 66 false positives and 102 false negatives. Corresponding weighted alternative free response operating characteristic figure-of-merits (wAFROC FOM) for the radiologists range from 0.54 to 0.87 compared to a value of 0.81 (CI 0.75–0.87) for the best performing CNN. The CNN did not perform significantly better against the combined average of the 9 readers (p = 0.49). Paramediastinal nodules accounted for most false positive and false negative detections by readers, which can be explained by the presence of more tissue in this area.Manuel SchultheissPhilipp SchmetteJannis BoddenJuliane AicheleChristina Müller-LeisseFelix G. GassertFlorian T. GassertJoshua F. GawlitzaFelix C. HofmannDaniel SasseClaudio E. von SchackySebastian ZiegelmayerFabio De MarcoBernhard RengerMarcus R. MakowskiFranz PfeifferDaniela PfeifferNature PortfolioarticleMedicineRScienceQENScientific Reports, Vol 11, Iss 1, Pp 1-10 (2021) |
| institution |
DOAJ |
| collection |
DOAJ |
| language |
EN |
| topic |
Medicine R Science Q |
| spellingShingle |
Medicine R Science Q Manuel Schultheiss Philipp Schmette Jannis Bodden Juliane Aichele Christina Müller-Leisse Felix G. Gassert Florian T. Gassert Joshua F. Gawlitza Felix C. Hofmann Daniel Sasse Claudio E. von Schacky Sebastian Ziegelmayer Fabio De Marco Bernhard Renger Marcus R. Makowski Franz Pfeiffer Daniela Pfeiffer Lung nodule detection in chest X-rays using synthetic ground-truth data comparing CNN-based diagnosis to human performance |
| description |
Abstract We present a method to generate synthetic thorax radiographs with realistic nodules from CT scans, and a perfect ground truth knowledge. We evaluated the detection performance of nine radiologists and two convolutional neural networks in a reader study. Nodules were artificially inserted into the lung of a CT volume and synthetic radiographs were obtained by forward-projecting the volume. Hence, our framework allowed for a detailed evaluation of CAD systems’ and radiologists’ performance due to the availability of accurate ground-truth labels for nodules from synthetic data. Radiographs for network training (U-Net and RetinaNet) were generated from 855 CT scans of a public dataset. For the reader study, 201 radiographs were generated from 21 nodule-free CT scans with altering nodule positions, sizes and nodule counts of inserted nodules. Average true positive detections by nine radiologists were 248.8 nodules, 51.7 false positive predicted nodules and 121.2 false negative predicted nodules. The best performing CAD system achieved 268 true positives, 66 false positives and 102 false negatives. Corresponding weighted alternative free response operating characteristic figure-of-merits (wAFROC FOM) for the radiologists range from 0.54 to 0.87 compared to a value of 0.81 (CI 0.75–0.87) for the best performing CNN. The CNN did not perform significantly better against the combined average of the 9 readers (p = 0.49). Paramediastinal nodules accounted for most false positive and false negative detections by readers, which can be explained by the presence of more tissue in this area. |
| format |
article |
| author |
Manuel Schultheiss Philipp Schmette Jannis Bodden Juliane Aichele Christina Müller-Leisse Felix G. Gassert Florian T. Gassert Joshua F. Gawlitza Felix C. Hofmann Daniel Sasse Claudio E. von Schacky Sebastian Ziegelmayer Fabio De Marco Bernhard Renger Marcus R. Makowski Franz Pfeiffer Daniela Pfeiffer |
| author_facet |
Manuel Schultheiss Philipp Schmette Jannis Bodden Juliane Aichele Christina Müller-Leisse Felix G. Gassert Florian T. Gassert Joshua F. Gawlitza Felix C. Hofmann Daniel Sasse Claudio E. von Schacky Sebastian Ziegelmayer Fabio De Marco Bernhard Renger Marcus R. Makowski Franz Pfeiffer Daniela Pfeiffer |
| author_sort |
Manuel Schultheiss |
| title |
Lung nodule detection in chest X-rays using synthetic ground-truth data comparing CNN-based diagnosis to human performance |
| title_short |
Lung nodule detection in chest X-rays using synthetic ground-truth data comparing CNN-based diagnosis to human performance |
| title_full |
Lung nodule detection in chest X-rays using synthetic ground-truth data comparing CNN-based diagnosis to human performance |
| title_fullStr |
Lung nodule detection in chest X-rays using synthetic ground-truth data comparing CNN-based diagnosis to human performance |
| title_full_unstemmed |
Lung nodule detection in chest X-rays using synthetic ground-truth data comparing CNN-based diagnosis to human performance |
| title_sort |
lung nodule detection in chest x-rays using synthetic ground-truth data comparing cnn-based diagnosis to human performance |
| publisher |
Nature Portfolio |
| publishDate |
2021 |
| url |
https://doaj.org/article/d2e234157a904202804449b039ffdf57 |
| work_keys_str_mv |
AT manuelschultheiss lungnoduledetectioninchestxraysusingsyntheticgroundtruthdatacomparingcnnbaseddiagnosistohumanperformance AT philippschmette lungnoduledetectioninchestxraysusingsyntheticgroundtruthdatacomparingcnnbaseddiagnosistohumanperformance AT jannisbodden lungnoduledetectioninchestxraysusingsyntheticgroundtruthdatacomparingcnnbaseddiagnosistohumanperformance AT julianeaichele lungnoduledetectioninchestxraysusingsyntheticgroundtruthdatacomparingcnnbaseddiagnosistohumanperformance AT christinamullerleisse lungnoduledetectioninchestxraysusingsyntheticgroundtruthdatacomparingcnnbaseddiagnosistohumanperformance AT felixggassert lungnoduledetectioninchestxraysusingsyntheticgroundtruthdatacomparingcnnbaseddiagnosistohumanperformance AT floriantgassert lungnoduledetectioninchestxraysusingsyntheticgroundtruthdatacomparingcnnbaseddiagnosistohumanperformance AT joshuafgawlitza lungnoduledetectioninchestxraysusingsyntheticgroundtruthdatacomparingcnnbaseddiagnosistohumanperformance AT felixchofmann lungnoduledetectioninchestxraysusingsyntheticgroundtruthdatacomparingcnnbaseddiagnosistohumanperformance AT danielsasse lungnoduledetectioninchestxraysusingsyntheticgroundtruthdatacomparingcnnbaseddiagnosistohumanperformance AT claudioevonschacky lungnoduledetectioninchestxraysusingsyntheticgroundtruthdatacomparingcnnbaseddiagnosistohumanperformance AT sebastianziegelmayer lungnoduledetectioninchestxraysusingsyntheticgroundtruthdatacomparingcnnbaseddiagnosistohumanperformance AT fabiodemarco lungnoduledetectioninchestxraysusingsyntheticgroundtruthdatacomparingcnnbaseddiagnosistohumanperformance AT bernhardrenger lungnoduledetectioninchestxraysusingsyntheticgroundtruthdatacomparingcnnbaseddiagnosistohumanperformance AT marcusrmakowski lungnoduledetectioninchestxraysusingsyntheticgroundtruthdatacomparingcnnbaseddiagnosistohumanperformance AT franzpfeiffer lungnoduledetectioninchestxraysusingsyntheticgroundtruthdatacomparingcnnbaseddiagnosistohumanperformance AT danielapfeiffer lungnoduledetectioninchestxraysusingsyntheticgroundtruthdatacomparingcnnbaseddiagnosistohumanperformance |
| _version_ |
1718377592287920128 |