Measuring the bias of incorrect application of feature selection when using cross-validation in radiomics

Abstract Background Many studies in radiomics are using feature selection methods to identify the most predictive features. At the same time, they employ cross-validation to estimate the performance of the developed models. However, if the feature selection is performed before the cross-validation,...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autor principal:	Aydin Demircioğlu
Formato:	article
Lenguaje:	EN
Publicado:	SpringerOpen 2021
Materias:	Radiomics Feature selection Cross-validation Bias Machine learning Medical physics. Medical radiology. Nuclear medicine R895-920
Acceso en línea:	https://doaj.org/article/e05fc95b049c4fdc8f28b17d1566ac18
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:e05fc95b049c4fdc8f28b17d1566ac18
record_format	dspace
spelling	oai:doaj.org-article:e05fc95b049c4fdc8f28b17d1566ac182021-11-28T12:08:45ZMeasuring the bias of incorrect application of feature selection when using cross-validation in radiomics10.1186/s13244-021-01115-11869-4101https://doaj.org/article/e05fc95b049c4fdc8f28b17d1566ac182021-11-01T00:00:00Zhttps://doi.org/10.1186/s13244-021-01115-1https://doaj.org/toc/1869-4101Abstract Background Many studies in radiomics are using feature selection methods to identify the most predictive features. At the same time, they employ cross-validation to estimate the performance of the developed models. However, if the feature selection is performed before the cross-validation, data leakage can occur, and the results can be biased. To measure the extent of this bias, we collected ten publicly available radiomics datasets and conducted two experiments. First, the models were developed by incorrectly applying the feature selection prior to cross-validation. Then, the same experiment was conducted by applying feature selection correctly within cross-validation to each fold. The resulting models were then evaluated against each other in terms of AUC-ROC, AUC-F1, and Accuracy. Results Applying the feature selection incorrectly prior to the cross-validation showed a bias of up to 0.15 in AUC-ROC, 0.29 in AUC-F1, and 0.17 in Accuracy. Conclusions Incorrect application of feature selection and cross-validation can lead to highly biased results for radiomic datasets.Aydin DemircioğluSpringerOpenarticleRadiomicsFeature selectionCross-validationBiasMachine learningMedical physics. Medical radiology. Nuclear medicineR895-920ENInsights into Imaging, Vol 12, Iss 1, Pp 1-10 (2021)
institution	DOAJ
collection	DOAJ
language	EN
topic	Radiomics Feature selection Cross-validation Bias Machine learning Medical physics. Medical radiology. Nuclear medicine R895-920
spellingShingle	Radiomics Feature selection Cross-validation Bias Machine learning Medical physics. Medical radiology. Nuclear medicine R895-920 Aydin Demircioğlu Measuring the bias of incorrect application of feature selection when using cross-validation in radiomics
description	Abstract Background Many studies in radiomics are using feature selection methods to identify the most predictive features. At the same time, they employ cross-validation to estimate the performance of the developed models. However, if the feature selection is performed before the cross-validation, data leakage can occur, and the results can be biased. To measure the extent of this bias, we collected ten publicly available radiomics datasets and conducted two experiments. First, the models were developed by incorrectly applying the feature selection prior to cross-validation. Then, the same experiment was conducted by applying feature selection correctly within cross-validation to each fold. The resulting models were then evaluated against each other in terms of AUC-ROC, AUC-F1, and Accuracy. Results Applying the feature selection incorrectly prior to the cross-validation showed a bias of up to 0.15 in AUC-ROC, 0.29 in AUC-F1, and 0.17 in Accuracy. Conclusions Incorrect application of feature selection and cross-validation can lead to highly biased results for radiomic datasets.
format	article
author	Aydin Demircioğlu
author_facet	Aydin Demircioğlu
author_sort	Aydin Demircioğlu
title	Measuring the bias of incorrect application of feature selection when using cross-validation in radiomics
title_short	Measuring the bias of incorrect application of feature selection when using cross-validation in radiomics
title_full	Measuring the bias of incorrect application of feature selection when using cross-validation in radiomics
title_fullStr	Measuring the bias of incorrect application of feature selection when using cross-validation in radiomics
title_full_unstemmed	Measuring the bias of incorrect application of feature selection when using cross-validation in radiomics
title_sort	measuring the bias of incorrect application of feature selection when using cross-validation in radiomics
publisher	SpringerOpen
publishDate	2021
url	https://doaj.org/article/e05fc95b049c4fdc8f28b17d1566ac18
work_keys_str_mv	AT aydindemircioglu measuringthebiasofincorrectapplicationoffeatureselectionwhenusingcrossvalidationinradiomics
_version_	1718408229604556800

Measuring the bias of incorrect application of feature selection when using cross-validation in radiomics

Ejemplares similares