The effect of noise on the predictive limit of QSAR models

Abstract A key challenge in the field of Quantitative Structure Activity Relationships (QSAR) is how to effectively treat experimental error in the training and evaluation of computational models. It is often assumed in the field of QSAR that models cannot produce predictions which are more accurate...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	Scott S. Kolmar, Christopher M. Grulke
Formato:	article
Lenguaje:	EN
Publicado:	BMC 2021
Materias:	Error Prediction error Model evaluation Gaussian process Information technology T58.5-58.64 Chemistry QD1-999
Acceso en línea:	https://doaj.org/article/78e54c4475464378b24bac9127e8e253
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:78e54c4475464378b24bac9127e8e253
record_format	dspace
spelling	oai:doaj.org-article:78e54c4475464378b24bac9127e8e2532021-11-28T12:30:24ZThe effect of noise on the predictive limit of QSAR models10.1186/s13321-021-00571-71758-2946https://doaj.org/article/78e54c4475464378b24bac9127e8e2532021-11-01T00:00:00Zhttps://doi.org/10.1186/s13321-021-00571-7https://doaj.org/toc/1758-2946Abstract A key challenge in the field of Quantitative Structure Activity Relationships (QSAR) is how to effectively treat experimental error in the training and evaluation of computational models. It is often assumed in the field of QSAR that models cannot produce predictions which are more accurate than their training data. Additionally, it is implicitly assumed, by necessity, that data points in test sets or validation sets do not contain error, and that each data point is a population mean. This work proposes the hypothesis that QSAR models can make predictions which are more accurate than their training data and that the error-free test set assumption leads to a significant misevaluation of model performance. This work used 8 datasets with six different common QSAR endpoints, because different endpoints should have different amounts of experimental error associated with varying complexity of the measurements. Up to 15 levels of simulated Gaussian distributed random error was added to the datasets, and models were built on the error laden datasets using five different algorithms. The models were trained on the error laden data, evaluated on error-laden test sets, and evaluated on error-free test sets. The results show that for each level of added error, the RMSE for evaluation on the error free test sets was always better. The results support the hypothesis that, at least under the conditions of Gaussian distributed random error, QSAR models can make predictions which are more accurate than their training data, and that the evaluation of models on error laden test and validation sets may give a flawed measure of model performance. These results have implications for how QSAR models are evaluated, especially for disciplines where experimental error is very large, such as in computational toxicology. Graphical AbstractScott S. KolmarChristopher M. GrulkeBMCarticleErrorPrediction errorModel evaluationGaussian processInformation technologyT58.5-58.64ChemistryQD1-999ENJournal of Cheminformatics, Vol 13, Iss 1, Pp 1-19 (2021)
institution	DOAJ
collection	DOAJ
language	EN
topic	Error Prediction error Model evaluation Gaussian process Information technology T58.5-58.64 Chemistry QD1-999
spellingShingle	Error Prediction error Model evaluation Gaussian process Information technology T58.5-58.64 Chemistry QD1-999 Scott S. Kolmar Christopher M. Grulke The effect of noise on the predictive limit of QSAR models
description	Abstract A key challenge in the field of Quantitative Structure Activity Relationships (QSAR) is how to effectively treat experimental error in the training and evaluation of computational models. It is often assumed in the field of QSAR that models cannot produce predictions which are more accurate than their training data. Additionally, it is implicitly assumed, by necessity, that data points in test sets or validation sets do not contain error, and that each data point is a population mean. This work proposes the hypothesis that QSAR models can make predictions which are more accurate than their training data and that the error-free test set assumption leads to a significant misevaluation of model performance. This work used 8 datasets with six different common QSAR endpoints, because different endpoints should have different amounts of experimental error associated with varying complexity of the measurements. Up to 15 levels of simulated Gaussian distributed random error was added to the datasets, and models were built on the error laden datasets using five different algorithms. The models were trained on the error laden data, evaluated on error-laden test sets, and evaluated on error-free test sets. The results show that for each level of added error, the RMSE for evaluation on the error free test sets was always better. The results support the hypothesis that, at least under the conditions of Gaussian distributed random error, QSAR models can make predictions which are more accurate than their training data, and that the evaluation of models on error laden test and validation sets may give a flawed measure of model performance. These results have implications for how QSAR models are evaluated, especially for disciplines where experimental error is very large, such as in computational toxicology. Graphical Abstract
format	article
author	Scott S. Kolmar Christopher M. Grulke
author_facet	Scott S. Kolmar Christopher M. Grulke
author_sort	Scott S. Kolmar
title	The effect of noise on the predictive limit of QSAR models
title_short	The effect of noise on the predictive limit of QSAR models
title_full	The effect of noise on the predictive limit of QSAR models
title_fullStr	The effect of noise on the predictive limit of QSAR models
title_full_unstemmed	The effect of noise on the predictive limit of QSAR models
title_sort	effect of noise on the predictive limit of qsar models
publisher	BMC
publishDate	2021
url	https://doaj.org/article/78e54c4475464378b24bac9127e8e253
work_keys_str_mv	AT scottskolmar theeffectofnoiseonthepredictivelimitofqsarmodels AT christophermgrulke theeffectofnoiseonthepredictivelimitofqsarmodels AT scottskolmar effectofnoiseonthepredictivelimitofqsarmodels AT christophermgrulke effectofnoiseonthepredictivelimitofqsarmodels
_version_	1718407964473163776

The effect of noise on the predictive limit of QSAR models

Ejemplares similares