The unbiased estimation of the fraction of variance explained by a model.
The correlation coefficient squared, r2, is commonly used to validate quantitative models on neural data, yet it is biased by trial-to-trial variability: as trial-to-trial variability increases, measured correlation to a model's predictions decreases. As a result, models that perfectly explain...
Guardado en:
| Autores principales: | , |
|---|---|
| Formato: | article |
| Lenguaje: | EN |
| Publicado: |
Public Library of Science (PLoS)
2021
|
| Materias: | |
| Acceso en línea: | https://doaj.org/article/4217079778f2440a98e5cedb15d03483 |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| id |
oai:doaj.org-article:4217079778f2440a98e5cedb15d03483 |
|---|---|
| record_format |
dspace |
| spelling |
oai:doaj.org-article:4217079778f2440a98e5cedb15d034832021-12-02T19:58:08ZThe unbiased estimation of the fraction of variance explained by a model.1553-734X1553-735810.1371/journal.pcbi.1009212https://doaj.org/article/4217079778f2440a98e5cedb15d034832021-08-01T00:00:00Zhttps://doi.org/10.1371/journal.pcbi.1009212https://doaj.org/toc/1553-734Xhttps://doaj.org/toc/1553-7358The correlation coefficient squared, r2, is commonly used to validate quantitative models on neural data, yet it is biased by trial-to-trial variability: as trial-to-trial variability increases, measured correlation to a model's predictions decreases. As a result, models that perfectly explain neural tuning can appear to perform poorly. Many solutions to this problem have been proposed, but no consensus has been reached on which is the least biased estimator. Some currently used methods substantially overestimate model fit, and the utility of even the best performing methods is limited by the lack of confidence intervals and asymptotic analysis. We provide a new estimator, [Formula: see text], that outperforms all prior estimators in our testing, and we provide confidence intervals and asymptotic guarantees. We apply our estimator to a variety of neural data to validate its utility. We find that neural noise is often so great that confidence intervals of the estimator cover the entire possible range of values ([0, 1]), preventing meaningful evaluation of the quality of a model's predictions. This leads us to propose the use of the signal-to-noise ratio (SNR) as a quality metric for making quantitative comparisons across neural recordings. Analyzing a variety of neural data sets, we find that up to ∼ 40% of some state-of-the-art neural recordings do not pass even a liberal SNR criterion. Moving toward more reliable estimates of correlation, and quantitatively comparing quality across recording modalities and data sets, will be critical to accelerating progress in modeling biological phenomena.Dean A PospisilWyeth BairPublic Library of Science (PLoS)articleBiology (General)QH301-705.5ENPLoS Computational Biology, Vol 17, Iss 8, p e1009212 (2021) |
| institution |
DOAJ |
| collection |
DOAJ |
| language |
EN |
| topic |
Biology (General) QH301-705.5 |
| spellingShingle |
Biology (General) QH301-705.5 Dean A Pospisil Wyeth Bair The unbiased estimation of the fraction of variance explained by a model. |
| description |
The correlation coefficient squared, r2, is commonly used to validate quantitative models on neural data, yet it is biased by trial-to-trial variability: as trial-to-trial variability increases, measured correlation to a model's predictions decreases. As a result, models that perfectly explain neural tuning can appear to perform poorly. Many solutions to this problem have been proposed, but no consensus has been reached on which is the least biased estimator. Some currently used methods substantially overestimate model fit, and the utility of even the best performing methods is limited by the lack of confidence intervals and asymptotic analysis. We provide a new estimator, [Formula: see text], that outperforms all prior estimators in our testing, and we provide confidence intervals and asymptotic guarantees. We apply our estimator to a variety of neural data to validate its utility. We find that neural noise is often so great that confidence intervals of the estimator cover the entire possible range of values ([0, 1]), preventing meaningful evaluation of the quality of a model's predictions. This leads us to propose the use of the signal-to-noise ratio (SNR) as a quality metric for making quantitative comparisons across neural recordings. Analyzing a variety of neural data sets, we find that up to ∼ 40% of some state-of-the-art neural recordings do not pass even a liberal SNR criterion. Moving toward more reliable estimates of correlation, and quantitatively comparing quality across recording modalities and data sets, will be critical to accelerating progress in modeling biological phenomena. |
| format |
article |
| author |
Dean A Pospisil Wyeth Bair |
| author_facet |
Dean A Pospisil Wyeth Bair |
| author_sort |
Dean A Pospisil |
| title |
The unbiased estimation of the fraction of variance explained by a model. |
| title_short |
The unbiased estimation of the fraction of variance explained by a model. |
| title_full |
The unbiased estimation of the fraction of variance explained by a model. |
| title_fullStr |
The unbiased estimation of the fraction of variance explained by a model. |
| title_full_unstemmed |
The unbiased estimation of the fraction of variance explained by a model. |
| title_sort |
unbiased estimation of the fraction of variance explained by a model. |
| publisher |
Public Library of Science (PLoS) |
| publishDate |
2021 |
| url |
https://doaj.org/article/4217079778f2440a98e5cedb15d03483 |
| work_keys_str_mv |
AT deanapospisil theunbiasedestimationofthefractionofvarianceexplainedbyamodel AT wyethbair theunbiasedestimationofthefractionofvarianceexplainedbyamodel AT deanapospisil unbiasedestimationofthefractionofvarianceexplainedbyamodel AT wyethbair unbiasedestimationofthefractionofvarianceexplainedbyamodel |
| _version_ |
1718375774110613504 |