The meaning of significant mean group differences for biomarker discovery.

Over the past decade, biomarker discovery has become a key goal in psychiatry to aid in the more reliable diagnosis and prognosis of heterogeneous psychiatric conditions and the development of tailored therapies. Nevertheless, the prevailing statistical approach is still the mean group comparison be...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	Eva Loth, Jumana Ahmad, Chris Chatham, Beatriz López, Ben Carter, Daisy Crawley, Bethany Oakley, Hannah Hayward, Jennifer Cooke, Antonia San José Cáceres, Danilo Bzdok, Emily Jones, Tony Charman, Christian Beckmann, Thomas Bourgeron, Roberto Toro, Jan Buitelaar, Declan Murphy, Guillaume Dumas
Formato:	article
Lenguaje:	EN
Publicado:	Public Library of Science (PLoS) 2021
Materias:	Biology (General) QH301-705.5
Acceso en línea:	https://doaj.org/article/629a873022144e9c8bfb48ce61106e14
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:629a873022144e9c8bfb48ce61106e14
record_format	dspace
spelling	oai:doaj.org-article:629a873022144e9c8bfb48ce61106e142021-12-02T19:57:57ZThe meaning of significant mean group differences for biomarker discovery.1553-734X1553-735810.1371/journal.pcbi.1009477https://doaj.org/article/629a873022144e9c8bfb48ce61106e142021-11-01T00:00:00Zhttps://doi.org/10.1371/journal.pcbi.1009477https://doaj.org/toc/1553-734Xhttps://doaj.org/toc/1553-7358Over the past decade, biomarker discovery has become a key goal in psychiatry to aid in the more reliable diagnosis and prognosis of heterogeneous psychiatric conditions and the development of tailored therapies. Nevertheless, the prevailing statistical approach is still the mean group comparison between "cases" and "controls," which tends to ignore within-group variability. In this educational article, we used empirical data simulations to investigate how effect size, sample size, and the shape of distributions impact the interpretation of mean group differences for biomarker discovery. We then applied these statistical criteria to evaluate biomarker discovery in one area of psychiatric research-autism research. Across the most influential areas of autism research, effect size estimates ranged from small (d = 0.21, anatomical structure) to medium (d = 0.36 electrophysiology, d = 0.5, eye-tracking) to large (d = 1.1 theory of mind). We show that in normal distributions, this translates to approximately 45% to 63% of cases performing within 1 standard deviation (SD) of the typical range, i.e., they do not have a deficit/atypicality in a statistical sense. For a measure to have diagnostic utility as defined by 80% sensitivity and 80% specificity, Cohen's d of 1.66 is required, with still 40% of cases falling within 1 SD. However, in both normal and nonnormal distributions, 1 (skewness) or 2 (platykurtic, bimodal) biologically plausible subgroups may exist despite small or even nonsignificant mean group differences. This conclusion drastically contrasts the way mean group differences are frequently reported. Over 95% of studies omitted the "on average" when summarising their findings in their abstracts ("autistic people have deficits in X"), which can be misleading as it implies that the group-level difference applies to all individuals in that group. We outline practical approaches and steps for researchers to explore mean group comparisons for the discovery of stratification biomarkers.Eva LothJumana AhmadChris ChathamBeatriz LópezBen CarterDaisy CrawleyBethany OakleyHannah HaywardJennifer CookeAntonia San José CáceresDanilo BzdokEmily JonesTony CharmanChristian BeckmannThomas BourgeronRoberto ToroJan BuitelaarDeclan MurphyGuillaume DumasPublic Library of Science (PLoS)articleBiology (General)QH301-705.5ENPLoS Computational Biology, Vol 17, Iss 11, p e1009477 (2021)
institution	DOAJ
collection	DOAJ
language	EN
topic	Biology (General) QH301-705.5
spellingShingle	Biology (General) QH301-705.5 Eva Loth Jumana Ahmad Chris Chatham Beatriz López Ben Carter Daisy Crawley Bethany Oakley Hannah Hayward Jennifer Cooke Antonia San José Cáceres Danilo Bzdok Emily Jones Tony Charman Christian Beckmann Thomas Bourgeron Roberto Toro Jan Buitelaar Declan Murphy Guillaume Dumas The meaning of significant mean group differences for biomarker discovery.
description	Over the past decade, biomarker discovery has become a key goal in psychiatry to aid in the more reliable diagnosis and prognosis of heterogeneous psychiatric conditions and the development of tailored therapies. Nevertheless, the prevailing statistical approach is still the mean group comparison between "cases" and "controls," which tends to ignore within-group variability. In this educational article, we used empirical data simulations to investigate how effect size, sample size, and the shape of distributions impact the interpretation of mean group differences for biomarker discovery. We then applied these statistical criteria to evaluate biomarker discovery in one area of psychiatric research-autism research. Across the most influential areas of autism research, effect size estimates ranged from small (d = 0.21, anatomical structure) to medium (d = 0.36 electrophysiology, d = 0.5, eye-tracking) to large (d = 1.1 theory of mind). We show that in normal distributions, this translates to approximately 45% to 63% of cases performing within 1 standard deviation (SD) of the typical range, i.e., they do not have a deficit/atypicality in a statistical sense. For a measure to have diagnostic utility as defined by 80% sensitivity and 80% specificity, Cohen's d of 1.66 is required, with still 40% of cases falling within 1 SD. However, in both normal and nonnormal distributions, 1 (skewness) or 2 (platykurtic, bimodal) biologically plausible subgroups may exist despite small or even nonsignificant mean group differences. This conclusion drastically contrasts the way mean group differences are frequently reported. Over 95% of studies omitted the "on average" when summarising their findings in their abstracts ("autistic people have deficits in X"), which can be misleading as it implies that the group-level difference applies to all individuals in that group. We outline practical approaches and steps for researchers to explore mean group comparisons for the discovery of stratification biomarkers.
format	article
author	Eva Loth Jumana Ahmad Chris Chatham Beatriz López Ben Carter Daisy Crawley Bethany Oakley Hannah Hayward Jennifer Cooke Antonia San José Cáceres Danilo Bzdok Emily Jones Tony Charman Christian Beckmann Thomas Bourgeron Roberto Toro Jan Buitelaar Declan Murphy Guillaume Dumas
author_facet	Eva Loth Jumana Ahmad Chris Chatham Beatriz López Ben Carter Daisy Crawley Bethany Oakley Hannah Hayward Jennifer Cooke Antonia San José Cáceres Danilo Bzdok Emily Jones Tony Charman Christian Beckmann Thomas Bourgeron Roberto Toro Jan Buitelaar Declan Murphy Guillaume Dumas
author_sort	Eva Loth
title	The meaning of significant mean group differences for biomarker discovery.
title_short	The meaning of significant mean group differences for biomarker discovery.
title_full	The meaning of significant mean group differences for biomarker discovery.
title_fullStr	The meaning of significant mean group differences for biomarker discovery.
title_full_unstemmed	The meaning of significant mean group differences for biomarker discovery.
title_sort	meaning of significant mean group differences for biomarker discovery.
publisher	Public Library of Science (PLoS)
publishDate	2021
url	https://doaj.org/article/629a873022144e9c8bfb48ce61106e14
work_keys_str_mv	AT evaloth themeaningofsignificantmeangroupdifferencesforbiomarkerdiscovery AT jumanaahmad themeaningofsignificantmeangroupdifferencesforbiomarkerdiscovery AT chrischatham themeaningofsignificantmeangroupdifferencesforbiomarkerdiscovery AT beatrizlopez themeaningofsignificantmeangroupdifferencesforbiomarkerdiscovery AT bencarter themeaningofsignificantmeangroupdifferencesforbiomarkerdiscovery AT daisycrawley themeaningofsignificantmeangroupdifferencesforbiomarkerdiscovery AT bethanyoakley themeaningofsignificantmeangroupdifferencesforbiomarkerdiscovery AT hannahhayward themeaningofsignificantmeangroupdifferencesforbiomarkerdiscovery AT jennifercooke themeaningofsignificantmeangroupdifferencesforbiomarkerdiscovery AT antoniasanjosecaceres themeaningofsignificantmeangroupdifferencesforbiomarkerdiscovery AT danilobzdok themeaningofsignificantmeangroupdifferencesforbiomarkerdiscovery AT emilyjones themeaningofsignificantmeangroupdifferencesforbiomarkerdiscovery AT tonycharman themeaningofsignificantmeangroupdifferencesforbiomarkerdiscovery AT christianbeckmann themeaningofsignificantmeangroupdifferencesforbiomarkerdiscovery AT thomasbourgeron themeaningofsignificantmeangroupdifferencesforbiomarkerdiscovery AT robertotoro themeaningofsignificantmeangroupdifferencesforbiomarkerdiscovery AT janbuitelaar themeaningofsignificantmeangroupdifferencesforbiomarkerdiscovery AT declanmurphy themeaningofsignificantmeangroupdifferencesforbiomarkerdiscovery AT guillaumedumas themeaningofsignificantmeangroupdifferencesforbiomarkerdiscovery AT evaloth meaningofsignificantmeangroupdifferencesforbiomarkerdiscovery AT jumanaahmad meaningofsignificantmeangroupdifferencesforbiomarkerdiscovery AT chrischatham meaningofsignificantmeangroupdifferencesforbiomarkerdiscovery AT beatrizlopez meaningofsignificantmeangroupdifferencesforbiomarkerdiscovery AT bencarter meaningofsignificantmeangroupdifferencesforbiomarkerdiscovery AT daisycrawley meaningofsignificantmeangroupdifferencesforbiomarkerdiscovery AT bethanyoakley meaningofsignificantmeangroupdifferencesforbiomarkerdiscovery AT hannahhayward meaningofsignificantmeangroupdifferencesforbiomarkerdiscovery AT jennifercooke meaningofsignificantmeangroupdifferencesforbiomarkerdiscovery AT antoniasanjosecaceres meaningofsignificantmeangroupdifferencesforbiomarkerdiscovery AT danilobzdok meaningofsignificantmeangroupdifferencesforbiomarkerdiscovery AT emilyjones meaningofsignificantmeangroupdifferencesforbiomarkerdiscovery AT tonycharman meaningofsignificantmeangroupdifferencesforbiomarkerdiscovery AT christianbeckmann meaningofsignificantmeangroupdifferencesforbiomarkerdiscovery AT thomasbourgeron meaningofsignificantmeangroupdifferencesforbiomarkerdiscovery AT robertotoro meaningofsignificantmeangroupdifferencesforbiomarkerdiscovery AT janbuitelaar meaningofsignificantmeangroupdifferencesforbiomarkerdiscovery AT declanmurphy meaningofsignificantmeangroupdifferencesforbiomarkerdiscovery AT guillaumedumas meaningofsignificantmeangroupdifferencesforbiomarkerdiscovery
_version_	1718375779770826752

The meaning of significant mean group differences for biomarker discovery.

Ejemplares similares