Crowdsourced privacy-preserved feature tagging of short home videos for machine learning ASD detection

Abstract Standard medical diagnosis of mental health conditions requires licensed experts who are increasingly outnumbered by those at risk, limiting reach. We test the hypothesis that a trustworthy crowd of non-experts can efficiently annotate behavioral features needed for accurate machine learnin...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	Peter Washington, Qandeel Tariq, Emilie Leblanc, Brianna Chrisman, Kaitlyn Dunlap, Aaron Kline, Haik Kalantarian, Yordan Penev, Kelley Paskov, Catalin Voss, Nathaniel Stockham, Maya Varma, Arman Husic, Jack Kent, Nick Haber, Terry Winograd, Dennis P. Wall
Formato:	article
Lenguaje:	EN
Publicado:	Nature Portfolio 2021
Materias:	Medicine R Science Q
Acceso en línea:	https://doaj.org/article/7c1e977803b04ec7833a0f2e6600ab08
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:7c1e977803b04ec7833a0f2e6600ab08
record_format	dspace
spelling	oai:doaj.org-article:7c1e977803b04ec7833a0f2e6600ab082021-12-02T14:37:07ZCrowdsourced privacy-preserved feature tagging of short home videos for machine learning ASD detection10.1038/s41598-021-87059-42045-2322https://doaj.org/article/7c1e977803b04ec7833a0f2e6600ab082021-04-01T00:00:00Zhttps://doi.org/10.1038/s41598-021-87059-4https://doaj.org/toc/2045-2322Abstract Standard medical diagnosis of mental health conditions requires licensed experts who are increasingly outnumbered by those at risk, limiting reach. We test the hypothesis that a trustworthy crowd of non-experts can efficiently annotate behavioral features needed for accurate machine learning detection of the common childhood developmental disorder Autism Spectrum Disorder (ASD) for children under 8 years old. We implement a novel process for identifying and certifying a trustworthy distributed workforce for video feature extraction, selecting a workforce of 102 workers from a pool of 1,107. Two previously validated ASD logistic regression classifiers, evaluated against parent-reported diagnoses, were used to assess the accuracy of the trusted crowd’s ratings of unstructured home videos. A representative balanced sample (N = 50 videos) of videos were evaluated with and without face box and pitch shift privacy alterations, with AUROC and AUPRC scores > 0.98. With both privacy-preserving modifications, sensitivity is preserved (96.0%) while maintaining specificity (80.0%) and accuracy (88.0%) at levels comparable to prior classification methods without alterations. We find that machine learning classification from features extracted by a certified nonexpert crowd achieves high performance for ASD detection from natural home videos of the child at risk and maintains high sensitivity when privacy-preserving mechanisms are applied. These results suggest that privacy-safeguarded crowdsourced analysis of short home videos can help enable rapid and mobile machine-learning detection of developmental delays in children.Peter WashingtonQandeel TariqEmilie LeblancBrianna ChrismanKaitlyn DunlapAaron KlineHaik KalantarianYordan PenevKelley PaskovCatalin VossNathaniel StockhamMaya VarmaArman HusicJack KentNick HaberTerry WinogradDennis P. WallNature PortfolioarticleMedicineRScienceQENScientific Reports, Vol 11, Iss 1, Pp 1-11 (2021)
institution	DOAJ
collection	DOAJ
language	EN
topic	Medicine R Science Q
spellingShingle	Medicine R Science Q Peter Washington Qandeel Tariq Emilie Leblanc Brianna Chrisman Kaitlyn Dunlap Aaron Kline Haik Kalantarian Yordan Penev Kelley Paskov Catalin Voss Nathaniel Stockham Maya Varma Arman Husic Jack Kent Nick Haber Terry Winograd Dennis P. Wall Crowdsourced privacy-preserved feature tagging of short home videos for machine learning ASD detection
description	Abstract Standard medical diagnosis of mental health conditions requires licensed experts who are increasingly outnumbered by those at risk, limiting reach. We test the hypothesis that a trustworthy crowd of non-experts can efficiently annotate behavioral features needed for accurate machine learning detection of the common childhood developmental disorder Autism Spectrum Disorder (ASD) for children under 8 years old. We implement a novel process for identifying and certifying a trustworthy distributed workforce for video feature extraction, selecting a workforce of 102 workers from a pool of 1,107. Two previously validated ASD logistic regression classifiers, evaluated against parent-reported diagnoses, were used to assess the accuracy of the trusted crowd’s ratings of unstructured home videos. A representative balanced sample (N = 50 videos) of videos were evaluated with and without face box and pitch shift privacy alterations, with AUROC and AUPRC scores > 0.98. With both privacy-preserving modifications, sensitivity is preserved (96.0%) while maintaining specificity (80.0%) and accuracy (88.0%) at levels comparable to prior classification methods without alterations. We find that machine learning classification from features extracted by a certified nonexpert crowd achieves high performance for ASD detection from natural home videos of the child at risk and maintains high sensitivity when privacy-preserving mechanisms are applied. These results suggest that privacy-safeguarded crowdsourced analysis of short home videos can help enable rapid and mobile machine-learning detection of developmental delays in children.
format	article
author	Peter Washington Qandeel Tariq Emilie Leblanc Brianna Chrisman Kaitlyn Dunlap Aaron Kline Haik Kalantarian Yordan Penev Kelley Paskov Catalin Voss Nathaniel Stockham Maya Varma Arman Husic Jack Kent Nick Haber Terry Winograd Dennis P. Wall
author_facet	Peter Washington Qandeel Tariq Emilie Leblanc Brianna Chrisman Kaitlyn Dunlap Aaron Kline Haik Kalantarian Yordan Penev Kelley Paskov Catalin Voss Nathaniel Stockham Maya Varma Arman Husic Jack Kent Nick Haber Terry Winograd Dennis P. Wall
author_sort	Peter Washington
title	Crowdsourced privacy-preserved feature tagging of short home videos for machine learning ASD detection
title_short	Crowdsourced privacy-preserved feature tagging of short home videos for machine learning ASD detection
title_full	Crowdsourced privacy-preserved feature tagging of short home videos for machine learning ASD detection
title_fullStr	Crowdsourced privacy-preserved feature tagging of short home videos for machine learning ASD detection
title_full_unstemmed	Crowdsourced privacy-preserved feature tagging of short home videos for machine learning ASD detection
title_sort	crowdsourced privacy-preserved feature tagging of short home videos for machine learning asd detection
publisher	Nature Portfolio
publishDate	2021
url	https://doaj.org/article/7c1e977803b04ec7833a0f2e6600ab08
work_keys_str_mv	AT peterwashington crowdsourcedprivacypreservedfeaturetaggingofshorthomevideosformachinelearningasddetection AT qandeeltariq crowdsourcedprivacypreservedfeaturetaggingofshorthomevideosformachinelearningasddetection AT emilieleblanc crowdsourcedprivacypreservedfeaturetaggingofshorthomevideosformachinelearningasddetection AT briannachrisman crowdsourcedprivacypreservedfeaturetaggingofshorthomevideosformachinelearningasddetection AT kaitlyndunlap crowdsourcedprivacypreservedfeaturetaggingofshorthomevideosformachinelearningasddetection AT aaronkline crowdsourcedprivacypreservedfeaturetaggingofshorthomevideosformachinelearningasddetection AT haikkalantarian crowdsourcedprivacypreservedfeaturetaggingofshorthomevideosformachinelearningasddetection AT yordanpenev crowdsourcedprivacypreservedfeaturetaggingofshorthomevideosformachinelearningasddetection AT kelleypaskov crowdsourcedprivacypreservedfeaturetaggingofshorthomevideosformachinelearningasddetection AT catalinvoss crowdsourcedprivacypreservedfeaturetaggingofshorthomevideosformachinelearningasddetection AT nathanielstockham crowdsourcedprivacypreservedfeaturetaggingofshorthomevideosformachinelearningasddetection AT mayavarma crowdsourcedprivacypreservedfeaturetaggingofshorthomevideosformachinelearningasddetection AT armanhusic crowdsourcedprivacypreservedfeaturetaggingofshorthomevideosformachinelearningasddetection AT jackkent crowdsourcedprivacypreservedfeaturetaggingofshorthomevideosformachinelearningasddetection AT nickhaber crowdsourcedprivacypreservedfeaturetaggingofshorthomevideosformachinelearningasddetection AT terrywinograd crowdsourcedprivacypreservedfeaturetaggingofshorthomevideosformachinelearningasddetection AT dennispwall crowdsourcedprivacypreservedfeaturetaggingofshorthomevideosformachinelearningasddetection
_version_	1718391015208910848

Crowdsourced privacy-preserved feature tagging of short home videos for machine learning ASD detection

Ejemplares similares