Partitioning variability in animal behavioral videos using semi-supervised variational autoencoders.
Recent neuroscience studies demonstrate that a deeper understanding of brain function requires a deeper understanding of behavior. Detailed behavioral measurements are now often collected using video cameras, resulting in an increased need for computer vision algorithms that extract useful informati...
Guardado en:
| Autores principales: | , , , , , , , , , , , , , , , , , |
|---|---|
| Formato: | article |
| Lenguaje: | EN |
| Publicado: |
Public Library of Science (PLoS)
2021
|
| Materias: | |
| Acceso en línea: | https://doaj.org/article/8c044a2fffaa489890ca5e3b6a6b6664 |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| id |
oai:doaj.org-article:8c044a2fffaa489890ca5e3b6a6b6664 |
|---|---|
| record_format |
dspace |
| spelling |
oai:doaj.org-article:8c044a2fffaa489890ca5e3b6a6b66642021-12-02T19:57:44ZPartitioning variability in animal behavioral videos using semi-supervised variational autoencoders.1553-734X1553-735810.1371/journal.pcbi.1009439https://doaj.org/article/8c044a2fffaa489890ca5e3b6a6b66642021-09-01T00:00:00Zhttps://doi.org/10.1371/journal.pcbi.1009439https://doaj.org/toc/1553-734Xhttps://doaj.org/toc/1553-7358Recent neuroscience studies demonstrate that a deeper understanding of brain function requires a deeper understanding of behavior. Detailed behavioral measurements are now often collected using video cameras, resulting in an increased need for computer vision algorithms that extract useful information from video data. Here we introduce a new video analysis tool that combines the output of supervised pose estimation algorithms (e.g. DeepLabCut) with unsupervised dimensionality reduction methods to produce interpretable, low-dimensional representations of behavioral videos that extract more information than pose estimates alone. We demonstrate this tool by extracting interpretable behavioral features from videos of three different head-fixed mouse preparations, as well as a freely moving mouse in an open field arena, and show how these interpretable features can facilitate downstream behavioral and neural analyses. We also show how the behavioral features produced by our model improve the precision and interpretation of these downstream analyses compared to using the outputs of either fully supervised or fully unsupervised methods alone.Matthew R WhitewayDan BidermanYoni FriedmanMario DipoppaE Kelly BuchananAnqi WuJohn ZhouNiccolò BonacchiNathaniel J MiskaJean-Paul NoelErica RodriguezMichael SchartnerKarolina SochaAnne E UraiC Daniel SalzmanInternational Brain LaboratoryJohn P CunninghamLiam PaninskiPublic Library of Science (PLoS)articleBiology (General)QH301-705.5ENPLoS Computational Biology, Vol 17, Iss 9, p e1009439 (2021) |
| institution |
DOAJ |
| collection |
DOAJ |
| language |
EN |
| topic |
Biology (General) QH301-705.5 |
| spellingShingle |
Biology (General) QH301-705.5 Matthew R Whiteway Dan Biderman Yoni Friedman Mario Dipoppa E Kelly Buchanan Anqi Wu John Zhou Niccolò Bonacchi Nathaniel J Miska Jean-Paul Noel Erica Rodriguez Michael Schartner Karolina Socha Anne E Urai C Daniel Salzman International Brain Laboratory John P Cunningham Liam Paninski Partitioning variability in animal behavioral videos using semi-supervised variational autoencoders. |
| description |
Recent neuroscience studies demonstrate that a deeper understanding of brain function requires a deeper understanding of behavior. Detailed behavioral measurements are now often collected using video cameras, resulting in an increased need for computer vision algorithms that extract useful information from video data. Here we introduce a new video analysis tool that combines the output of supervised pose estimation algorithms (e.g. DeepLabCut) with unsupervised dimensionality reduction methods to produce interpretable, low-dimensional representations of behavioral videos that extract more information than pose estimates alone. We demonstrate this tool by extracting interpretable behavioral features from videos of three different head-fixed mouse preparations, as well as a freely moving mouse in an open field arena, and show how these interpretable features can facilitate downstream behavioral and neural analyses. We also show how the behavioral features produced by our model improve the precision and interpretation of these downstream analyses compared to using the outputs of either fully supervised or fully unsupervised methods alone. |
| format |
article |
| author |
Matthew R Whiteway Dan Biderman Yoni Friedman Mario Dipoppa E Kelly Buchanan Anqi Wu John Zhou Niccolò Bonacchi Nathaniel J Miska Jean-Paul Noel Erica Rodriguez Michael Schartner Karolina Socha Anne E Urai C Daniel Salzman International Brain Laboratory John P Cunningham Liam Paninski |
| author_facet |
Matthew R Whiteway Dan Biderman Yoni Friedman Mario Dipoppa E Kelly Buchanan Anqi Wu John Zhou Niccolò Bonacchi Nathaniel J Miska Jean-Paul Noel Erica Rodriguez Michael Schartner Karolina Socha Anne E Urai C Daniel Salzman International Brain Laboratory John P Cunningham Liam Paninski |
| author_sort |
Matthew R Whiteway |
| title |
Partitioning variability in animal behavioral videos using semi-supervised variational autoencoders. |
| title_short |
Partitioning variability in animal behavioral videos using semi-supervised variational autoencoders. |
| title_full |
Partitioning variability in animal behavioral videos using semi-supervised variational autoencoders. |
| title_fullStr |
Partitioning variability in animal behavioral videos using semi-supervised variational autoencoders. |
| title_full_unstemmed |
Partitioning variability in animal behavioral videos using semi-supervised variational autoencoders. |
| title_sort |
partitioning variability in animal behavioral videos using semi-supervised variational autoencoders. |
| publisher |
Public Library of Science (PLoS) |
| publishDate |
2021 |
| url |
https://doaj.org/article/8c044a2fffaa489890ca5e3b6a6b6664 |
| work_keys_str_mv |
AT matthewrwhiteway partitioningvariabilityinanimalbehavioralvideosusingsemisupervisedvariationalautoencoders AT danbiderman partitioningvariabilityinanimalbehavioralvideosusingsemisupervisedvariationalautoencoders AT yonifriedman partitioningvariabilityinanimalbehavioralvideosusingsemisupervisedvariationalautoencoders AT mariodipoppa partitioningvariabilityinanimalbehavioralvideosusingsemisupervisedvariationalautoencoders AT ekellybuchanan partitioningvariabilityinanimalbehavioralvideosusingsemisupervisedvariationalautoencoders AT anqiwu partitioningvariabilityinanimalbehavioralvideosusingsemisupervisedvariationalautoencoders AT johnzhou partitioningvariabilityinanimalbehavioralvideosusingsemisupervisedvariationalautoencoders AT niccolobonacchi partitioningvariabilityinanimalbehavioralvideosusingsemisupervisedvariationalautoencoders AT nathanieljmiska partitioningvariabilityinanimalbehavioralvideosusingsemisupervisedvariationalautoencoders AT jeanpaulnoel partitioningvariabilityinanimalbehavioralvideosusingsemisupervisedvariationalautoencoders AT ericarodriguez partitioningvariabilityinanimalbehavioralvideosusingsemisupervisedvariationalautoencoders AT michaelschartner partitioningvariabilityinanimalbehavioralvideosusingsemisupervisedvariationalautoencoders AT karolinasocha partitioningvariabilityinanimalbehavioralvideosusingsemisupervisedvariationalautoencoders AT anneeurai partitioningvariabilityinanimalbehavioralvideosusingsemisupervisedvariationalautoencoders AT cdanielsalzman partitioningvariabilityinanimalbehavioralvideosusingsemisupervisedvariationalautoencoders AT internationalbrainlaboratory partitioningvariabilityinanimalbehavioralvideosusingsemisupervisedvariationalautoencoders AT johnpcunningham partitioningvariabilityinanimalbehavioralvideosusingsemisupervisedvariationalautoencoders AT liampaninski partitioningvariabilityinanimalbehavioralvideosusingsemisupervisedvariationalautoencoders |
| _version_ |
1718375789350617088 |