DeepEthogram, a machine learning pipeline for supervised behavior classification from raw pixels
Videos of animal behavior are used to quantify researcher-defined behaviors of interest to study neural function, gene mutations, and pharmacological therapies. Behaviors of interest are often scored manually, which is time-consuming, limited to few behaviors, and variable across researchers. We cre...
Guardado en:
| Autores principales: | , , , , , , , , , , |
|---|---|
| Formato: | article |
| Lenguaje: | EN |
| Publicado: |
eLife Sciences Publications Ltd
2021
|
| Materias: | |
| Acceso en línea: | https://doaj.org/article/0490bc2a85ba4831a27480dbae549b72 |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| id |
oai:doaj.org-article:0490bc2a85ba4831a27480dbae549b72 |
|---|---|
| record_format |
dspace |
| spelling |
oai:doaj.org-article:0490bc2a85ba4831a27480dbae549b722021-11-25T14:36:42ZDeepEthogram, a machine learning pipeline for supervised behavior classification from raw pixels10.7554/eLife.633772050-084Xe63377https://doaj.org/article/0490bc2a85ba4831a27480dbae549b722021-09-01T00:00:00Zhttps://elifesciences.org/articles/63377https://doaj.org/toc/2050-084XVideos of animal behavior are used to quantify researcher-defined behaviors of interest to study neural function, gene mutations, and pharmacological therapies. Behaviors of interest are often scored manually, which is time-consuming, limited to few behaviors, and variable across researchers. We created DeepEthogram: software that uses supervised machine learning to convert raw video pixels into an ethogram, the behaviors of interest present in each video frame. DeepEthogram is designed to be general-purpose and applicable across species, behaviors, and video-recording hardware. It uses convolutional neural networks to compute motion, extract features from motion and images, and classify features into behaviors. Behaviors are classified with above 90% accuracy on single frames in videos of mice and flies, matching expert-level human performance. DeepEthogram accurately predicts rare behaviors, requires little training data, and generalizes across subjects. A graphical interface allows beginning-to-end analysis without end-user programming. DeepEthogram’s rapid, automatic, and reproducible labeling of researcher-defined behaviors of interest may accelerate and enhance supervised behavior analysis. Code is available at: https://github.com/jbohnslav/deepethogram.James P BohnslavNivanthika K WimalasenaKelsey J ClausingYu Y DaiDavid A YarmolinskyTomás CruzAdam D KashlanM Eugenia ChiappeLauren L OreficeClifford J WoolfChristopher D HarveyeLife Sciences Publications Ltdarticlebehavior analysisdeep learningcomputer visionMedicineRScienceQBiology (General)QH301-705.5ENeLife, Vol 10 (2021) |
| institution |
DOAJ |
| collection |
DOAJ |
| language |
EN |
| topic |
behavior analysis deep learning computer vision Medicine R Science Q Biology (General) QH301-705.5 |
| spellingShingle |
behavior analysis deep learning computer vision Medicine R Science Q Biology (General) QH301-705.5 James P Bohnslav Nivanthika K Wimalasena Kelsey J Clausing Yu Y Dai David A Yarmolinsky Tomás Cruz Adam D Kashlan M Eugenia Chiappe Lauren L Orefice Clifford J Woolf Christopher D Harvey DeepEthogram, a machine learning pipeline for supervised behavior classification from raw pixels |
| description |
Videos of animal behavior are used to quantify researcher-defined behaviors of interest to study neural function, gene mutations, and pharmacological therapies. Behaviors of interest are often scored manually, which is time-consuming, limited to few behaviors, and variable across researchers. We created DeepEthogram: software that uses supervised machine learning to convert raw video pixels into an ethogram, the behaviors of interest present in each video frame. DeepEthogram is designed to be general-purpose and applicable across species, behaviors, and video-recording hardware. It uses convolutional neural networks to compute motion, extract features from motion and images, and classify features into behaviors. Behaviors are classified with above 90% accuracy on single frames in videos of mice and flies, matching expert-level human performance. DeepEthogram accurately predicts rare behaviors, requires little training data, and generalizes across subjects. A graphical interface allows beginning-to-end analysis without end-user programming. DeepEthogram’s rapid, automatic, and reproducible labeling of researcher-defined behaviors of interest may accelerate and enhance supervised behavior analysis. Code is available at: https://github.com/jbohnslav/deepethogram. |
| format |
article |
| author |
James P Bohnslav Nivanthika K Wimalasena Kelsey J Clausing Yu Y Dai David A Yarmolinsky Tomás Cruz Adam D Kashlan M Eugenia Chiappe Lauren L Orefice Clifford J Woolf Christopher D Harvey |
| author_facet |
James P Bohnslav Nivanthika K Wimalasena Kelsey J Clausing Yu Y Dai David A Yarmolinsky Tomás Cruz Adam D Kashlan M Eugenia Chiappe Lauren L Orefice Clifford J Woolf Christopher D Harvey |
| author_sort |
James P Bohnslav |
| title |
DeepEthogram, a machine learning pipeline for supervised behavior classification from raw pixels |
| title_short |
DeepEthogram, a machine learning pipeline for supervised behavior classification from raw pixels |
| title_full |
DeepEthogram, a machine learning pipeline for supervised behavior classification from raw pixels |
| title_fullStr |
DeepEthogram, a machine learning pipeline for supervised behavior classification from raw pixels |
| title_full_unstemmed |
DeepEthogram, a machine learning pipeline for supervised behavior classification from raw pixels |
| title_sort |
deepethogram, a machine learning pipeline for supervised behavior classification from raw pixels |
| publisher |
eLife Sciences Publications Ltd |
| publishDate |
2021 |
| url |
https://doaj.org/article/0490bc2a85ba4831a27480dbae549b72 |
| work_keys_str_mv |
AT jamespbohnslav deepethogramamachinelearningpipelineforsupervisedbehaviorclassificationfromrawpixels AT nivanthikakwimalasena deepethogramamachinelearningpipelineforsupervisedbehaviorclassificationfromrawpixels AT kelseyjclausing deepethogramamachinelearningpipelineforsupervisedbehaviorclassificationfromrawpixels AT yuydai deepethogramamachinelearningpipelineforsupervisedbehaviorclassificationfromrawpixels AT davidayarmolinsky deepethogramamachinelearningpipelineforsupervisedbehaviorclassificationfromrawpixels AT tomascruz deepethogramamachinelearningpipelineforsupervisedbehaviorclassificationfromrawpixels AT adamdkashlan deepethogramamachinelearningpipelineforsupervisedbehaviorclassificationfromrawpixels AT meugeniachiappe deepethogramamachinelearningpipelineforsupervisedbehaviorclassificationfromrawpixels AT laurenlorefice deepethogramamachinelearningpipelineforsupervisedbehaviorclassificationfromrawpixels AT cliffordjwoolf deepethogramamachinelearningpipelineforsupervisedbehaviorclassificationfromrawpixels AT christopherdharvey deepethogramamachinelearningpipelineforsupervisedbehaviorclassificationfromrawpixels |
| _version_ |
1718413359661973504 |