Control of neural systems at multiple scales using model-free, deep reinforcement learning

Abstract Recent improvements in hardware and data collection have lowered the barrier to practical neural control. Most of the current contributions to the field have focus on model-based control, however, models of neural systems are quite complex and difficult to design. To circumvent these issues...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	B. A. Mitchell, L. R. Petzold
Formato:	article
Lenguaje:	EN
Publicado:	Nature Portfolio 2018
Materias:	Medicine R Science Q
Acceso en línea:	https://doaj.org/article/912bcbee82b946038c3664f0651695fa
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:912bcbee82b946038c3664f0651695fa
record_format	dspace
spelling	oai:doaj.org-article:912bcbee82b946038c3664f0651695fa2021-12-02T12:32:35ZControl of neural systems at multiple scales using model-free, deep reinforcement learning10.1038/s41598-018-29134-x2045-2322https://doaj.org/article/912bcbee82b946038c3664f0651695fa2018-07-01T00:00:00Zhttps://doi.org/10.1038/s41598-018-29134-xhttps://doaj.org/toc/2045-2322Abstract Recent improvements in hardware and data collection have lowered the barrier to practical neural control. Most of the current contributions to the field have focus on model-based control, however, models of neural systems are quite complex and difficult to design. To circumvent these issues, we adapt a model-free method from the reinforcement learning literature, Deep Deterministic Policy Gradients (DDPG). Model-free reinforcement learning presents an attractive framework because of the flexibility it offers, allowing the user to avoid modeling system dynamics. We make use of this feature by applying DDPG to models of low-level and high-level neural dynamics. We show that while model-free, DDPG is able to solve more difficult problems than can be solved by current methods. These problems include the induction of global synchrony by entrainment of weakly coupled oscillators and the control of trajectories through a latent phase space of an underactuated network of neurons. While this work has been performed on simulated systems, it suggests that advances in modern reinforcement learning may enable the solution of fundamental problems in neural control and movement towards more complex objectives in real systems.B. A. MitchellL. R. PetzoldNature PortfolioarticleMedicineRScienceQENScientific Reports, Vol 8, Iss 1, Pp 1-12 (2018)
institution	DOAJ
collection	DOAJ
language	EN
topic	Medicine R Science Q
spellingShingle	Medicine R Science Q B. A. Mitchell L. R. Petzold Control of neural systems at multiple scales using model-free, deep reinforcement learning
description	Abstract Recent improvements in hardware and data collection have lowered the barrier to practical neural control. Most of the current contributions to the field have focus on model-based control, however, models of neural systems are quite complex and difficult to design. To circumvent these issues, we adapt a model-free method from the reinforcement learning literature, Deep Deterministic Policy Gradients (DDPG). Model-free reinforcement learning presents an attractive framework because of the flexibility it offers, allowing the user to avoid modeling system dynamics. We make use of this feature by applying DDPG to models of low-level and high-level neural dynamics. We show that while model-free, DDPG is able to solve more difficult problems than can be solved by current methods. These problems include the induction of global synchrony by entrainment of weakly coupled oscillators and the control of trajectories through a latent phase space of an underactuated network of neurons. While this work has been performed on simulated systems, it suggests that advances in modern reinforcement learning may enable the solution of fundamental problems in neural control and movement towards more complex objectives in real systems.
format	article
author	B. A. Mitchell L. R. Petzold
author_facet	B. A. Mitchell L. R. Petzold
author_sort	B. A. Mitchell
title	Control of neural systems at multiple scales using model-free, deep reinforcement learning
title_short	Control of neural systems at multiple scales using model-free, deep reinforcement learning
title_full	Control of neural systems at multiple scales using model-free, deep reinforcement learning
title_fullStr	Control of neural systems at multiple scales using model-free, deep reinforcement learning
title_full_unstemmed	Control of neural systems at multiple scales using model-free, deep reinforcement learning
title_sort	control of neural systems at multiple scales using model-free, deep reinforcement learning
publisher	Nature Portfolio
publishDate	2018
url	https://doaj.org/article/912bcbee82b946038c3664f0651695fa
work_keys_str_mv	AT bamitchell controlofneuralsystemsatmultiplescalesusingmodelfreedeepreinforcementlearning AT lrpetzold controlofneuralsystemsatmultiplescalesusingmodelfreedeepreinforcementlearning
_version_	1718394048285245440

Control of neural systems at multiple scales using model-free, deep reinforcement learning

Ejemplares similares