Context-dependent extinction learning emerging from raw sensory inputs: a reinforcement learning approach

Abstract The context-dependence of extinction learning has been well studied and requires the hippocampus. However, the underlying neural mechanisms are still poorly understood. Using memory-driven reinforcement learning and deep neural networks, we developed a model that learns to navigate autonomo...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	Thomas Walther, Nicolas Diekmann, Sandhiya Vijayabaskaran, José R. Donoso, Denise Manahan-Vaughan, Laurenz Wiskott, Sen Cheng
Formato:	article
Lenguaje:	EN
Publicado:	Nature Portfolio 2021
Materias:	Medicine R Science Q
Acceso en línea:	https://doaj.org/article/45e2a707d2c04f75a6f4100661e4bbbe
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:45e2a707d2c04f75a6f4100661e4bbbe
record_format	dspace
spelling	oai:doaj.org-article:45e2a707d2c04f75a6f4100661e4bbbe2021-12-02T10:44:16ZContext-dependent extinction learning emerging from raw sensory inputs: a reinforcement learning approach10.1038/s41598-021-81157-z2045-2322https://doaj.org/article/45e2a707d2c04f75a6f4100661e4bbbe2021-02-01T00:00:00Zhttps://doi.org/10.1038/s41598-021-81157-zhttps://doaj.org/toc/2045-2322Abstract The context-dependence of extinction learning has been well studied and requires the hippocampus. However, the underlying neural mechanisms are still poorly understood. Using memory-driven reinforcement learning and deep neural networks, we developed a model that learns to navigate autonomously in biologically realistic virtual reality environments based on raw camera inputs alone. Neither is context represented explicitly in our model, nor is context change signaled. We find that memory-intact agents learn distinct context representations, and develop ABA renewal, whereas memory-impaired agents do not. These findings reproduce the behavior of control and hippocampal animals, respectively. We therefore propose that the role of the hippocampus in the context-dependence of extinction learning might stem from its function in episodic-like memory and not in context-representation per se. We conclude that context-dependence can emerge from raw visual inputs.Thomas WaltherNicolas DiekmannSandhiya VijayabaskaranJosé R. DonosoDenise Manahan-VaughanLaurenz WiskottSen ChengNature PortfolioarticleMedicineRScienceQENScientific Reports, Vol 11, Iss 1, Pp 1-14 (2021)
institution	DOAJ
collection	DOAJ
language	EN
topic	Medicine R Science Q
spellingShingle	Medicine R Science Q Thomas Walther Nicolas Diekmann Sandhiya Vijayabaskaran José R. Donoso Denise Manahan-Vaughan Laurenz Wiskott Sen Cheng Context-dependent extinction learning emerging from raw sensory inputs: a reinforcement learning approach
description	Abstract The context-dependence of extinction learning has been well studied and requires the hippocampus. However, the underlying neural mechanisms are still poorly understood. Using memory-driven reinforcement learning and deep neural networks, we developed a model that learns to navigate autonomously in biologically realistic virtual reality environments based on raw camera inputs alone. Neither is context represented explicitly in our model, nor is context change signaled. We find that memory-intact agents learn distinct context representations, and develop ABA renewal, whereas memory-impaired agents do not. These findings reproduce the behavior of control and hippocampal animals, respectively. We therefore propose that the role of the hippocampus in the context-dependence of extinction learning might stem from its function in episodic-like memory and not in context-representation per se. We conclude that context-dependence can emerge from raw visual inputs.
format	article
author	Thomas Walther Nicolas Diekmann Sandhiya Vijayabaskaran José R. Donoso Denise Manahan-Vaughan Laurenz Wiskott Sen Cheng
author_facet	Thomas Walther Nicolas Diekmann Sandhiya Vijayabaskaran José R. Donoso Denise Manahan-Vaughan Laurenz Wiskott Sen Cheng
author_sort	Thomas Walther
title	Context-dependent extinction learning emerging from raw sensory inputs: a reinforcement learning approach
title_short	Context-dependent extinction learning emerging from raw sensory inputs: a reinforcement learning approach
title_full	Context-dependent extinction learning emerging from raw sensory inputs: a reinforcement learning approach
title_fullStr	Context-dependent extinction learning emerging from raw sensory inputs: a reinforcement learning approach
title_full_unstemmed	Context-dependent extinction learning emerging from raw sensory inputs: a reinforcement learning approach
title_sort	context-dependent extinction learning emerging from raw sensory inputs: a reinforcement learning approach
publisher	Nature Portfolio
publishDate	2021
url	https://doaj.org/article/45e2a707d2c04f75a6f4100661e4bbbe
work_keys_str_mv	AT thomaswalther contextdependentextinctionlearningemergingfromrawsensoryinputsareinforcementlearningapproach AT nicolasdiekmann contextdependentextinctionlearningemergingfromrawsensoryinputsareinforcementlearningapproach AT sandhiyavijayabaskaran contextdependentextinctionlearningemergingfromrawsensoryinputsareinforcementlearningapproach AT joserdonoso contextdependentextinctionlearningemergingfromrawsensoryinputsareinforcementlearningapproach AT denisemanahanvaughan contextdependentextinctionlearningemergingfromrawsensoryinputsareinforcementlearningapproach AT laurenzwiskott contextdependentextinctionlearningemergingfromrawsensoryinputsareinforcementlearningapproach AT sencheng contextdependentextinctionlearningemergingfromrawsensoryinputsareinforcementlearningapproach
_version_	1718396759112155136

Context-dependent extinction learning emerging from raw sensory inputs: a reinforcement learning approach

Ejemplares similares