Democratic population decisions result in robust policy-gradient learning: a parametric study with GPU simulations.

High performance computing on the Graphics Processing Unit (GPU) is an emerging field driven by the promise of high computational power at a low cost. However, GPU programming is a non-trivial task and moreover architectural limitations raise the question of whether investing effort in this directio...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	Paul Richmond, Lars Buesing, Michele Giugliano, Eleni Vasilaki
Formato:	article
Lenguaje:	EN
Publicado:	Public Library of Science (PLoS) 2011
Materias:	Medicine R Science Q
Acceso en línea:	https://doaj.org/article/78c4d4314a024d3c8c5eafcccbd88530
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:78c4d4314a024d3c8c5eafcccbd88530
record_format	dspace
spelling	oai:doaj.org-article:78c4d4314a024d3c8c5eafcccbd885302021-11-18T06:54:31ZDemocratic population decisions result in robust policy-gradient learning: a parametric study with GPU simulations.1932-620310.1371/journal.pone.0018539https://doaj.org/article/78c4d4314a024d3c8c5eafcccbd885302011-05-01T00:00:00Zhttps://www.ncbi.nlm.nih.gov/pmc/articles/pmid/21572529/?tool=EBIhttps://doaj.org/toc/1932-6203High performance computing on the Graphics Processing Unit (GPU) is an emerging field driven by the promise of high computational power at a low cost. However, GPU programming is a non-trivial task and moreover architectural limitations raise the question of whether investing effort in this direction may be worthwhile. In this work, we use GPU programming to simulate a two-layer network of Integrate-and-Fire neurons with varying degrees of recurrent connectivity and investigate its ability to learn a simplified navigation task using a policy-gradient learning rule stemming from Reinforcement Learning. The purpose of this paper is twofold. First, we want to support the use of GPUs in the field of Computational Neuroscience. Second, using GPU computing power, we investigate the conditions under which the said architecture and learning rule demonstrate best performance. Our work indicates that networks featuring strong Mexican-Hat-shaped recurrent connections in the top layer, where decision making is governed by the formation of a stable activity bump in the neural population (a "non-democratic" mechanism), achieve mediocre learning results at best. In absence of recurrent connections, where all neurons "vote" independently ("democratic") for a decision via population vector readout, the task is generally learned better and more robustly. Our study would have been extremely difficult on a desktop computer without the use of GPU programming. We present the routines developed for this purpose and show that a speed improvement of 5x up to 42x is provided versus optimised Python code. The higher speed is achieved when we exploit the parallelism of the GPU in the search of learning parameters. This suggests that efficient GPU programming can significantly reduce the time needed for simulating networks of spiking neurons, particularly when multiple parameter configurations are investigated.Paul RichmondLars BuesingMichele GiuglianoEleni VasilakiPublic Library of Science (PLoS)articleMedicineRScienceQENPLoS ONE, Vol 6, Iss 5, p e18539 (2011)
institution	DOAJ
collection	DOAJ
language	EN
topic	Medicine R Science Q
spellingShingle	Medicine R Science Q Paul Richmond Lars Buesing Michele Giugliano Eleni Vasilaki Democratic population decisions result in robust policy-gradient learning: a parametric study with GPU simulations.
description	High performance computing on the Graphics Processing Unit (GPU) is an emerging field driven by the promise of high computational power at a low cost. However, GPU programming is a non-trivial task and moreover architectural limitations raise the question of whether investing effort in this direction may be worthwhile. In this work, we use GPU programming to simulate a two-layer network of Integrate-and-Fire neurons with varying degrees of recurrent connectivity and investigate its ability to learn a simplified navigation task using a policy-gradient learning rule stemming from Reinforcement Learning. The purpose of this paper is twofold. First, we want to support the use of GPUs in the field of Computational Neuroscience. Second, using GPU computing power, we investigate the conditions under which the said architecture and learning rule demonstrate best performance. Our work indicates that networks featuring strong Mexican-Hat-shaped recurrent connections in the top layer, where decision making is governed by the formation of a stable activity bump in the neural population (a "non-democratic" mechanism), achieve mediocre learning results at best. In absence of recurrent connections, where all neurons "vote" independently ("democratic") for a decision via population vector readout, the task is generally learned better and more robustly. Our study would have been extremely difficult on a desktop computer without the use of GPU programming. We present the routines developed for this purpose and show that a speed improvement of 5x up to 42x is provided versus optimised Python code. The higher speed is achieved when we exploit the parallelism of the GPU in the search of learning parameters. This suggests that efficient GPU programming can significantly reduce the time needed for simulating networks of spiking neurons, particularly when multiple parameter configurations are investigated.
format	article
author	Paul Richmond Lars Buesing Michele Giugliano Eleni Vasilaki
author_facet	Paul Richmond Lars Buesing Michele Giugliano Eleni Vasilaki
author_sort	Paul Richmond
title	Democratic population decisions result in robust policy-gradient learning: a parametric study with GPU simulations.
title_short	Democratic population decisions result in robust policy-gradient learning: a parametric study with GPU simulations.
title_full	Democratic population decisions result in robust policy-gradient learning: a parametric study with GPU simulations.
title_fullStr	Democratic population decisions result in robust policy-gradient learning: a parametric study with GPU simulations.
title_full_unstemmed	Democratic population decisions result in robust policy-gradient learning: a parametric study with GPU simulations.
title_sort	democratic population decisions result in robust policy-gradient learning: a parametric study with gpu simulations.
publisher	Public Library of Science (PLoS)
publishDate	2011
url	https://doaj.org/article/78c4d4314a024d3c8c5eafcccbd88530
work_keys_str_mv	AT paulrichmond democraticpopulationdecisionsresultinrobustpolicygradientlearningaparametricstudywithgpusimulations AT larsbuesing democraticpopulationdecisionsresultinrobustpolicygradientlearningaparametricstudywithgpusimulations AT michelegiugliano democraticpopulationdecisionsresultinrobustpolicygradientlearningaparametricstudywithgpusimulations AT elenivasilaki democraticpopulationdecisionsresultinrobustpolicygradientlearningaparametricstudywithgpusimulations
_version_	1718424237190938624

Democratic population decisions result in robust policy-gradient learning: a parametric study with GPU simulations.

Ejemplares similares