Emergence of Lie Symmetries in Functional Architectures Learned by CNNs

In this paper we study the spontaneous development of symmetries in the early layers of a Convolutional Neural Network (CNN) during learning on natural images. Our architecture is built in such a way to mimic some properties of the early stages of biological visual systems. In particular, it contain...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	Federico Bertoni, Noemi Montobbio, Alessandro Sarti, Giovanna Citti
Formato:	article
Lenguaje:	EN
Publicado:	Frontiers Media S.A. 2021
Materias:	lie symmetries CNN-convolutional neural network primary visual cortex (V1) lateral connection lateral geniculate sub-Riemannian geometries Neurosciences. Biological psychiatry. Neuropsychiatry RC321-571
Acceso en línea:	https://doaj.org/article/b4104313267940bdbb1857785f437314
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:b4104313267940bdbb1857785f437314
record_format	dspace
spelling	oai:doaj.org-article:b4104313267940bdbb1857785f4373142021-11-22T06:40:15ZEmergence of Lie Symmetries in Functional Architectures Learned by CNNs1662-518810.3389/fncom.2021.694505https://doaj.org/article/b4104313267940bdbb1857785f4373142021-11-01T00:00:00Zhttps://www.frontiersin.org/articles/10.3389/fncom.2021.694505/fullhttps://doaj.org/toc/1662-5188In this paper we study the spontaneous development of symmetries in the early layers of a Convolutional Neural Network (CNN) during learning on natural images. Our architecture is built in such a way to mimic some properties of the early stages of biological visual systems. In particular, it contains a pre-filtering step ℓ0 defined in analogy with the Lateral Geniculate Nucleus (LGN). Moreover, the first convolutional layer is equipped with lateral connections defined as a propagation driven by a learned connectivity kernel, in analogy with the horizontal connectivity of the primary visual cortex (V1). We first show that the ℓ0 filter evolves during the training to reach a radially symmetric pattern well approximated by a Laplacian of Gaussian (LoG), which is a well-known model of the receptive profiles of LGN cells. In line with previous works on CNNs, the learned convolutional filters in the first layer can be approximated by Gabor functions, in agreement with well-established models for the receptive profiles of V1 simple cells. Here, we focus on the geometric properties of the learned lateral connectivity kernel of this layer, showing the emergence of orientation selectivity w.r.t. the tuning of the learned filters. We also examine the short-range connectivity and association fields induced by this connectivity kernel, and show qualitative and quantitative comparisons with known group-based models of V1 horizontal connections. These geometric properties arise spontaneously during the training of the CNN architecture, analogously to the emergence of symmetries in visual systems thanks to brain plasticity driven by external stimuli.Federico BertoniFederico BertoniFederico BertoniNoemi MontobbioAlessandro SartiGiovanna CittiGiovanna CittiFrontiers Media S.A.articlelie symmetriesCNN-convolutional neural networkprimary visual cortex (V1)lateral connectionlateral geniculatesub-Riemannian geometriesNeurosciences. Biological psychiatry. NeuropsychiatryRC321-571ENFrontiers in Computational Neuroscience, Vol 15 (2021)
institution	DOAJ
collection	DOAJ
language	EN
topic	lie symmetries CNN-convolutional neural network primary visual cortex (V1) lateral connection lateral geniculate sub-Riemannian geometries Neurosciences. Biological psychiatry. Neuropsychiatry RC321-571
spellingShingle	lie symmetries CNN-convolutional neural network primary visual cortex (V1) lateral connection lateral geniculate sub-Riemannian geometries Neurosciences. Biological psychiatry. Neuropsychiatry RC321-571 Federico Bertoni Federico Bertoni Federico Bertoni Noemi Montobbio Alessandro Sarti Giovanna Citti Giovanna Citti Emergence of Lie Symmetries in Functional Architectures Learned by CNNs
description	In this paper we study the spontaneous development of symmetries in the early layers of a Convolutional Neural Network (CNN) during learning on natural images. Our architecture is built in such a way to mimic some properties of the early stages of biological visual systems. In particular, it contains a pre-filtering step ℓ0 defined in analogy with the Lateral Geniculate Nucleus (LGN). Moreover, the first convolutional layer is equipped with lateral connections defined as a propagation driven by a learned connectivity kernel, in analogy with the horizontal connectivity of the primary visual cortex (V1). We first show that the ℓ0 filter evolves during the training to reach a radially symmetric pattern well approximated by a Laplacian of Gaussian (LoG), which is a well-known model of the receptive profiles of LGN cells. In line with previous works on CNNs, the learned convolutional filters in the first layer can be approximated by Gabor functions, in agreement with well-established models for the receptive profiles of V1 simple cells. Here, we focus on the geometric properties of the learned lateral connectivity kernel of this layer, showing the emergence of orientation selectivity w.r.t. the tuning of the learned filters. We also examine the short-range connectivity and association fields induced by this connectivity kernel, and show qualitative and quantitative comparisons with known group-based models of V1 horizontal connections. These geometric properties arise spontaneously during the training of the CNN architecture, analogously to the emergence of symmetries in visual systems thanks to brain plasticity driven by external stimuli.
format	article
author	Federico Bertoni Federico Bertoni Federico Bertoni Noemi Montobbio Alessandro Sarti Giovanna Citti Giovanna Citti
author_facet	Federico Bertoni Federico Bertoni Federico Bertoni Noemi Montobbio Alessandro Sarti Giovanna Citti Giovanna Citti
author_sort	Federico Bertoni
title	Emergence of Lie Symmetries in Functional Architectures Learned by CNNs
title_short	Emergence of Lie Symmetries in Functional Architectures Learned by CNNs
title_full	Emergence of Lie Symmetries in Functional Architectures Learned by CNNs
title_fullStr	Emergence of Lie Symmetries in Functional Architectures Learned by CNNs
title_full_unstemmed	Emergence of Lie Symmetries in Functional Architectures Learned by CNNs
title_sort	emergence of lie symmetries in functional architectures learned by cnns
publisher	Frontiers Media S.A.
publishDate	2021
url	https://doaj.org/article/b4104313267940bdbb1857785f437314
work_keys_str_mv	AT federicobertoni emergenceofliesymmetriesinfunctionalarchitectureslearnedbycnns AT federicobertoni emergenceofliesymmetriesinfunctionalarchitectureslearnedbycnns AT federicobertoni emergenceofliesymmetriesinfunctionalarchitectureslearnedbycnns AT noemimontobbio emergenceofliesymmetriesinfunctionalarchitectureslearnedbycnns AT alessandrosarti emergenceofliesymmetriesinfunctionalarchitectureslearnedbycnns AT giovannacitti emergenceofliesymmetriesinfunctionalarchitectureslearnedbycnns AT giovannacitti emergenceofliesymmetriesinfunctionalarchitectureslearnedbycnns
_version_	1718418121884172288

Emergence of Lie Symmetries in Functional Architectures Learned by CNNs

Ejemplares similares