Combining Accuracy and Plasticity in Convolutional Neural Networks Based on Resistive Memory Arrays for Autonomous Learning

Nowadays, artificial neural networks (ANNs) can outperform the human brain’s ability in specific tasks. However, ANNs cannot replicate the efficient and low-power learning, adaptation, and consolidation typical of biological organisms. Here, we present a hardware design based on arrays of...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	Stefano Bianchi, Irene Munoz-Martin, Erika Covi, Alessandro Bricalli, Giuseppe Piccoloni, Amir Regev, Gabriel Molas, Jean Francois Nodin, Francois Andrieu, Daniele Ielmini
Formato:	article
Lenguaje:	EN
Publicado:	IEEE 2021
Materias:	Catastrophic forgetting complementary learning systems continual learning convolutional neural networks (CNNs) resistive switching random-access memory (RRAM) spike-timing-dependent plasticity (STDP) Computer engineering. Computer hardware TK7885-7895
Acceso en línea:	https://doaj.org/article/37b5a4a1ad6f48178c41617d2bac5f0f
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:37b5a4a1ad6f48178c41617d2bac5f0f
record_format	dspace
spelling	oai:doaj.org-article:37b5a4a1ad6f48178c41617d2bac5f0f2021-11-18T00:11:25ZCombining Accuracy and Plasticity in Convolutional Neural Networks Based on Resistive Memory Arrays for Autonomous Learning2329-923110.1109/JXCDC.2021.3118061https://doaj.org/article/37b5a4a1ad6f48178c41617d2bac5f0f2021-01-01T00:00:00Zhttps://ieeexplore.ieee.org/document/9565857/https://doaj.org/toc/2329-9231Nowadays, artificial neural networks (ANNs) can outperform the human brain’s ability in specific tasks. However, ANNs cannot replicate the efficient and low-power learning, adaptation, and consolidation typical of biological organisms. Here, we present a hardware design based on arrays of SiO<sub><italic>x</italic></sub> resistive switching random-access memory (RRAM), which allows combining the accuracy of convolutional neural networks with the flexibility of bio-inspired neuronal plasticity. In order to enable the combination of the stable and the plastic attributes of the network, we exploit the spike-frequency adaptation of the neurons relying on the multilevel programming of the RRAM devices. This procedure enhances the efficiency and accuracy of the network for MNIST, noisy MNIST (N-MNIST), Fashion-MNIST, and CIFAR-10 datasets, with inference accuracies of about 99%–89%, respectively. We also demonstrate that the hardware is capable of asynchronous self-adaptation of its operative frequency according to the fire rate of the spiking neuron, thus optimizing the whole behavior of the network. We finally show that the system enables fast and accurate filter retraining to overcome catastrophic forgetting, showing high efficiency in terms of operations per second and robustness against device non-idealities. This work paves the way for the theoretical modeling and hardware realization of resilient autonomous systems in dynamic environments.Stefano BianchiIrene Munoz-MartinErika CoviAlessandro BricalliGiuseppe PiccoloniAmir RegevGabriel MolasJean Francois NodinFrancois AndrieuDaniele IelminiIEEEarticleCatastrophic forgettingcomplementary learning systemscontinual learningconvolutional neural networks (CNNs)resistive switching random-access memory (RRAM)spike-timing-dependent plasticity (STDP)Computer engineering. Computer hardwareTK7885-7895ENIEEE Journal on Exploratory Solid-State Computational Devices and Circuits, Vol 7, Iss 2, Pp 132-140 (2021)
institution	DOAJ
collection	DOAJ
language	EN
topic	Catastrophic forgetting complementary learning systems continual learning convolutional neural networks (CNNs) resistive switching random-access memory (RRAM) spike-timing-dependent plasticity (STDP) Computer engineering. Computer hardware TK7885-7895
spellingShingle	Catastrophic forgetting complementary learning systems continual learning convolutional neural networks (CNNs) resistive switching random-access memory (RRAM) spike-timing-dependent plasticity (STDP) Computer engineering. Computer hardware TK7885-7895 Stefano Bianchi Irene Munoz-Martin Erika Covi Alessandro Bricalli Giuseppe Piccoloni Amir Regev Gabriel Molas Jean Francois Nodin Francois Andrieu Daniele Ielmini Combining Accuracy and Plasticity in Convolutional Neural Networks Based on Resistive Memory Arrays for Autonomous Learning
description	Nowadays, artificial neural networks (ANNs) can outperform the human brain’s ability in specific tasks. However, ANNs cannot replicate the efficient and low-power learning, adaptation, and consolidation typical of biological organisms. Here, we present a hardware design based on arrays of SiO<sub><italic>x</italic></sub> resistive switching random-access memory (RRAM), which allows combining the accuracy of convolutional neural networks with the flexibility of bio-inspired neuronal plasticity. In order to enable the combination of the stable and the plastic attributes of the network, we exploit the spike-frequency adaptation of the neurons relying on the multilevel programming of the RRAM devices. This procedure enhances the efficiency and accuracy of the network for MNIST, noisy MNIST (N-MNIST), Fashion-MNIST, and CIFAR-10 datasets, with inference accuracies of about 99%–89%, respectively. We also demonstrate that the hardware is capable of asynchronous self-adaptation of its operative frequency according to the fire rate of the spiking neuron, thus optimizing the whole behavior of the network. We finally show that the system enables fast and accurate filter retraining to overcome catastrophic forgetting, showing high efficiency in terms of operations per second and robustness against device non-idealities. This work paves the way for the theoretical modeling and hardware realization of resilient autonomous systems in dynamic environments.
format	article
author	Stefano Bianchi Irene Munoz-Martin Erika Covi Alessandro Bricalli Giuseppe Piccoloni Amir Regev Gabriel Molas Jean Francois Nodin Francois Andrieu Daniele Ielmini
author_facet	Stefano Bianchi Irene Munoz-Martin Erika Covi Alessandro Bricalli Giuseppe Piccoloni Amir Regev Gabriel Molas Jean Francois Nodin Francois Andrieu Daniele Ielmini
author_sort	Stefano Bianchi
title	Combining Accuracy and Plasticity in Convolutional Neural Networks Based on Resistive Memory Arrays for Autonomous Learning
title_short	Combining Accuracy and Plasticity in Convolutional Neural Networks Based on Resistive Memory Arrays for Autonomous Learning
title_full	Combining Accuracy and Plasticity in Convolutional Neural Networks Based on Resistive Memory Arrays for Autonomous Learning
title_fullStr	Combining Accuracy and Plasticity in Convolutional Neural Networks Based on Resistive Memory Arrays for Autonomous Learning
title_full_unstemmed	Combining Accuracy and Plasticity in Convolutional Neural Networks Based on Resistive Memory Arrays for Autonomous Learning
title_sort	combining accuracy and plasticity in convolutional neural networks based on resistive memory arrays for autonomous learning
publisher	IEEE
publishDate	2021
url	https://doaj.org/article/37b5a4a1ad6f48178c41617d2bac5f0f
work_keys_str_mv	AT stefanobianchi combiningaccuracyandplasticityinconvolutionalneuralnetworksbasedonresistivememoryarraysforautonomouslearning AT irenemunozmartin combiningaccuracyandplasticityinconvolutionalneuralnetworksbasedonresistivememoryarraysforautonomouslearning AT erikacovi combiningaccuracyandplasticityinconvolutionalneuralnetworksbasedonresistivememoryarraysforautonomouslearning AT alessandrobricalli combiningaccuracyandplasticityinconvolutionalneuralnetworksbasedonresistivememoryarraysforautonomouslearning AT giuseppepiccoloni combiningaccuracyandplasticityinconvolutionalneuralnetworksbasedonresistivememoryarraysforautonomouslearning AT amirregev combiningaccuracyandplasticityinconvolutionalneuralnetworksbasedonresistivememoryarraysforautonomouslearning AT gabrielmolas combiningaccuracyandplasticityinconvolutionalneuralnetworksbasedonresistivememoryarraysforautonomouslearning AT jeanfrancoisnodin combiningaccuracyandplasticityinconvolutionalneuralnetworksbasedonresistivememoryarraysforautonomouslearning AT francoisandrieu combiningaccuracyandplasticityinconvolutionalneuralnetworksbasedonresistivememoryarraysforautonomouslearning AT danieleielmini combiningaccuracyandplasticityinconvolutionalneuralnetworksbasedonresistivememoryarraysforautonomouslearning
_version_	1718425155628171264

Combining Accuracy and Plasticity in Convolutional Neural Networks Based on Resistive Memory Arrays for Autonomous Learning

Ejemplares similares