Combinatorial optimization by weight annealing in memristive hopfield networks

Abstract The increasing utility of specialized circuits and growing applications of optimization call for the development of efficient hardware accelerator for solving optimization problems. Hopfield neural network is a promising approach for solving combinatorial optimization problems due to the re...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	Z. Fahimi, M. R. Mahmoodi, H. Nili, Valentin Polishchuk, D. B. Strukov
Formato:	article
Lenguaje:	EN
Publicado:	Nature Portfolio 2021
Materias:	Medicine R Science Q
Acceso en línea:	https://doaj.org/article/a9912a0b21814b2e85547987c06591ba
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:a9912a0b21814b2e85547987c06591ba
record_format	dspace
spelling	oai:doaj.org-article:a9912a0b21814b2e85547987c06591ba2021-12-02T16:27:55ZCombinatorial optimization by weight annealing in memristive hopfield networks10.1038/s41598-020-78944-52045-2322https://doaj.org/article/a9912a0b21814b2e85547987c06591ba2021-08-01T00:00:00Zhttps://doi.org/10.1038/s41598-020-78944-5https://doaj.org/toc/2045-2322Abstract The increasing utility of specialized circuits and growing applications of optimization call for the development of efficient hardware accelerator for solving optimization problems. Hopfield neural network is a promising approach for solving combinatorial optimization problems due to the recent demonstrations of efficient mixed-signal implementation based on emerging non-volatile memory devices. Such mixed-signal accelerators also enable very efficient implementation of various annealing techniques, which are essential for finding optimal solutions. Here we propose a “weight annealing” approach, whose main idea is to ease convergence to the global minima by keeping the network close to its ground state. This is achieved by initially setting all synaptic weights to zero, thus ensuring a quick transition of the Hopfield network to its trivial global minima state and then gradually introducing weights during the annealing process. The extensive numerical simulations show that our approach leads to a better, on average, solutions for several representative combinatorial problems compared to prior Hopfield neural network solvers with chaotic or stochastic annealing. As a proof of concept, a 13-node graph partitioning problem and a 7-node maximum-weight independent set problem are solved experimentally using mixed-signal circuits based on, correspondingly, a 20 × 20 analog-grade TiO2 memristive crossbar and a 12 × 10 eFlash memory array.Z. FahimiM. R. MahmoodiH. NiliValentin PolishchukD. B. StrukovNature PortfolioarticleMedicineRScienceQENScientific Reports, Vol 11, Iss 1, Pp 1-10 (2021)
institution	DOAJ
collection	DOAJ
language	EN
topic	Medicine R Science Q
spellingShingle	Medicine R Science Q Z. Fahimi M. R. Mahmoodi H. Nili Valentin Polishchuk D. B. Strukov Combinatorial optimization by weight annealing in memristive hopfield networks
description	Abstract The increasing utility of specialized circuits and growing applications of optimization call for the development of efficient hardware accelerator for solving optimization problems. Hopfield neural network is a promising approach for solving combinatorial optimization problems due to the recent demonstrations of efficient mixed-signal implementation based on emerging non-volatile memory devices. Such mixed-signal accelerators also enable very efficient implementation of various annealing techniques, which are essential for finding optimal solutions. Here we propose a “weight annealing” approach, whose main idea is to ease convergence to the global minima by keeping the network close to its ground state. This is achieved by initially setting all synaptic weights to zero, thus ensuring a quick transition of the Hopfield network to its trivial global minima state and then gradually introducing weights during the annealing process. The extensive numerical simulations show that our approach leads to a better, on average, solutions for several representative combinatorial problems compared to prior Hopfield neural network solvers with chaotic or stochastic annealing. As a proof of concept, a 13-node graph partitioning problem and a 7-node maximum-weight independent set problem are solved experimentally using mixed-signal circuits based on, correspondingly, a 20 × 20 analog-grade TiO2 memristive crossbar and a 12 × 10 eFlash memory array.
format	article
author	Z. Fahimi M. R. Mahmoodi H. Nili Valentin Polishchuk D. B. Strukov
author_facet	Z. Fahimi M. R. Mahmoodi H. Nili Valentin Polishchuk D. B. Strukov
author_sort	Z. Fahimi
title	Combinatorial optimization by weight annealing in memristive hopfield networks
title_short	Combinatorial optimization by weight annealing in memristive hopfield networks
title_full	Combinatorial optimization by weight annealing in memristive hopfield networks
title_fullStr	Combinatorial optimization by weight annealing in memristive hopfield networks
title_full_unstemmed	Combinatorial optimization by weight annealing in memristive hopfield networks
title_sort	combinatorial optimization by weight annealing in memristive hopfield networks
publisher	Nature Portfolio
publishDate	2021
url	https://doaj.org/article/a9912a0b21814b2e85547987c06591ba
work_keys_str_mv	AT zfahimi combinatorialoptimizationbyweightannealinginmemristivehopfieldnetworks AT mrmahmoodi combinatorialoptimizationbyweightannealinginmemristivehopfieldnetworks AT hnili combinatorialoptimizationbyweightannealinginmemristivehopfieldnetworks AT valentinpolishchuk combinatorialoptimizationbyweightannealinginmemristivehopfieldnetworks AT dbstrukov combinatorialoptimizationbyweightannealinginmemristivehopfieldnetworks
_version_	1718384009911730176

Combinatorial optimization by weight annealing in memristive hopfield networks

Ejemplares similares