Mode-assisted joint training of deep Boltzmann machines

Abstract The deep extension of the restricted Boltzmann machine (RBM), known as the deep Boltzmann machine (DBM), is an expressive family of machine learning models which can serve as compact representations of complex probability distributions. However, jointly training DBMs in the unsupervised set...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	Haik Manukian, Massimiliano Di Ventra
Formato:	article
Lenguaje:	EN
Publicado:	Nature Portfolio 2021
Materias:	Medicine R Science Q
Acceso en línea:	https://doaj.org/article/95741509c9624ceab2c63fcbc316be4c
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:95741509c9624ceab2c63fcbc316be4c
record_format	dspace
spelling	oai:doaj.org-article:95741509c9624ceab2c63fcbc316be4c2021-12-02T18:48:09ZMode-assisted joint training of deep Boltzmann machines10.1038/s41598-021-98404-y2045-2322https://doaj.org/article/95741509c9624ceab2c63fcbc316be4c2021-09-01T00:00:00Zhttps://doi.org/10.1038/s41598-021-98404-yhttps://doaj.org/toc/2045-2322Abstract The deep extension of the restricted Boltzmann machine (RBM), known as the deep Boltzmann machine (DBM), is an expressive family of machine learning models which can serve as compact representations of complex probability distributions. However, jointly training DBMs in the unsupervised setting has proven to be a formidable task. A recent technique we have proposed, called mode-assisted training, has shown great success in improving the unsupervised training of RBMs. Here, we show that the performance gains of the mode-assisted training are even more dramatic for DBMs. In fact, DBMs jointly trained with the mode-assisted algorithm can represent the same data set with orders of magnitude lower number of total parameters compared to state-of-the-art training procedures and even with respect to RBMs, provided a fan-in network topology is also introduced. This substantial saving in number of parameters makes this training method very appealing also for hardware implementations.Haik ManukianMassimiliano Di VentraNature PortfolioarticleMedicineRScienceQENScientific Reports, Vol 11, Iss 1, Pp 1-8 (2021)
institution	DOAJ
collection	DOAJ
language	EN
topic	Medicine R Science Q
spellingShingle	Medicine R Science Q Haik Manukian Massimiliano Di Ventra Mode-assisted joint training of deep Boltzmann machines
description	Abstract The deep extension of the restricted Boltzmann machine (RBM), known as the deep Boltzmann machine (DBM), is an expressive family of machine learning models which can serve as compact representations of complex probability distributions. However, jointly training DBMs in the unsupervised setting has proven to be a formidable task. A recent technique we have proposed, called mode-assisted training, has shown great success in improving the unsupervised training of RBMs. Here, we show that the performance gains of the mode-assisted training are even more dramatic for DBMs. In fact, DBMs jointly trained with the mode-assisted algorithm can represent the same data set with orders of magnitude lower number of total parameters compared to state-of-the-art training procedures and even with respect to RBMs, provided a fan-in network topology is also introduced. This substantial saving in number of parameters makes this training method very appealing also for hardware implementations.
format	article
author	Haik Manukian Massimiliano Di Ventra
author_facet	Haik Manukian Massimiliano Di Ventra
author_sort	Haik Manukian
title	Mode-assisted joint training of deep Boltzmann machines
title_short	Mode-assisted joint training of deep Boltzmann machines
title_full	Mode-assisted joint training of deep Boltzmann machines
title_fullStr	Mode-assisted joint training of deep Boltzmann machines
title_full_unstemmed	Mode-assisted joint training of deep Boltzmann machines
title_sort	mode-assisted joint training of deep boltzmann machines
publisher	Nature Portfolio
publishDate	2021
url	https://doaj.org/article/95741509c9624ceab2c63fcbc316be4c
work_keys_str_mv	AT haikmanukian modeassistedjointtrainingofdeepboltzmannmachines AT massimilianodiventra modeassistedjointtrainingofdeepboltzmannmachines
_version_	1718377638688456704

Mode-assisted joint training of deep Boltzmann machines

Ejemplares similares