DropConnect is effective in modeling uncertainty of Bayesian deep networks

Abstract Deep neural networks (DNNs) have achieved state-of-the-art performance in many important domains, including medical diagnosis, security, and autonomous driving. In domains where safety is highly critical, an erroneous decision can result in serious consequences. While a perfect prediction a...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	Aryan Mobiny, Pengyu Yuan, Supratik K. Moulik, Naveen Garg, Carol C. Wu, Hien Van Nguyen
Formato:	article
Lenguaje:	EN
Publicado:	Nature Portfolio 2021
Materias:	Medicine R Science Q
Acceso en línea:	https://doaj.org/article/ff6a04b4a5f649c98b73aae38d3ec8c3
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:ff6a04b4a5f649c98b73aae38d3ec8c3
record_format	dspace
spelling	oai:doaj.org-article:ff6a04b4a5f649c98b73aae38d3ec8c32021-12-02T15:53:46ZDropConnect is effective in modeling uncertainty of Bayesian deep networks10.1038/s41598-021-84854-x2045-2322https://doaj.org/article/ff6a04b4a5f649c98b73aae38d3ec8c32021-03-01T00:00:00Zhttps://doi.org/10.1038/s41598-021-84854-xhttps://doaj.org/toc/2045-2322Abstract Deep neural networks (DNNs) have achieved state-of-the-art performance in many important domains, including medical diagnosis, security, and autonomous driving. In domains where safety is highly critical, an erroneous decision can result in serious consequences. While a perfect prediction accuracy is not always achievable, recent work on Bayesian deep networks shows that it is possible to know when DNNs are more likely to make mistakes. Knowing what DNNs do not know is desirable to increase the safety of deep learning technology in sensitive applications; Bayesian neural networks attempt to address this challenge. Traditional approaches are computationally intractable and do not scale well to large, complex neural network architectures. In this paper, we develop a theoretical framework to approximate Bayesian inference for DNNs by imposing a Bernoulli distribution on the model weights. This method called Monte Carlo DropConnect (MC-DropConnect) gives us a tool to represent the model uncertainty with little change in the overall model structure or computational cost. We extensively validate the proposed algorithm on multiple network architectures and datasets for classification and semantic segmentation tasks. We also propose new metrics to quantify uncertainty estimates. This enables an objective comparison between MC-DropConnect and prior approaches. Our empirical results demonstrate that the proposed framework yields significant improvement in both prediction accuracy and uncertainty estimation quality compared to the state of the art.Aryan MobinyPengyu YuanSupratik K. MoulikNaveen GargCarol C. WuHien Van NguyenNature PortfolioarticleMedicineRScienceQENScientific Reports, Vol 11, Iss 1, Pp 1-14 (2021)
institution	DOAJ
collection	DOAJ
language	EN
topic	Medicine R Science Q
spellingShingle	Medicine R Science Q Aryan Mobiny Pengyu Yuan Supratik K. Moulik Naveen Garg Carol C. Wu Hien Van Nguyen DropConnect is effective in modeling uncertainty of Bayesian deep networks
description	Abstract Deep neural networks (DNNs) have achieved state-of-the-art performance in many important domains, including medical diagnosis, security, and autonomous driving. In domains where safety is highly critical, an erroneous decision can result in serious consequences. While a perfect prediction accuracy is not always achievable, recent work on Bayesian deep networks shows that it is possible to know when DNNs are more likely to make mistakes. Knowing what DNNs do not know is desirable to increase the safety of deep learning technology in sensitive applications; Bayesian neural networks attempt to address this challenge. Traditional approaches are computationally intractable and do not scale well to large, complex neural network architectures. In this paper, we develop a theoretical framework to approximate Bayesian inference for DNNs by imposing a Bernoulli distribution on the model weights. This method called Monte Carlo DropConnect (MC-DropConnect) gives us a tool to represent the model uncertainty with little change in the overall model structure or computational cost. We extensively validate the proposed algorithm on multiple network architectures and datasets for classification and semantic segmentation tasks. We also propose new metrics to quantify uncertainty estimates. This enables an objective comparison between MC-DropConnect and prior approaches. Our empirical results demonstrate that the proposed framework yields significant improvement in both prediction accuracy and uncertainty estimation quality compared to the state of the art.
format	article
author	Aryan Mobiny Pengyu Yuan Supratik K. Moulik Naveen Garg Carol C. Wu Hien Van Nguyen
author_facet	Aryan Mobiny Pengyu Yuan Supratik K. Moulik Naveen Garg Carol C. Wu Hien Van Nguyen
author_sort	Aryan Mobiny
title	DropConnect is effective in modeling uncertainty of Bayesian deep networks
title_short	DropConnect is effective in modeling uncertainty of Bayesian deep networks
title_full	DropConnect is effective in modeling uncertainty of Bayesian deep networks
title_fullStr	DropConnect is effective in modeling uncertainty of Bayesian deep networks
title_full_unstemmed	DropConnect is effective in modeling uncertainty of Bayesian deep networks
title_sort	dropconnect is effective in modeling uncertainty of bayesian deep networks
publisher	Nature Portfolio
publishDate	2021
url	https://doaj.org/article/ff6a04b4a5f649c98b73aae38d3ec8c3
work_keys_str_mv	AT aryanmobiny dropconnectiseffectiveinmodelinguncertaintyofbayesiandeepnetworks AT pengyuyuan dropconnectiseffectiveinmodelinguncertaintyofbayesiandeepnetworks AT supratikkmoulik dropconnectiseffectiveinmodelinguncertaintyofbayesiandeepnetworks AT naveengarg dropconnectiseffectiveinmodelinguncertaintyofbayesiandeepnetworks AT carolcwu dropconnectiseffectiveinmodelinguncertaintyofbayesiandeepnetworks AT hienvannguyen dropconnectiseffectiveinmodelinguncertaintyofbayesiandeepnetworks
_version_	1718385549790674944

DropConnect is effective in modeling uncertainty of Bayesian deep networks

Ejemplares similares