An Adaptive Deep Learning Optimization Method Based on Radius of Curvature

An adaptive clamping method (SGD-MS) based on the radius of curvature is designed to alleviate the local optimal oscillation problem in deep neural network, which combines the radius of curvature of the objective function and the gradient descent of the optimizer. The radius of curvature is consider...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	Jiahui Zhang, Xinhao Yang, Ke Zhang, Chenrui Wen
Formato:	article
Lenguaje:	EN
Publicado:	Hindawi Limited 2021
Materias:	Computer applications to medicine. Medical informatics R858-859.7 Neurosciences. Biological psychiatry. Neuropsychiatry RC321-571
Acceso en línea:	https://doaj.org/article/acab22f5e532433c807e6c5f9bb9d3fc
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:acab22f5e532433c807e6c5f9bb9d3fc
record_format	dspace
spelling	oai:doaj.org-article:acab22f5e532433c807e6c5f9bb9d3fc2021-11-22T01:10:33ZAn Adaptive Deep Learning Optimization Method Based on Radius of Curvature1687-527310.1155/2021/9882068https://doaj.org/article/acab22f5e532433c807e6c5f9bb9d3fc2021-01-01T00:00:00Zhttp://dx.doi.org/10.1155/2021/9882068https://doaj.org/toc/1687-5273An adaptive clamping method (SGD-MS) based on the radius of curvature is designed to alleviate the local optimal oscillation problem in deep neural network, which combines the radius of curvature of the objective function and the gradient descent of the optimizer. The radius of curvature is considered as the threshold to separate the momentum term or the future gradient moving average term adaptively. In addition, on this basis, we propose an accelerated version (SGD-MA), which further improves the convergence speed by using the method of aggregated momentum. Experimental results on several datasets show that the proposed methods effectively alleviate the local optimal oscillation problem and greatly improve the convergence speed and accuracy. A novel parameter updating algorithm is also provided in this paper for deep neural network.Jiahui ZhangXinhao YangKe ZhangChenrui WenHindawi LimitedarticleComputer applications to medicine. Medical informaticsR858-859.7Neurosciences. Biological psychiatry. NeuropsychiatryRC321-571ENComputational Intelligence and Neuroscience, Vol 2021 (2021)
institution	DOAJ
collection	DOAJ
language	EN
topic	Computer applications to medicine. Medical informatics R858-859.7 Neurosciences. Biological psychiatry. Neuropsychiatry RC321-571
spellingShingle	Computer applications to medicine. Medical informatics R858-859.7 Neurosciences. Biological psychiatry. Neuropsychiatry RC321-571 Jiahui Zhang Xinhao Yang Ke Zhang Chenrui Wen An Adaptive Deep Learning Optimization Method Based on Radius of Curvature
description	An adaptive clamping method (SGD-MS) based on the radius of curvature is designed to alleviate the local optimal oscillation problem in deep neural network, which combines the radius of curvature of the objective function and the gradient descent of the optimizer. The radius of curvature is considered as the threshold to separate the momentum term or the future gradient moving average term adaptively. In addition, on this basis, we propose an accelerated version (SGD-MA), which further improves the convergence speed by using the method of aggregated momentum. Experimental results on several datasets show that the proposed methods effectively alleviate the local optimal oscillation problem and greatly improve the convergence speed and accuracy. A novel parameter updating algorithm is also provided in this paper for deep neural network.
format	article
author	Jiahui Zhang Xinhao Yang Ke Zhang Chenrui Wen
author_facet	Jiahui Zhang Xinhao Yang Ke Zhang Chenrui Wen
author_sort	Jiahui Zhang
title	An Adaptive Deep Learning Optimization Method Based on Radius of Curvature
title_short	An Adaptive Deep Learning Optimization Method Based on Radius of Curvature
title_full	An Adaptive Deep Learning Optimization Method Based on Radius of Curvature
title_fullStr	An Adaptive Deep Learning Optimization Method Based on Radius of Curvature
title_full_unstemmed	An Adaptive Deep Learning Optimization Method Based on Radius of Curvature
title_sort	adaptive deep learning optimization method based on radius of curvature
publisher	Hindawi Limited
publishDate	2021
url	https://doaj.org/article/acab22f5e532433c807e6c5f9bb9d3fc
work_keys_str_mv	AT jiahuizhang anadaptivedeeplearningoptimizationmethodbasedonradiusofcurvature AT xinhaoyang anadaptivedeeplearningoptimizationmethodbasedonradiusofcurvature AT kezhang anadaptivedeeplearningoptimizationmethodbasedonradiusofcurvature AT chenruiwen anadaptivedeeplearningoptimizationmethodbasedonradiusofcurvature AT jiahuizhang adaptivedeeplearningoptimizationmethodbasedonradiusofcurvature AT xinhaoyang adaptivedeeplearningoptimizationmethodbasedonradiusofcurvature AT kezhang adaptivedeeplearningoptimizationmethodbasedonradiusofcurvature AT chenruiwen adaptivedeeplearningoptimizationmethodbasedonradiusofcurvature
_version_	1718418361543557120

An Adaptive Deep Learning Optimization Method Based on Radius of Curvature

Ejemplares similares