Hyper-parameter optimization for support vector machines using stochastic gradient descent and dual coordinate descent

We developed a gradient-based method to optimize the regularization hyper-parameter, C, for support vector machines in a bilevel optimization framework. On the upper level, we optimized the hyper-parameter C to minimize the prediction loss on validation data using stochastic gradient descent. On the...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	W.e.i. Jiang, Sauleh Siddiqui
Formato:	article
Lenguaje:	EN
Publicado:	Elsevier 2020
Materias:	90–08 Applied mathematics. Quantitative methods T57-57.97 Electronic computers. Computer science QA75.5-76.95
Acceso en línea:	https://doaj.org/article/f82d7c90108a43cf8c8fd9386871b915
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:f82d7c90108a43cf8c8fd9386871b915
record_format	dspace
spelling	oai:doaj.org-article:f82d7c90108a43cf8c8fd9386871b9152021-12-02T05:00:33ZHyper-parameter optimization for support vector machines using stochastic gradient descent and dual coordinate descent2192-440610.1007/s13675-019-00115-7https://doaj.org/article/f82d7c90108a43cf8c8fd9386871b9152020-03-01T00:00:00Zhttp://www.sciencedirect.com/science/article/pii/S2192440621000083https://doaj.org/toc/2192-4406We developed a gradient-based method to optimize the regularization hyper-parameter, C, for support vector machines in a bilevel optimization framework. On the upper level, we optimized the hyper-parameter C to minimize the prediction loss on validation data using stochastic gradient descent. On the lower level, we used dual coordinate descent to optimize the parameters of support vector machines to minimize the loss on training data. The gradient of the loss function on the upper level with respect to the hyper-parameter, C, was computed using the implicit function theorem combined with the optimality condition of the lower-level problem, i.e., the dual problem of support vector machines. We compared our method with the existing gradient-based method in the literature on several datasets. Numerical results showed that our method converges faster to the optimal solution and achieves better prediction accuracy for large-scale support vector machine problems.W.e.i. JiangSauleh SiddiquiElsevierarticle90–08Applied mathematics. Quantitative methodsT57-57.97Electronic computers. Computer scienceQA75.5-76.95ENEURO Journal on Computational Optimization, Vol 8, Iss 1, Pp 85-101 (2020)
institution	DOAJ
collection	DOAJ
language	EN
topic	90–08 Applied mathematics. Quantitative methods T57-57.97 Electronic computers. Computer science QA75.5-76.95
spellingShingle	90–08 Applied mathematics. Quantitative methods T57-57.97 Electronic computers. Computer science QA75.5-76.95 W.e.i. Jiang Sauleh Siddiqui Hyper-parameter optimization for support vector machines using stochastic gradient descent and dual coordinate descent
description	We developed a gradient-based method to optimize the regularization hyper-parameter, C, for support vector machines in a bilevel optimization framework. On the upper level, we optimized the hyper-parameter C to minimize the prediction loss on validation data using stochastic gradient descent. On the lower level, we used dual coordinate descent to optimize the parameters of support vector machines to minimize the loss on training data. The gradient of the loss function on the upper level with respect to the hyper-parameter, C, was computed using the implicit function theorem combined with the optimality condition of the lower-level problem, i.e., the dual problem of support vector machines. We compared our method with the existing gradient-based method in the literature on several datasets. Numerical results showed that our method converges faster to the optimal solution and achieves better prediction accuracy for large-scale support vector machine problems.
format	article
author	W.e.i. Jiang Sauleh Siddiqui
author_facet	W.e.i. Jiang Sauleh Siddiqui
author_sort	W.e.i. Jiang
title	Hyper-parameter optimization for support vector machines using stochastic gradient descent and dual coordinate descent
title_short	Hyper-parameter optimization for support vector machines using stochastic gradient descent and dual coordinate descent
title_full	Hyper-parameter optimization for support vector machines using stochastic gradient descent and dual coordinate descent
title_fullStr	Hyper-parameter optimization for support vector machines using stochastic gradient descent and dual coordinate descent
title_full_unstemmed	Hyper-parameter optimization for support vector machines using stochastic gradient descent and dual coordinate descent
title_sort	hyper-parameter optimization for support vector machines using stochastic gradient descent and dual coordinate descent
publisher	Elsevier
publishDate	2020
url	https://doaj.org/article/f82d7c90108a43cf8c8fd9386871b915
work_keys_str_mv	AT weijiang hyperparameteroptimizationforsupportvectormachinesusingstochasticgradientdescentanddualcoordinatedescent AT saulehsiddiqui hyperparameteroptimizationforsupportvectormachinesusingstochasticgradientdescentanddualcoordinatedescent
_version_	1718400867341697024

Hyper-parameter optimization for support vector machines using stochastic gradient descent and dual coordinate descent

Ejemplares similares