Fast Approximations of the Jeffreys Divergence between Univariate Gaussian Mixtures via Mixture Conversions to Exponential-Polynomial Distributions

The Jeffreys divergence is a renown arithmetic symmetrization of the oriented Kullback–Leibler divergence broadly used in information sciences. Since the Jeffreys divergence between Gaussian mixture models is not available in closed-form, various techniques with advantages and disadvantages have bee...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autor principal:	Frank Nielsen
Formato:	article
Lenguaje:	EN
Publicado:	MDPI AG 2021
Materias:	Gaussian mixture model Jeffreys divergence mixture family exponential-polynomial family Maximum Likelihood Estimator Score Matching Estimator Science Q Astrophysics QB460-466 Physics QC1-999
Acceso en línea:	https://doaj.org/article/f8ed504ac684481fb7f860145ec5bc48
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:f8ed504ac684481fb7f860145ec5bc48
record_format	dspace
spelling	oai:doaj.org-article:f8ed504ac684481fb7f860145ec5bc482021-11-25T17:29:30ZFast Approximations of the Jeffreys Divergence between Univariate Gaussian Mixtures via Mixture Conversions to Exponential-Polynomial Distributions10.3390/e231114171099-4300https://doaj.org/article/f8ed504ac684481fb7f860145ec5bc482021-10-01T00:00:00Zhttps://www.mdpi.com/1099-4300/23/11/1417https://doaj.org/toc/1099-4300The Jeffreys divergence is a renown arithmetic symmetrization of the oriented Kullback–Leibler divergence broadly used in information sciences. Since the Jeffreys divergence between Gaussian mixture models is not available in closed-form, various techniques with advantages and disadvantages have been proposed in the literature to either estimate, approximate, or lower and upper bound this divergence. In this paper, we propose a simple yet fast heuristic to approximate the Jeffreys divergence between two univariate Gaussian mixtures with arbitrary number of components. Our heuristic relies on converting the mixtures into pairs of dually parameterized probability densities belonging to an exponential-polynomial family. To measure with a closed-form formula the goodness of fit between a Gaussian mixture and an exponential-polynomial density approximating it, we generalize the Hyvärinen divergence to <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mi>α</mi></semantics></math></inline-formula>-Hyvärinen divergences. In particular, the 2-Hyvärinen divergence allows us to perform model selection by choosing the order of the exponential-polynomial densities used to approximate the mixtures. We experimentally demonstrate that our heuristic to approximate the Jeffreys divergence between mixtures improves over the computational time of stochastic Monte Carlo estimations by several orders of magnitude while approximating the Jeffreys divergence reasonably well, especially when the mixtures have a very small number of modes.Frank NielsenMDPI AGarticleGaussian mixture modelJeffreys divergencemixture familyexponential-polynomial familyMaximum Likelihood EstimatorScore Matching EstimatorScienceQAstrophysicsQB460-466PhysicsQC1-999ENEntropy, Vol 23, Iss 1417, p 1417 (2021)
institution	DOAJ
collection	DOAJ
language	EN
topic	Gaussian mixture model Jeffreys divergence mixture family exponential-polynomial family Maximum Likelihood Estimator Score Matching Estimator Science Q Astrophysics QB460-466 Physics QC1-999
spellingShingle	Gaussian mixture model Jeffreys divergence mixture family exponential-polynomial family Maximum Likelihood Estimator Score Matching Estimator Science Q Astrophysics QB460-466 Physics QC1-999 Frank Nielsen Fast Approximations of the Jeffreys Divergence between Univariate Gaussian Mixtures via Mixture Conversions to Exponential-Polynomial Distributions
description	The Jeffreys divergence is a renown arithmetic symmetrization of the oriented Kullback–Leibler divergence broadly used in information sciences. Since the Jeffreys divergence between Gaussian mixture models is not available in closed-form, various techniques with advantages and disadvantages have been proposed in the literature to either estimate, approximate, or lower and upper bound this divergence. In this paper, we propose a simple yet fast heuristic to approximate the Jeffreys divergence between two univariate Gaussian mixtures with arbitrary number of components. Our heuristic relies on converting the mixtures into pairs of dually parameterized probability densities belonging to an exponential-polynomial family. To measure with a closed-form formula the goodness of fit between a Gaussian mixture and an exponential-polynomial density approximating it, we generalize the Hyvärinen divergence to <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mi>α</mi></semantics></math></inline-formula>-Hyvärinen divergences. In particular, the 2-Hyvärinen divergence allows us to perform model selection by choosing the order of the exponential-polynomial densities used to approximate the mixtures. We experimentally demonstrate that our heuristic to approximate the Jeffreys divergence between mixtures improves over the computational time of stochastic Monte Carlo estimations by several orders of magnitude while approximating the Jeffreys divergence reasonably well, especially when the mixtures have a very small number of modes.
format	article
author	Frank Nielsen
author_facet	Frank Nielsen
author_sort	Frank Nielsen
title	Fast Approximations of the Jeffreys Divergence between Univariate Gaussian Mixtures via Mixture Conversions to Exponential-Polynomial Distributions
title_short	Fast Approximations of the Jeffreys Divergence between Univariate Gaussian Mixtures via Mixture Conversions to Exponential-Polynomial Distributions
title_full	Fast Approximations of the Jeffreys Divergence between Univariate Gaussian Mixtures via Mixture Conversions to Exponential-Polynomial Distributions
title_fullStr	Fast Approximations of the Jeffreys Divergence between Univariate Gaussian Mixtures via Mixture Conversions to Exponential-Polynomial Distributions
title_full_unstemmed	Fast Approximations of the Jeffreys Divergence between Univariate Gaussian Mixtures via Mixture Conversions to Exponential-Polynomial Distributions
title_sort	fast approximations of the jeffreys divergence between univariate gaussian mixtures via mixture conversions to exponential-polynomial distributions
publisher	MDPI AG
publishDate	2021
url	https://doaj.org/article/f8ed504ac684481fb7f860145ec5bc48
work_keys_str_mv	AT franknielsen fastapproximationsofthejeffreysdivergencebetweenunivariategaussianmixturesviamixtureconversionstoexponentialpolynomialdistributions
_version_	1718412324166959104

Fast Approximations of the Jeffreys Divergence between Univariate Gaussian Mixtures via Mixture Conversions to Exponential-Polynomial Distributions

Ejemplares similares