Regularized Divergences Between Covariance Operators and Gaussian Measures on Hilbert Spaces
Tóm tắt
This work presents an infinite-dimensional generalization of the correspondence between the Kullback–Leibler and Rényi divergences between Gaussian measures on Euclidean space and the Alpha Log-Determinant divergences between symmetric, positive definite matrices. Specifically, we present the regularized Kullback–Leibler and Rényi divergences between covariance operators and Gaussian measures on an infinite-dimensional Hilbert space, which are defined using the infinite-dimensional Alpha Log-Determinant divergences between positive definite trace class operators. We show that, as the regularization parameter approaches zero, the regularized Kullback–Leibler and Rényi divergences between two equivalent Gaussian measures on a Hilbert space converge to the corresponding true divergences. The explicit formulas for the divergences involved are presented in the most general Gaussian setting.
Từ khóa
Tài liệu tham khảo
Abramowitz, M., Stegun, I.A.: Handbook of Mathematical Functions With Formulas, Graphs, and Mathematical Tables. Applied Mathematics Series 55. National Bureau of Standards (1972)
Alexanderian, A., Gloor, P.J., Ghattas, O.: On Bayesian A-and D-optimal experimental designs in infinite dimensions. Bayesian Anal. 11(3), 671–695 (2016)
Arsigny, V., Fillard, P., Pennec, X., Ayache, N.: Geometric means in a novel vector space structure on symmetric positive-definite matrices. SIAM J. Matrix Anal. Appl. 29(1), 328–347 (2007)
Bogachev, V.: Gaussian Measures. American Mathematical Society, Providence (1998)
Capon, J.: Randon–Nikodym derivatives of stationary Gaussian measures. Ann. Math. Stat. 35(2), 517–531 (1964). 06
Chebbi, Z., Moakher, M.: Means of Hermitian positive-definite matrices based on the log-determinant \(\alpha \)-divergence function. Linear Algebra Appl. 436(7), 1872–1889 (2012)
Da Prato, G.: An Introduction to Infinite-Dimensional Analysis. Springer, Berlin (2006)
Da Prato, G., Zabczyk, J.: Second Order Partial Differential Equations in Hilbert Spaces, vol. 293. Cambridge University Press, Cambridge (2002)
Feldman, J.: Equivalence and perpendicularity of Gaussian processes. Pacific J. Math. 8(4), 699–708 (1958)
Fillmore, P.A., Williams, J.P.: On operator ranges. Adv. Math. 7(3), 254–281 (1971)
Folland, G.B.: Real Analysis: Modern Techniques and Their Applications. Wiley, New York (1999)
Hájek, J.: On a property of normal distributions of any stochastic process. Czechoslov. Math. J. 08(4), 610–618 (1958)
Harandi, M., Salzmann, M., Porikli, F.: Bregman divergences for infinite dimensional covariance matrices. In: CVPR (2014)
Henrich, C.J.: Equivalence and Radon–Nikodym derivatives of Gaussian measures. J. Math. Anal. Appl. 37(1), 255–270 (1972)
Larotonda, G.: Nonpositive curvature: a geometrical approach to Hilbert–Schmidt operators. Differ. Geom. Appl. 25, 679–700 (2007)
Michálek, J.: The Rényi distances of Gaussian measures. Kybernetika 35(3), 333–352 (1999)
Minh, H.Q., San Biagio, M., Murino, V.: Log-Hilbert–Schmidt metric between positive definite operators on Hilbert spaces. In: Advances in Neural Information Processing Systems 27 (NIPS 2014), pp. 388–396 (2014)
Minh, H.Q.: Affine-invariant Riemannian distance between infinite-dimensional covariance operators. In: International Conference on Geometric Science of Information, pp. 30–38. Springer (2015)
Minh, H.Q.: Infinite-dimensional Log-Determinant divergences between positive definite trace class operators. Linear Algebra Appl. 528, 331–383 (2017)
Minh, H.Q.: Log-determinant divergences between positive definite Hilbert–Schmidt operators. In: International Conference on Geometric Science of Information, pp. 505–513. Springer (2017)
Minh, H.Q.: Alpha-Beta Log-Determinant divergences between positive definite trace class operators. Inf. Geom. 2(2), 101–176 (2019)
Minh, H.Q.: Infinite-dimensional log-determinant divergences between positive definite Hilbert-Schmidt operators. Positivity (2019). https://doi.org/10.1007/s11117-019-00701-4
Minh, H.Q.: A unified formulation for the Bures-Wasserstein and Log-Euclidean/Log-Hilbert–Schmidt distances between positive definite operators. In: International Conference on Geometric Science of Information. Springer (2019)
Minh, H.Q., Murino, V.: Covariances in computer vision and machine learning. Synth. Lect. Comput. Vis. 7(4), 1–170 (2017)
Minh, H.Q., San Biagio, M., Bazzani, L., Murino, V.: Approximate Log-Hilbert–Schmidt distances between covariance operators for image classification. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)
Pardo, L.: Statistical Inference Based on Divergence Measures. CRC Press, Cambridge (2005)
Pennec, X., Fillard, P., Ayache, N.: A Riemannian framework for tensor computing. Int. J. Comput. Vis. 66(1), 41–66 (2006)
Pinski, F.J., Simpson, G., Stuart, A.M., Weber, H.: Algorithms for Kullback–Leibler approximation of probability measures in infinite dimensions. SIAM J. Sci. Comput. 37(6), A2733–A2757 (2015)
Pinski, F.J., Simpson, G., Stuart, A.M., Weber, H.: Kullback–Leibler approximation for probability measures on infinite dimensional spaces. SIAM J. Math. Anal. 47(6), 4091–4122 (2015)
Reed, M., Simon, B.: Methods of Modern Mathematical Physics: Functional analysis. Academic Press, New York (1975)
Rényi, A.: On measures of entropy and information. In: Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, Volume 1: Contributions to the Theory of Statistics, pp. 547–561, Berkeley, California. University of California Press (1961)
Rudin, W.: Real and Complex Analysis. McGraw-Hill, New York (1987)
Shepp, L.A.: Radon–Nikodym derivatives of Gaussian measures. Ann. Math. Stat. 37, 321–354 (1966)
Simon, B.: Notes on infinite determinants of Hilbert space operators. Adv. Math. 24, 244–273 (1977)
Stuart, A.M.: Inverse problems: a Bayesian perspective. Acta Numer. 19, 451–559 (2010)
Zhou, S.K., Chellappa, R.: From sample similarity to ensemble similarity: probabilistic distance measures in reproducing kernel Hilbert space. TPAMI 28(6), 917–929 (2006)