K n -nearest neighbor estimators of entropy
Tóm tắt
For estimating the entropy of an absolutely continuous multivariate distribution, we propose nonparametric estimators based on the Euclidean distances between the n sample points and their k
n
-nearest neighbors, where {k
n
: n = 1, 2, …} is a sequence of positive integers varying with n. The proposed estimators are shown to be asymptotically unbiased and consistent.
Tài liệu tham khảo
E. Demchuk and H. Singh, “Statistical Thermodynamics of Hindered Rotation from Computer Simulations”, Molecular Physics 99, 627–636 (2001).
D. Scott, Multivariate Density Estimation: Theory, Practice and Visualization (Wiley, New York, 1992).
L. F. Kozachenko and N. N. Leonenko, “Sample Estimates of Entropy of a Random Vector”, Probl. Inform. Transmission 23, 95–101 (1987).
H. Singh, N. Misra, V. Hnizdo, A. Fedorowicz, and F. Demchuk, “Nearest Neighbor Estimates of Entropy”, Amer. J. Math. and Management Sci. 23, 301–321 (2003).
M. N. Goria, N. N. Leonenko, V. V. Mergel, and P. L. Novi Inverardi, “A New Class of Random Vector Entropy Estimators and Its Applications in Testing Statistical Hypotheses”, J. Nonparam. Statist. 17, 277–297 (2005).
O. Vasicek, “A Test of Normality Based on Sample Entropy”, J. Roy. Statist. Soc. Ser. B 38, 254–256 (1976).
E. J. Dudewicz and E. C. van der Meulen, (1981). “Entropy-Based Tests of Unifority”, J. Amer. Statist. Assos. 76, 967–974 (1981).
M. Abramowitz and A. Stegun, Handbook of Mathematical Functions (Dover, New York, 1965).
B. van Es, “Estimating Functionals Related to a Density by Class of Statistics Based on Spacing”, Scand. J. Statist. 19, 61–72 (1992).
P. Hall and S. C. Morton, “On the Estimation of Entropy”, Ann. Inst. Statist. Math. 45, 69–88 (1993).
J. Beirlant, E. J. Dudewicz, L. Gyorfi, and E. C. van der Meulen, “Nonparametric Entropy Estimation: an Overview”, Internat. J. Math. Statist. Sci. 6, 17–39 (1997).
H. Joe, “Estimation of Entropy and Other Functionals of a Multivariate Density”, Ann. Inst. Statist. Math. 41, 683–697 (1989).
A. V. Ivanov and M. N. Rozhkova, “Properties of the Statistical Estimate of the Entropy of a Random Vector with a Probability Density”, Probl. Inform. Transmission 17, 171–178 (1981).
A. B. Tsybakov and E. C. van der Meulen, “Root-n Consistent Estimators of Entropy for Densities with Unbounded Support”, Scand. J. Statist. 23, 75–83 (1996).
D. O. Loftsgaarden and C. P. Quesenberry, “A Non-Parametric Estimate of a Multivariate Density Function”, Ann. Math. Statist. 36, 1049–1051 (1965).
N. Leonenko, L. Pronzato, and V. Savani, “A Class of Rényi Information Estimators for Multidimensional Densities”, Ann. Statist. (2007) (in press).
P. Billingsley, Convergence of Probability Measures (Wiley, New York, 1968).
P. Billingsley, Probability and Measure (Wiley, New York, 1995).
E. V. Khmaladze, The Statistical Analysis of a Large Number of Rare Events, Technical Report MS-R8804 (Centre of Mathematics and Computer Science, Amsterdam, 1988).
M. Loève, Probability Theory (Springer, New York, 1977), Vol. I.
K. V. Mardia and P. E. Jupp, Directional Statistics (Wiley, New York, 2000).
N. Misra, H. Singh, and V. Hnizdo, Nearest Neighbor Estimates of Entropy for Multivariate Circular Distributions, Preprint (2006).
H. Singh, V. Hnizdo, and E. Demchuk, “Probabilistic Model for Two Dependent Circular Variables”, Biometrika 89, 719–723 (2002).