On convergence properties of Shannon entropy

Antos, A. and Kontoyiannis, I., Convergence Properties of Functional Estimates of Discrete Distributions, Random Structures Algorithms, 2001, vol. 19, no. 3–4, pp. 163–193.

Hall, P. and Morton, S.C., On the Estimation of Entropy, Ann. Inst. Statist. Math., 1993, vol. 45, no. 1, pp. 69–88.

Wieczorkowski, R. and Grzegorzewski, P., Entropy Estimators-Improvements and Comparisons, Comm. Statist. Simulation Comput., 1999, vol. 28, no. 2, pp. 541–567.

Wyner, A. and Foster, D., On the Lower Limits of Entropy Estimation, Tech. Rep. Dept. Statistics, Wharton School, Univ. Pennsylvania, 2003. Available at http://www-stat.wharton.upenn.edu/~ajw/lowlimitsentropy.pdf .

Blahut, R.E., Principles and Practice of Information Theory, Reading: Addison-Wesley, 1987.

Cover, T.M. and Thomas, J.A., Elements of Information Theory, New York: Wiley, 1991.

Ho, S.-W. and Yeung, R.W., On the Discontinuity of the Shannon Information Measures, Proc. IEEE Int. Sympos. on Information Theory, Adelaide, Australia, 2005, pp. 159–163.

Barron, A.R., Entropy and the Central Limit Theorem, Ann. Probab., 1986, vol. 14, no. 1, pp. 336–342.

Barron, A.R., The Strong Ergodic Theorem for Densities: Generalized Shannon-McMillan-Breiman Theorem, Ann. Probab., 1985, vol. 13, no. 4, pp. 1292–1303.

Harremoës, P., Information Topologies with Applications, Entropy, Search, Complexity, Csiszár, I., Katona, G.O.H., Tardos, G., Eds., Bolyai Soc. Math. Stud., vol. 16, Berlin: Springer, 2007, pp. 113–150.

Dobrushin, R.L., Passage to the Limit under the Information and Entropy Signs, Teor. Veroyatnost. i Primenen., 1960, vol. 5, no. 1, pp. 29–37 [Theory Probab. Appl. (Engl. Transl.), 1960, vol. 5, no. 1, pp. 25–32].

Fristedt, B. and Gray, L., A Modern Approach to Probability Theory, Boston: Birkhäuser, 1997.

Kallenberg, O., Foundation of Modern Probability, New York: Springer, 2002, 2nd ed.

Billingsley, P., Convergence of Probability Measures, New York: Wiley, 1999, 2nd ed.

Shiryaev, A.N., Veroyatnost’, Moscow: Nauka, 1989, 2nd ed. Translated under the title Probability, New York: Springer, 1996.

Royden, H.L., Real Analysis, New York: Macmillan, 1988, 3rd ed.

Williams, D., Probability with Martingales, Cambridge: Cambridge Univ. Press, 1991.