Entropic risk minimization for nonparametric estimation of mixing distributions

Machine Learning - Tập 99 - Trang 119-136 - 2014
Kazuho Watanabe1, Shiro Ikeda2
1Department of Computer Science and Engineering, Toyohashi University of Technology, Toyohashi, Japan
2The Institute of Statistical Mathematics, Tachikawa-shi, Japan

Tóm tắt

We discuss a nonparametric estimation method for the mixing distributions in mixture models. The problem is formalized as a minimization of a one-parameter objective functional, which becomes the maximum likelihood estimation or the kernel vector quantization in special cases. Generalizing the theorem for the nonparametric maximum likelihood estimation, we prove the existence and discreteness of the optimal mixing distribution and provide an algorithm to calculate it. It is demonstrated that with an appropriate choice of the parameter, the proposed method is less prone to overfitting than the maximum likelihood method. We further discuss the connection between the unifying estimation framework and the rate-distortion problem.

Tài liệu tham khảo