Algorithms for Nonnegative Matrix Factorization with the β-Divergence
Tóm tắt
This letter describes algorithms for nonnegative matrix factorization (NMF) with the β-divergence (β-NMF). The β-divergence is a family of cost functions parameterized by a single shape parameter β that takes the Euclidean distance, the Kullback-Leibler divergence, and the Itakura-Saito divergence as special cases (β = 2, 1, 0 respectively). The proposed algorithms are based on a surrogate auxiliary function (a local majorization of the criterion function). We first describe a majorization-minimization algorithm that leads to multiplicative updates, which differ from standard heuristic multiplicative updates by a β-dependent power exponent. The monotonicity of the heuristic algorithm can, however, be proven for β ∈ (0, 1) using the proposed auxiliary function. Then we introduce the concept of the majorization-equalization (ME) algorithm, which produces updates that move along constant level sets of the auxiliary function and lead to larger steps than MM. Simulations on synthetic and real data illustrate the faster convergence of the ME approach. The letter also describes how the proposed algorithms can be adapted to two common variants of NMF: penalized NMF (when a penalty function of the factors is added to the criterion function) and convex NMF (when the dictionary is assumed to belong to a known subspace).
Từ khóa
Tài liệu tham khảo
Dessein A., 2010, Proc. 11th International Society for Music Information Retrieval Conference
Dhillon I. S., 2005, Advances in neural information processing systems, 19
Donoho D., 2004, Advances in neural information processing systems, 16
Drakakis K., 2007, International Mathematical Forum, 3, 1853
Févotte C., 2009, Proc. 17th European Signal Processing Conference (EUSIPCO), 1913
FitzGerald D., 2009, Proc. Irish Signals and Systems Conference
Gonzalez E. F., 2005, Accelerating the Lee-Seung algorithm for non-negative matrix factorizations
Lantéri H., 2010, Proc. 18th European Signal Processing Conference (EUSIPCO)
Lee D. D., 2001, Advances in neural and information processing systems, 13, 556
Mairal J., 2010, Journal of Machine Learning Research, 11, 10
Mørup M., 2009, Proc. 17th European Signal Processing Conference (EUSIPCO’09)
Nakano M., 2010, Proc. IEEE International Workshop on Machine Learning for Signal Processing
Salakhutdinov R., 2003, Proc. International Conference on Machine Learning, 664