Algorithms for Nonnegative Matrix Factorization with the β-Divergence

Neural Computation - Tập 23 Số 9 - Trang 2421-2456 - 2011
Cédric Févotte1, Jérôme Idier2
1LTCI - Laboratoire Traitement et Communication de l'Information (46 rue Barrault F-75634 Paris Cedex 13 - France)
2Institute de Recherche en Communications et Cybernetique de Nantes (CNRS, Ecole Centrale de Nantes, Ecole des Mines de Nantes, and Université de Nantes), 44000 Nantes, France

Tóm tắt

This letter describes algorithms for nonnegative matrix factorization (NMF) with the β-divergence (β-NMF). The β-divergence is a family of cost functions parameterized by a single shape parameter β that takes the Euclidean distance, the Kullback-Leibler divergence, and the Itakura-Saito divergence as special cases (β = 2, 1, 0 respectively). The proposed algorithms are based on a surrogate auxiliary function (a local majorization of the criterion function). We first describe a majorization-minimization algorithm that leads to multiplicative updates, which differ from standard heuristic multiplicative updates by a β-dependent power exponent. The monotonicity of the heuristic algorithm can, however, be proven for β ∈ (0, 1) using the proposed auxiliary function. Then we introduce the concept of the majorization-equalization (ME) algorithm, which produces updates that move along constant level sets of the auxiliary function and lead to larger steps than MM. Simulations on synthetic and real data illustrate the faster convergence of the ME approach. The letter also describes how the proposed algorithms can be adapted to two common variants of NMF: penalized NMF (when a penalty function of the factors is added to the criterion function) and convex NMF (when the dictionary is assumed to belong to a known subspace).

Từ khóa


Tài liệu tham khảo

10.1109/TNN.2010.2076831

10.1093/biomet/85.3.549

10.1016/j.csda.2006.11.006

10.1109/ICASSP.2009.4959891

10.1073/pnas.0308531101

10.1109/83.743861

10.3390/e12061532

10.1016/j.patrec.2008.02.016

10.1007/11679363_5

10.1109/TMI.1986.4307748

10.1109/42.232263

Dessein A., 2010, Proc. 11th International Society for Music Information Retrieval Conference

Dhillon I. S., 2005, Advances in neural information processing systems, 19

10.1109/TPAMI.2008.277

Donoho D., 2004, Advances in neural information processing systems, 16

Drakakis K., 2007, International Mathematical Forum, 3, 1853

10.1162/neco.2008.04-08-771

Févotte C., 2009, Proc. 17th European Signal Processing Conference (EUSIPCO), 1913

FitzGerald D., 2009, Proc. Irish Signals and Systems Conference

10.1093/bioinformatics/bti653

Gonzalez E. F., 2005, Accelerating the Lee-Seung algorithm for non-negative matrix factorizations

10.1093/bioinformatics/btn286

10.1109/ICASSP.2010.5495733

10.1198/0003130042836

10.1002/sam.104

10.1162/neco.2007.19.3.780

10.1016/S0165-1684(00)00275-9

Lantéri H., 2010, Proc. 18th European Signal Processing Conference (EUSIPCO)

10.1038/44565

Lee D. D., 2001, Advances in neural and information processing systems, 13, 556

10.1109/TNN.2007.895831

10.1162/neco.2007.19.10.2756

10.1086/111605

Mairal J., 2010, Journal of Machine Learning Research, 11, 10

10.1162/089976602760128045

10.1109/MLSP.2007.4414278

Mørup M., 2009, Proc. 17th European Signal Processing Conference (EUSIPCO’09)

Nakano M., 2010, Proc. IEEE International Workshop on Machine Learning for Signal Processing

10.1016/j.neucom.2008.01.033

10.1016/j.laa.2005.06.025

10.1364/JOSA.62.000055

Salakhutdinov R., 2003, Proc. International Conference on Machine Learning, 664

10.1109/ASPAA.2003.1285860

10.1109/MLSP.2009.5306194

10.1109/TMI.1987.4307797

10.1109/TASL.2009.2034186

Thông tin
Thông tin xuất bản

Neural Computation

Tập 23 Số 9

2421-2456

Thông tin tác giả