Independent Factor Analysis

Neural Computation - Tập 11 Số 4 - Trang 803-851 - 1999
Hagai Attias1
1Sloan Center for Theoretical Neurobiology and W. M. Keck Foundation Center for Integrative Neuroscience, University of California at San Francisco, San Francisco, CA 94143-0444, U.S.A.

Tóm tắt

We introduce the independent factor analysis (IFA) method for recovering independent hidden sources from their observed mixtures. IFA generalizes and unifies ordinary factor analysis (FA), principal component analysis (PCA), and independent component analysis (ICA), and can handle not only square noiseless mixing but also the general case where the number of mixtures differs from the number of sources and the data are noisy. IFA is a two-step procedure. In the first step, the source densities, mixing matrix, and noise covariance are estimated from the observed data by maximum likelihood. For this purpose we present an expectation-maximization (EM) algorithm, which performs unsupervised learning of an associated probabilistic model of the mixing situation. Each source in our model is described by a mixture of gaussians; thus, all the probabilistic calculations can be performed analytically. In the second step, the sources are reconstructed from the observed data by an optimal nonlinear estimator. A variational approximation of this algorithm is derived for cases with a large number of sources, where the exact algorithm becomes intractable. Our IFA algorithm reduces to the one for ordinary FA when the sources become gaussian, and to an EM algorithm for PCA in the zero-noise limit. We derive an additional EM algorithm specifically for noiseless IFA. This algorithm is shown to be superior to ICA since it can learn arbitrary source densities from the data. Beyond blind separation, IFA can be used for modeling multidimensional data by a highly constrained mixture of gaussians and as a tool for nonlinear signal encoding.

Từ khóa


Tài liệu tham khảo

10.1162/neco.1998.10.6.1373

10.1162/neco.1995.7.6.1129

Belouchrani A., 1994, Proc. EUSIPCO, 768

10.1162/089976698300017953

10.1109/97.566704

10.1109/78.553476

10.1016/0165-1684(91)90080-3

10.1016/0165-1684(94)90029-9

10.1162/neco.1995.7.5.889

Dempster A. P., 1977, Journal of the Royal Statistical Society B, 39, 1

10.2307/2287576

10.1023/A:1007425814087

10.1126/science.7761831

10.1214/aos/1176349519

10.1162/neco.1997.9.7.1483

10.1162/neco.1994.6.2.181

10.1016/0165-1684(91)90079-X

10.1162/neco.1992.4.3.415

10.1038/381607a0

10.1016/S0042-6989(97)00169-7

10.1109/78.542183

Roweis S., 1997, Neural Computation, 11, 297

10.1007/BF02293851

10.1613/jair.251

Thông tin
Thông tin xuất bản

Neural Computation

Tập 11 Số 4

803-851

Thông tin tác giả