Maximum Likelihood from Incomplete Data Via the <i>EM</i> Algorithm

A. P. Dempster1, Nan M. Laird1, Donald B. Rubin1
1Harvard University and Educational Testing Service

Tóm tắt

Summary A broadly applicable algorithm for computing maximum likelihood estimates from incomplete data is presented at various levels of generality. Theory showing the monotone behaviour of the likelihood and convergence of the algorithm is derived. Many examples are sketched, including missing value situations, applications to grouped, censored or truncated data, finite mixture models, variance component estimation, hyperparameter estimation, iteratively reweighted least squares and factor analysis.

Từ khóa


Tài liệu tham khảo

Andrews, 1972, Robust Estimates of Location.

Baum, 1971, An inequality and associated maximization technique in statistical estimation for probabilistic functions of Markov processes, Inequalities, III: Proceedings of a Symposium.

Baum, 1967, An inequality with applications to statistical estimation for probabilistic functions of Markov processes and to a model for ecology, Bull. Amer. Math. Soc., 73, 360, 10.1090/S0002-9904-1967-11751-8

Baum, 1970, A maximization technique occurring in the statistical analysis of probabilistic functions of Markov chains, Ann. Math. Statists., 41, 164, 10.1214/aoms/1177697196

Beale, 1975, Missing values in multivariate analysis, J. R. Statist. Soc., B, 37, 129, 10.1111/j.2517-6161.1975.tb01037.x

Blight, 1970, Estimation from a censored sample for the exponential family, Biometrika, 57, 389, 10.1093/biomet/57.2.389

Brown, 1974, Identification of the sources of significance in two-way tables, Appl. Statist., 23, 405, 10.2307/2347132

Carter, 1973, Maximum likelihood estimation from linear combinations of discrete probability functions, J. Amer. Statist. Assoc, 68, 203, 10.1080/01621459.1973.10481366

Ceppellini, 1955, The estimation of gene frequencies in a random-mating population, Ann. Hum. Genet., 20, 97, 10.1111/j.1469-1809.1955.tb01360.x

Chen, 1976, The analysis of contingency tables with incompletely classified data, Biometrics, 32, 133, 10.2307/2529344

Corbeil, 1976, Restricted maximum likelihood (REML) estimation of variance components in the mixed model, Technometrics, 18, 31, 10.2307/1267913

Day, 1969, Estimating the components of a mixture of normal distributions, Biometrika, 56, 463, 10.1093/biomet/56.3.463

Dempster, 1972, Covariance selection, Biometrics, 28, 157, 10.2307/2528966

Efron, 1967, The two-sample problem with censored data, Proc. 5th Berkeley Symposium on Math. Statist. and Prob., 4, 831

Efron, 1975, Data analysis using Stein's estimator and its generalizations, J. Amer. Statist. Assoc, 70, 311, 10.1080/01621459.1975.10479864

Good, 1965, The Estimation of Probabilities: An Essay on Modern Bayesian Methods.

Good, 1956, On the estimation of small frequencies in contingency tables, J. R. Statist. Soc., B, 18, 113, 10.1111/j.2517-6161.1956.tb00216.x

Grundy, 1952, The fitting of grouped truncated and grouped censored normal distributions, Biometrika, 39, 252, 10.1093/biomet/39.3-4.252

Haberman, 1976, Iterative scaling procedures for log-linear models for frequency tables derived by indirect observation, Proc. Amer. Statist. Assoc. (Statist. Comp. Sect. 1975), 45

Hartley, 1958, Maximum likelihood estimation from incomplete data, Biometrics, 14, 174, 10.2307/2527783

Hartley, 1971, The analysis of incomplete data, Biometrics, 27, 783, 10.2307/2528820

Hartley, 1967, Maximum likelihood estimation for the mixed analysis of variance model, Biometrika, 54, 93, 10.1093/biomet/54.1-2.93

Harville, 1977, Maximum likelihood approaches to variance component estimation and to related problems, J. Amer. Statist. Assoc, 72

Hasselblad, 1966, Estimation of parameters for a mixture of normal distributions, Technometrics, 8, 431, 10.1080/00401706.1966.10490375

Hasselblad, 1969, Estimation of finite mixtures of distributions from the exponential family, J. Amer. Statist. Assoc, 64, 1459, 10.1080/01621459.1969.10501071

Healy, 1956, Missing values in experiments analysed on automatic computers, Appl. Statist., 5, 203, 10.2307/2985421

Hosmer, 1973, On the MLE of the parameters of a mixture of two normal distributions when the sample size is small, Comm. Statist., 1, 217, 10.1080/03610927308827019

Hosmer, 1973, A comparison of iterative maximum likelihood estimates of the parameters of a mixture of two normal distributions under three different types of sample, Biometrics, 29, 761, 10.2307/2529141

Huber, 1964, Robust estimation of a location parameter, Ann. Math. Statist., 35, 73, 10.1214/aoms/1177703732

Irwin, 1959, On the estimation of the mean of a Poisson distribution with the zero class missing, Biometrics, 15, 324, 10.2307/2527678

Irwin, 1963, The place of mathematics in medical and biological statistics, J. R. Statist. Soc., A, 126, 1, 10.2307/2982445

Jöreskog, 1969, A general approach to confirmatory maximum likelihood factor analysis, Psychometrika, 34, 183, 10.1007/BF02289343

McKendrick, 1926, Applications of mathematics to medical problems, Proc. Edin. Math. Soc., 44, 98, 10.1017/S0013091500034428

Mantel, 1967, Note: Equivalence of maximum likelihood and the method of moments in probit analysis, Biometrics, 23, 154, 10.2307/2528289

Maritz, 1970, Empirical Bayes Methods.

Martin, 1975, Bayesian estimation in unrestricted factor analysis: A treatment for Heywood cases, Psychometrika, 40, 505, 10.1007/BF02291552

Mosteller, 1964, Inference and Disputed Authorship: The Federalist.

Orchard, 1972, A missing information principle: theory and applications, Proc. 6th Berkeley Symposium on Math. Statist. and Prob., 1, 697

Patterson, 1971, Recovery of inter-block information when block sizes are unequal, Biometrika, 58, 545, 10.1093/biomet/58.3.545

Raiffa, 1961, Applied Statistical Decision Theory.

Rao, 1965, Linear Statistical Inference and its Applications.

Rubin, 1974, Characterizing the estimation of parameters in incomplete-data problems, J. Amer. Statist. Assoc, 69, 467, 10.1080/01621459.1974.10482976

Rubin, 1976, Inference and missing data, Biometrika, 63, 581, 10.1093/biomet/63.3.581

Sundberg, 1974, Maximum likelihood theory for incomplete data from an exponential family, Scand. J. Statist., 1, 49

Sundberg, 1976, An iterative method for solution of the likelihood equations for incomplete data from exponential families, Comm. Statist–Simula. Computa., B5, 55, 10.1080/03610917608812007

Turnbull, 1974, Nonparametric estimation of a survivorship function with doubly censored data, J. Amer. Statist. Assoc, 69, 169, 10.1080/01621459.1974.10480146

Turnbull, 1976, The empirical distribution function with arbitrarily grouped, censored and truncated data, J. R. Statist. Soc., B, 38, 290, 10.1111/j.2517-6161.1976.tb01597.x

Wolfe, 1970, Pattern clustering by multivariate mixture analysis, Multivariate Behavioral Research, 5, 329, 10.1207/s15327906mbr0503_6

Woodbury, 1971, Discussion of paper by Hartley and Hocking, Biometrics, 27, 808