Title
Acceleration of the EM algorithm by using quasi-Newton methods
Abbreviated Journal Title
J. R. Stat. Soc. Ser. B-Methodol.
Keywords
Aitken acceleration; conjugate gradient; covariance estimation; incomplete data; multivariate normal mixtures; Poisson mixtures; observed information matrix; MAXIMUM-LIKELIHOOD; CONVERGENCE; MATRICES; Statistics & Probability
Abstract
The EM algorithm is a popular method for maximum likelihood estimation. Its simplicity in many applications and desirable convergence properties make it very attractive. Its sometimes slow convergence, however, hats prompted researchers to propose methods to accelerate it. We review these methods, classifying them into three groups: pure, hybrid and EM-type accelerators. We propose a new pure and a new hybrid accelerator both based on quasi-Newton methods and numerically compare these and two other quasi-Newton accelerators. For this we use examples in each of three areas: Poisson mixtures, the estimation of covariance from incomplete data and multivariate normal mixtures. In these comparisons, the new hybrid accelerator was fastest on most of the examples and often dramatically so. In some cases it accelerated the EM algorithm by factors of over 100. The new pure accelerator is very simple to implement and competed well with the other accelerators. It accelerated the EM algorithm in some cases by factors of over 50. To obtain standard errors, we propose to approximate the inverse of the observed information matrix by using auxiliary output from the new hybrid accelerator. A numerical evaluation of these approximations indicates that they may be useful at least for exploratory purposes.
Journal Title
Journal of the Royal Statistical Society Series B-Methodological
Volume
59
Issue/Number
3
Publication Date
1-1-1997
Document Type
Article
Language
English
First Page
569
Last Page
587
WOS Identifier
ISSN
0035-9246
Recommended Citation
"Acceleration of the EM algorithm by using quasi-Newton methods" (1997). Faculty Bibliography 1990s. 1953.
https://stars.library.ucf.edu/facultybib1990/1953
Comments
Authors: contact us about adding a copy of your work at STARS@ucf.edu