Title

Acceleration of the EM algorithm by using quasi-Newton methods

Authors

Authors

M. Jamshidian;R. I. Jennrich

Comments

Authors: contact us about adding a copy of your work at STARS@ucf.edu

Abbreviated Journal Title

J. R. Stat. Soc. Ser. B-Methodol.

Keywords

Aitken acceleration; conjugate gradient; covariance estimation; incomplete data; multivariate normal mixtures; Poisson mixtures; observed information matrix; MAXIMUM-LIKELIHOOD; CONVERGENCE; MATRICES; Statistics & Probability

Abstract

The EM algorithm is a popular method for maximum likelihood estimation. Its simplicity in many applications and desirable convergence properties make it very attractive. Its sometimes slow convergence, however, hats prompted researchers to propose methods to accelerate it. We review these methods, classifying them into three groups: pure, hybrid and EM-type accelerators. We propose a new pure and a new hybrid accelerator both based on quasi-Newton methods and numerically compare these and two other quasi-Newton accelerators. For this we use examples in each of three areas: Poisson mixtures, the estimation of covariance from incomplete data and multivariate normal mixtures. In these comparisons, the new hybrid accelerator was fastest on most of the examples and often dramatically so. In some cases it accelerated the EM algorithm by factors of over 100. The new pure accelerator is very simple to implement and competed well with the other accelerators. It accelerated the EM algorithm in some cases by factors of over 50. To obtain standard errors, we propose to approximate the inverse of the observed information matrix by using auxiliary output from the new hybrid accelerator. A numerical evaluation of these approximations indicates that they may be useful at least for exploratory purposes.

Journal Title

Journal of the Royal Statistical Society Series B-Methodological

Volume

59

Issue/Number

3

Publication Date

1-1-1997

Document Type

Article

Language

English

First Page

569

Last Page

587

WOS Identifier

WOS:A1997XD52700003

ISSN

0035-9246

Share

COinS