Title

Acceleration of the EM algorithm by using quasi-Newton methods

Keywords

Aitken Acceleration; Conjugate Gradient; Covariance Estimation; Incomplete Data; Multivariate Normal Mixtures; Observed Information Matrix; Poisson Mixtures

Abstract

The EM algorithm is a popular method for maximum likelihood estimation. Its simplicity in many applications and desirable convergence properties make it very attractive. Its sometimes slow convergence, however, has prompted researchers to propose methods to accelerate it. We review these methods, classifying them into three groups: pure, hybrid and EM-type accelerators. We propose a new pure and a new hybrid accelerator both based on quasi-Newton methods and numerically compare these and two other quasi-Newton accelerators. For this we use examples in each of three areas: Poisson mixtures, the estimation of covariance from incomplete data and multivariate normal mixtures. In these comparisons, the new hybrid accelerator was fastest on most of the examples and often dramatically so. In some cases it accelerated the EM algorithm by factors of over 100. The new pure accelerator is very simple to implement and competed well with the other accelerators. It accelerated the EM algorithm in some cases by factors of over 50. To obtain standard errors, we propose to approximate the inverse of the observed information matrix by using auxiliary output from the new hybrid accelerator. A numerical evaluation of these approximations indicates that they may be useful at least for exploratory purposes. © 1997 Royal Statistical Society.

Publication Date

1-1-1997

Publication Title

Journal of the Royal Statistical Society. Series B: Statistical Methodology

Volume

59

Issue

3

Number of Pages

569-587

Document Type

Article

Personal Identifier

scopus

DOI Link

https://doi.org/10.1111/1467-9868.00083

Socpus ID

0009568899 (Scopus)

Source API URL

https://api.elsevier.com/content/abstract/scopus_id/0009568899

This document is currently not available here.

Share

COinS