#### Title

Estimating the probability that a simulated system will be the best

#### Abbreviated Journal Title

Nav. Res. Logist.

#### Keywords

multinomial; ranking and selection; simulation; Operations Research & Management Science

#### Abstract

Consider a stochastic simulation experiment consisting of v independent vector replications consisting of an observation from each of k independent systems. Typical system comparisons are based on mean (long-run) performance. However, the probability that a system will actually be the best is sometimes more relevant, and can provide a very different perspective than the systems' means. Empirically, we select one system as the best performer (i.e., it wins) on each replication. Each system has an unknown constant probability of winning on any replication and the numbers of wins for the individual systems follow a multinomial distribution. Procedures exist for selecting the system with the largest probability of being the best. This paper addresses the companion problem of estimating the probability that each system will be the best. The maximum likelihood estimators (MLEs) of the multinomial cell probabilities for a set of v vector replications across k systems are well known. We use these same c vector replications to form v(k) unique vectors (termed pseudo-replications) that contain one observation from each system and develop estimators based on AVC (All Vector Comparisons). In other words, we compare every observation from each system with every combination of observations from the remaining systems and note the best performer in each pseudo-replication. AVC provides lower variance estimators of the probability that each system will be the best than the MLEs. We also derive confidence intervals for the AVC point estimators, present a portion of an extensive empirical evaluation and provide a realistic example. (C) 2002 Wiley Periodicals, Inc.

#### Journal Title

Naval Research Logistics

#### Volume

49

#### Issue/Number

4

#### Publication Date

1-1-2002

#### Document Type

Article

#### DOI Link

#### Language

English

#### First Page

341

#### Last Page

358

#### WOS Identifier

#### ISSN

0894-069X

#### Recommended Citation

"Estimating the probability that a simulated system will be the best" (2002). *Faculty Bibliography 2000s*. 3366.

http://stars.library.ucf.edu/facultybib2000/3366

## Comments

Authors: contact us about adding a copy of your work at STARS@ucf.edu