Title
A Cautionary Note on Modeling Multitrait-Multirater Data Arising From Ill-Structured Measurement Designs
Abbreviated Journal Title
Organ. Res. Methods
Keywords
confirmatory factor analysis; job performance ratings; measurement; design; MULTISOURCE PERFORMANCE RATINGS; COVARIANCE STRUCTURE-ANALYSIS; CROSS-CLASSIFIED DATA; JOB-PERFORMANCE; MEASUREMENT EQUIVALENCE; MULTIMETHOD MATRIX; CONSTRUCT-VALIDITY; RATER; METAANALYSIS; SUBORDINATE; Psychology, Applied; Management
Abstract
The authors illustrate a problem with confirmatory factor analysis (CFA)-based strategies to model disaggregated multitrait-multirater (MTMR) data-the potential to find markedly different results with the same sample of ratees simply as a result of how one selects and identifies raters within the data set one has gathered for analysis. Using performance ratings gathered as part of a large criterion-related validation study, the authors show how such differences manifest themselves in several ways including variation in (a) covariance matrices that serve as input for the modeling effort, (b) model convergence, (c) admissibility of solutions, (d) overall model fit, (e) model parameter estimates, and (f) model selection. Implications of this study for past research and recommendations for future CFA-based MTMR modeling efforts are discussed.
Journal Title
Organizational Research Methods
Volume
14
Issue/Number
3
Publication Date
1-1-2011
Document Type
Article
Language
English
First Page
503
Last Page
529
WOS Identifier
ISSN
1094-4281
Recommended Citation
"A Cautionary Note on Modeling Multitrait-Multirater Data Arising From Ill-Structured Measurement Designs" (2011). Faculty Bibliography 2010s. 1782.
https://stars.library.ucf.edu/facultybib2010/1782
Comments
Authors: contact us about adding a copy of your work at STARS@ucf.edu