Title

A Comparison of a Subjective and Statistical Method for Establishing Score Comparability in an Organizational Culture Survey

Authors

Authors

N. T. Carter; L. M. Kotrba; D. L. Diab; B. C. Lin; S. Y. Pui; C. J. Lake; M. A. Gillespie; M. J. Zickar;A. Chao

Comments

Authors: contact us about adding a copy of your work at STARS@ucf.edu

Abbreviated Journal Title

J. Bus. Psychol.

Keywords

Differential item functioning; Invariance; Language-translated surveys; Multinational organizational surveys; Organizational culture surveys; ITEM RESPONSE THEORY; LIKELIHOOD RATIO TEST; DIFFERENTIAL ITEM; MEASUREMENT EQUIVALENCE; PERSONALITY-INVENTORY; VALIDITY; TESTS; TRANSLATIONS; INSTRUMENTS; ISSUES; Business; Psychology, Applied

Abstract

Purpose The purpose of this study was to compare the results of a subjective and a statistical method of detecting non-comparable items in 14 language-translated forms of a survey measuring organizational culture. Design/Methodology/Approach Data were obtained from a large multinational organization using a 60-item organizational culture survey. Each of 14 language-translated forms were administered to members of their respective language groups and compared to the original English (United States) form. Subjective reviews were conducted using fluent bilingual organizational members whom flagged items they did not believe were comparable. Statistical analyses using an item response theory (IRT) approach were used to detect problematic items from 14 samples with sizes ranging from 304 to 3,014. Detection patterns from these two approaches were compared. Findings The subjective approach identified far less items as problematic and did not agree with the statistical approach. Implications Our results suggest that the subjective approach as a pre-screening adaptation procedure has little added value over a careful translation/back-translation procedure. Originality/Value The use of language-adapted organizational surveys has become increasingly important to multinational organizations. In examining scores from such surveys the establishment of score comparability is essential. IRT analyses are often used to determine whether scores are comparable across language translations. It is also common for subjective reviews of item content to be utilized prior to statistical techniques to determine if the translated items are comparable to the original form. No research known to authors has compared modern statistical and subjective approaches to addressing these issues.

Journal Title

Journal of Business and Psychology

Volume

27

Issue/Number

4

Publication Date

1-1-2012

Document Type

Article

Language

English

First Page

451

Last Page

466

WOS Identifier

WOS:000310812000006

ISSN

0889-3268

Share

COinS