Classification Accuracy Of Mixed Format Tests: A Bi-Factor Item Response Theory Approach

Keywords

bi-factor model; classification accuracy; constructed response items; item response theory; mixed format test

Abstract

Mixed format tests (e.g., a test consisting of multiple-choice [MC] items and constructed response [CR] items) have become increasingly popular. However, the latent structure of item pools consisting of the two formats is still equivocal. Moreover, the implications of this latent structure are unclear: For example, do constructed response items tap reasoning skills that cannot be assessed with multiple choice items? This study explored the dimensionality of mixed format tests by applying bi-factor models to 10 tests of various subjects from the College Board's Advanced Placement (AP) Program and compared the accuracy of scores based on the bi-factor analysis with scores derived from a unidimensional analysis. More importantly, this study focused on a practical and important question—classification accuracy of the overall grade on a mixed format test. Our findings revealed that the degree of multidimensionality resulting from the mixed item format varied from subject to subject, depending on the disattenuated correlation between scores from MC and CR subtests. Moreover, remarkably small decrements in classification accuracy were found for the unidimensional analysis when the disattenuated correlations exceeded 0.90.

Publication Date

2-29-2016

Publication Title

Frontiers in Psychology

Volume

7

Document Type

Article

Personal Identifier

scopus

DOI Link

https://doi.org/10.3389/fpsyg.2016.00270

Socpus ID

85091885228 (Scopus)

Source API URL

https://api.elsevier.com/content/abstract/scopus_id/85091885228

This document is currently not available here.

Share

COinS