Keywords

Standard Setting, Nominal Response Model, Two-Parameter Logistic IRT, Embedded Standard Setting

Abstract

The validity of score interpretation and application for specific purposes is dependent upon both the appropriate assignment of test scores and the methods through which those scores are given meaning by defining performance benchmarks. This study seeks to examine the combined utility of various scoring (i.e., number correct, two-parameter logistic, and nominal response model) and standard setting methods in accurately predicting performance on an external assessment. The study will highlight the implementation of the embedded standard setting method, a newer standard setting procedure that is not yet well represented in the literature. Embedded standard setting will be compared to a variation of the bookmark method and to variation of norm-referenced standard setting, the equi-distribution method.

The results of the study suggest that each of the combined approach performance levels, which were evaluated separately in respective multilevel logistic regression analyses, were statistically significant predictors of performance on the statewide assessment as defined by whether a student passed or did not pass the assessment. Each model estimation explained additional variance in the overall outcome, as suggested by the pseudo-R square values, over the covariates only model that was used as a reference for comparison. Outcomes of the regression analyses for each scoring and standard setting combined approach, including statistical significance of the predictors, model fit, and classification accuracy, are presented and discussed.

Completion Date

2024

Semester

Fall

Committee Chair

Sivo, Stephen

Degree

Doctor of Philosophy (Ph.D.)

College

College of Community Innovation and Education

Department

Learning Sciences and Educational Research

Degree Program

Methodology, Measurement & Analysis

Format

PDF

Identifier

DP0029024

Language

English

Release Date

12-15-2024

Access Status

Dissertation

Campus Location

Orlando (Main) Campus

Accessibility Status

PDF accessibility verified using Adobe Acrobat Pro Accessibility Checker

Share

COinS