Keywords

Homogeneity Test, ROC Curve, AUC, Face Recognition, Covariate, Correlation

Abstract

The Receiver Operating Characteristic (ROC) curve is used to measure the classification accuracy of tests that yield ordinal or continuous scores. Ordinal scores are common in medical imaging studies and, more recently, in black-box studies on forensic identification accuracy (Phillips et al., 2018). To assess the accuracy of radiologists in medical imaging studies or the accuracy of forensic examiners in biometric studies, one needs to estimate the ROC curves from the ordinal scores and account for the covariates related to the radiologists or forensic examiners. In this thesis, we propose a homogeneity test to compare the performance of raters. We derive the asymptotic properties of estimated ROC curves and their corresponding Area Under the Curve (AUC) within an ordinal regression framework. Moreover, we investigate differences in ROC curves (and AUCs) among examiners in detail. We construct confidence intervals for the difference in AUCs and confidence bands for the difference in ROC curves for performance comparison purposes. First, we conduct simulations on data where scores are assumed to be normally distributed, and the features include both categorical and continuous covariates. Then, we apply our procedure to facial recognition data to compare forensic examiners.

The second part of this thesis addresses the correlation of decision scores among raters. In medical imaging studies and facial recognition, multiple raters assess the same subject pairs, leading to potential score correlations. Because of these correlated scores, standard methods for generalized linear models cannot be directly applied to estimate accuracy. In this thesis, we employ the generalized estimating equation to estimate covariate-specific and covariate-adjusted AUC values when correlations are present in ordinal scores. We conduct homogeneity tests on both covariate-specific and covariate-adjusted AUCs, investigating their statistical properties. To assess the finite sample properties of the test, we conduct simulation studies. Furthermore, we apply this test to real facial recognition data.

Completion Date

2023

Semester

Fall

Committee Chair

Tang, Liansheng

Degree

Doctor of Philosophy (Ph.D.)

College

College of Sciences

Department

Statistics and Data Science

Degree Program

Big Data Analytics

Format

application/pdf

Language

English

Release Date

6-15-2024

Length of Campus-only Access

None

Access Status

Doctoral Dissertation (Open Access)

Campus Location

Orlando (Main) Campus

Download

Share

COinS