Abstract

Selection effects systematically attenuate correlations and must be considered when performing meta-analyses. No research domain is immune to selection effects, evident whenever self-selection or attrition take place. In educational research, selection effects are unavoidable in studies of postsecondary admissions, placement testing, or teacher selection. While methods to correct for selection bias are well documented for univariate meta-analyses, they have gone unexamined in multivariate meta-analyses, which synthesize more than one correlation from each study (i.e., a correlation matrix). Multivariate meta-analyses of correlations provide opportunities to explore complex relationships and correcting for selection effects improves the summary effect estimates. I used Monte Carlo simulations to test two methods of correcting selection effects and evaluate a method for pooling the corrected matrices. First, I examined the performance of Thorndike's corrections (for both explicit and incidental selection) and Lawley's multivariate correction for selection on correlation matrices when explicit selection takes place on a single variable. Simulation conditions included a wide range of selection ratios, samples sizes, and population correlations. The results indicated that univariate and multivariate correction methods perform equivalently. I provide practical guidelines for choosing between the two methods. In a second Monte Carlo simulation, I examined the confidence interval coverage rates of a Robust Variance Estimation (RVE) procedure when it is used to pool correlation matrices corrected for selection effects under a random-effects model. The RVE procedure empirically estimates the standard errors of the corrected correlations and has the advantage of having no distributional assumptions. Simulation conditions included tau-squared ratio, within-study sample size, number of studies, and selection ratio. The results were mixed, with RVE performing well under higher selection ratios and larger unrestricted sample sizes. RVE performed consistently across values of tau-squared. I recommend applications of the results, especially for educational research, and opportunities for future research.

Notes

If this is your thesis or dissertation, and want to learn how to access it or for more information about readership statistics, contact us at STARS@ucf.edu

Graduation Date

2019

Semester

Summer

Advisor

Sivo, Stephen

Degree

Doctor of Philosophy (Ph.D.)

College

College of Community Innovation and Education

Department

School of Teacher Education

Degree Program

Education; Methodology, Measurement and Analysis

Format

application/pdf

Identifier

CFE0007680

URL

http://purl.fcla.edu/fcla/etd/CFE0007680

Language

English

Release Date

8-15-2022

Length of Campus-only Access

3 years

Access Status

Doctoral Dissertation (Open Access)

Download

Share

COinS