Comparative Regression Discontinuity: A Stress Test With Small Samples

Keywords

causal inference; comparative regression discontinuity; regression discontinuity design; within-study comparison

Abstract

Compared to the randomized experiment (RE), the regression discontinuity design (RDD) has three main limitations: (1) In expectation, its results are unbiased only at the treatment cutoff and not for the entire study population; (2) it is less efficient than the RE and so requires more cases for the same statistical power; and (3) it requires correctly specifying the functional form that relates the assignment and outcome variables. One way to overcome these limitations might be to add a no-treatment functional form to the basic RDD and including it in the outcome analysis as a comparison function rather than as a covariate to increase power. Doing this creates a comparative regression discontinuity design (CRD). It has three untreated regression lines. Two are in the untreated segment of the RDD—the usual RDD one and the added untreated comparison function—while the third is in the treated RDD segment. Also observed is the treated regression line in the treated segment. Recent studies comparing RE, RDD, and CRD causal estimates have found that CRD reduces imprecision compared to RDD and also produces valid causal estimates at the treatment cutoff and also along all the rest of the assignment variable. The present study seeks to replicate these results, but with considerably smaller sample sizes. The power difference between RDD and CRD is replicated, but not the bias results either at the treatment cutoff or away from it. We conclude that CRD without large samples can be dangerous.

Publication Date

2-1-2018

Publication Title

Evaluation Review

Volume

42

Issue

1

Number of Pages

111-143

Document Type

Article

Personal Identifier

scopus

DOI Link

https://doi.org/10.1177/0193841X18776881

Socpus ID

85047930373 (Scopus)

Source API URL

https://api.elsevier.com/content/abstract/scopus_id/85047930373

This document is currently not available here.

Share

COinS