Attributing effects to a cluster-randomized get-out-the-vote Campaign

Early in the twentieth century, Fisher and Neyman demonstrated how to infer effects of agricultural interventions using only the very weakest of assumptions, by randomly varying which plots were to be manipulated. Although the methods permitted uncontrolled variation between experimental units, they required strict control over assignment of interventions; this hindered their application to field studies with human subjects, who ordinarily could not be compelled to comply with experimenters' instructions. In 1996, however, Angrist, Imbens, and Rubin showed that inferences from randomized studies could accommodate noncompliance without significant strengthening of assumptions. Political scientists A. Gerber and D. Green responded quickly, fielding a randomized study of voter turnout campaigns in the November 1998 general election. Noncontacts and refusals were frequent, but Gerber and Green analyzed their data in the style of Angrist et al., avoiding the need to model nonresponse. They did use models for other purposes: to address complexities of the randomization scheme; to permit heterogeneity among voters and campaigners; to account for deviations from experimental protocol; and to take advantage of highly informative covariates. Although the added assumptions seemed straightforward and unassailable, a later analysis by Imai found them to be at odds with Gerber and Green's data. Using a different model, he reaches the very opposite of Gerber and Green's central conclusion about getting out the vote. This article shows that neither of the models are necessary, addressing all of the complications of Gerber and Green's study using methods in the tradition of Fisher and Neyman. To do this, it merges recent developments in randomization-based inference for comparative studies with somewhat older developments in design-based analysis of sample surveys. The method involves regression, but large-sample analysis and simulations demonstrate its lack of dependence on regression assumptions. Its substantive results have consequences both for the design of campaigns to increase voter participation and for theories of political behavior more generally.