TY - JOUR
T1 - Permutation test for heterogeneous treatment effects with a nuisance parameter
AU - Chung, Eun Yi
AU - Olivares, Mauricio
N1 - Funding Information:
We are very grateful to the Associate Editor and two referees for their careful and detailed comments that led to considerable improvement of the paper. We would also like to thank Roger Koenker, Jose Luis Montiel-Olea, Joseph Romano, and seminar participants at various institutions for useful comments and feedback on this paper.
Publisher Copyright:
© 2021 Elsevier B.V.
PY - 2021/12
Y1 - 2021/12
N2 - This paper proposes an asymptotically valid permutation test for heterogeneous treatment effects in the presence of an estimated nuisance parameter. Not accounting for the estimation error of the nuisance parameter results in statistics that depend on the particulars of the data generating process, and the resulting permutation test fails to control the Type 1 error, even asymptotically. In this paper we consider a permutation test based on a martingale transformation of the empirical process to render an asymptotically pivotal statistic, effectively nullifying the effect associated with the estimation error on the limiting distribution of the statistic. Under weak conditions, we show that the permutation test based on the martingale-transformed statistic results in the asymptotic rejection probability of α in general while retaining the exact control of the test level when testing for the more restrictive sharp null. We also show how our martingale-based permutation test extends to testing whether there exists treatment effect heterogeneity within subgroups defined by observable covariates. Our approach comprises testing the joint null hypothesis that treatment effects are constant within mutually exclusive subgroups while allowing the treatment effects to vary across subgroups. Monte Carlo simulations show that the permutation test presented here performs well in finite samples, and is comparable to those existing in the literature. To gain further understanding of the test to practical problems, we investigate the gift exchange hypothesis in the context of two field experiments from Gneezy and List (2006). Lastly, we provide the companion RATestR package to facilitate and encourage the application of our test in empirical research.
AB - This paper proposes an asymptotically valid permutation test for heterogeneous treatment effects in the presence of an estimated nuisance parameter. Not accounting for the estimation error of the nuisance parameter results in statistics that depend on the particulars of the data generating process, and the resulting permutation test fails to control the Type 1 error, even asymptotically. In this paper we consider a permutation test based on a martingale transformation of the empirical process to render an asymptotically pivotal statistic, effectively nullifying the effect associated with the estimation error on the limiting distribution of the statistic. Under weak conditions, we show that the permutation test based on the martingale-transformed statistic results in the asymptotic rejection probability of α in general while retaining the exact control of the test level when testing for the more restrictive sharp null. We also show how our martingale-based permutation test extends to testing whether there exists treatment effect heterogeneity within subgroups defined by observable covariates. Our approach comprises testing the joint null hypothesis that treatment effects are constant within mutually exclusive subgroups while allowing the treatment effects to vary across subgroups. Monte Carlo simulations show that the permutation test presented here performs well in finite samples, and is comparable to those existing in the literature. To gain further understanding of the test to practical problems, we investigate the gift exchange hypothesis in the context of two field experiments from Gneezy and List (2006). Lastly, we provide the companion RATestR package to facilitate and encourage the application of our test in empirical research.
KW - Heterogeneous treatment effect
KW - Martingale transformation
KW - Multiple hypothesis testing
KW - Permutation test
KW - Westfall–Young
UR - http://www.scopus.com/inward/record.url?scp=85108508074&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85108508074&partnerID=8YFLogxK
U2 - 10.1016/j.jeconom.2020.09.015
DO - 10.1016/j.jeconom.2020.09.015
M3 - Article
AN - SCOPUS:85108508074
SN - 0304-4076
VL - 225
SP - 148
EP - 174
JO - Journal of Econometrics
JF - Journal of Econometrics
IS - 2
ER -