Bias in Balance Optimization Subset Selection: Exploration through examples

When estimating a treatment effect from observational data, researchers encounter bias regardless of estimation methods. In this paper, we focus on a particular method of estimation called Balance Optimization Subset Selection (BOSS). This paper investigates all the possible cases that may lead to bias in the context of BOSS, provides examples for those cases and tries to mitigate the bias. While doing so, we define a balance hierarchy and a correct imbalance measure which corresponds to the form of the response functions. In addition, new imbalance measures drawn from the Cramer-von Mises test statistic are introduced. The cases of insufficient data and suboptimality that can arise in causal analysis with BOSS are also presented.