Confounded Coefficients: Extending Recent Advances in the Accurate Comparison of Logit and Probit Coefficients Across Groups

The logit and probit models are critical parts of the social scientists analytical arsenal. We often want to know if a covariate has the same effect for different groups, e.g., men and women. Unfortunately, many attempts to compare the effect of covariates across groups make the unwarranted assumption that each group has the same residual variation. If this assumption is false, comparisons of coefficients can reveal differences where none exist and conceal differences that do exist. Recent work has emphasized the theoretical potential for this problem and proposed a test of whether the effect of covariates differs across groups that is accurate, if limited, despite differences in residual variation. This paper extends these advances in three ways. First, it uses simulations to show that this theoretical problem is substantively significant under a wide range of common conditions, meaning that traditionally executed comparisons of logit coefficients should be viewed skeptically. Second, it uses simulations to assess the power of the test recently proposed to overcome the problem, finding that they are an improvement over naive comparisons of coefficients, but have significant limitations. Third, it proposes and tests two alternative means of comparing coefficients across groups that avoid the assumption of equal residual variation entirely. The article closes with implications for the practice of research.