The synthetic control method compared to difference-in differences for evaluating health policies

Abstract: This paper contrasts the synthetic control method with difference-in-differences (DiD) estimation for the evaluation of health policies. The synthetic control method aims to estimate treatment effects by constructing a weighted combination of control units, which represents what the treated group would have experienced in the absence of receiving the treatment. While DiD estimation assumes parallel trends in the outcomes between observations in the treated and control groups, the synthetic control method allows for these trends to differ. We extend the synthetic control approach to  settings where there are multiple treated units. We re-evaluate the findings from a recent study of the effects of a hospital pay-for-performance (P4P) scheme on risk-adjusted hospital mortality. In contrast to the original DiD analysis, the synthetic control method reports that, for the incentivised conditions, the P4P scheme did not significantly reduce mortality, and that there is a statistically significant increase in mortality for non-incentivised conditions.