Modelling healthcare performance indicators and identifying ‘unusual’ performance

Abstract: Measures of the performance of healthcare providers are often collected routinely at regular intervals over time, examples including mortality rates following surgery and Methicillin-resistant Staphylococcus aureus (MRSA) rates. As well as monitoring overall trends and planning resources, it is of interest to identify individual providers that are potentially unusual, for example any with notably high or low event rates during a particular time period, or that have experienced recent changes. As there are often large numbers of healthcare providers in such data sets, this procedure should be automated and the inherent multiple testing problem must be accounted for. Hierarchical models for performance indicators, while having several attractive properties, also present the additional difficulty that it is unclear what exactly constitutes ‘unusual’ based on such a model. Further, in testing for recent changes in performance we may be concerned about the phenomenon of regression-to-the-mean, whereby extreme observations in one time period will tend to move back towards the average in the next. I will describe our approaches to overcoming these statistical challenges. In addition, I will describe some options for extending hierarchical models to a longitudinal setting in this context, allowing us to model performance measures from multiple time points simultaneously.