The many faces of multiplicity in HTA

Abstract: I will consider the many areas of Health Technology Assessment where the problem of multiplicity arises. Multiplicity arises when we have a number of candidate statistical analyses and we use of the same data set for model selection and model estimation. This can lead to both biased estimates and an under-estimate of the uncertainty in these biased estimates. Multiplicity can occur when we select relevant patient subgroups for analysis, structural forms and co-variable sets for statistical models, and data sets for evidence synthesis.

I will review traditional solutions such as a requirement for pre-specification, traditional methods for correcting inference (such as the Bonferroni correction), and a requirement for biological plausibility, or simple regarding certain analyses as ‘exploratory’.

I will then look at the potential role of wider evidence synthesis and Bayesian methods in addressing some of the limitations of traditional solutions. Importantly, Bayesian methods can potentially take account of both the decision process underlying the selection of candidate analyses and the mechanism by which the final analysis (for the purposes of decision-making) is selected.