Illusive benefits and treatment effect heterogeneity in randomised experiments

Abstract: Prespecified subgroup analyses of randomised studies aim to assess whether individual characteristics moderate or mediate the treatment effect. Identifying baseline covariates in which treatment response may differ markedly from the population average within randomised experiments is of interest to policy makers but also challenging. Conventional statistical and econometric methods can result in simplistic and spurious estimates of treatment effect heterogeneity. We examine how Bayesian hierarchical linear models with a sceptical prior can be used to identify individuals who benefit most from aparticular treatment while accounting for multiplicity and statistical power. The use of sceptical priors ensures evidence of large heterogeneous effects are unlikely but also does not rule out the possibility that they may exist through the posterior distribution. Our analysis uses data from large multicentre randomised controlled trials in stroke to classify individuals into more credible subgroups based on a decision rule of three possible class memberships-responders, non-responders as well as an uncertain responder group in which further evidence is needed to inform clinical decision making. We will explore whether key baseline covariates used to determine credible subgroups simply shift the central tendency of the outcome distributionusing flexible regression methods to derive more individualised cost-effectiveness analyses.