On Biases and Heterogeneity in Value of Information Analysis Using Individual-Level Models

Abstract:  Health technology assessment (HTA) is used to allocate healthcare resources and inform evidence-based health policy and decision-making. Evidence used in HTA is often obtained from empirical studies, such as randomized control trials (RCTs) or observational studies.1 Current decision making at a population level traditionally focused on an ‘average patient’ using average treatment effect (ATE) estimates that may be suboptimal for two key reasons: (1) heterogeneity that may be explained by systematic differences between individuals or groups of individuals arising from patient characteristics, preferences, and/or contextual factors; and (2) poor transferability and/or relevance of existing evidence to the decision-making context or setting. In other words, the treatment effect estimated from these studies is biased, meaning that it is systematically different from the true value in the population of interest, resulting in a considerable opportunity cost routinely left unclear.

Policymakers are recently interested in making decisions for subgroups of patients. Furthermore, non-rigorous or poorly transferable research is rarely accounted for in real-world HTA and has yet to receive the attention it merits. Accordingly, future studies need to be adequately designed to address these two key issues. For (1), the variation in individual outcomes can be partly explained by conditioning on a set of observed predictors of an outcome of interest. For (2), the presence of different sources of bias may be reduced through an adequate design of new research.

In my talk, I will describe how to account for both heterogeneity and biases in economic evaluations using discrete-event simulation (modeling) techniques and quantify the opportunity cost of study biases stemming from non-rigorous or poorly transferable research.