Research prioritisation in the absence of previous evidence

Abstract: Research prioritisation is the practice of choosing to fund certain research proposals at the expense of not funding others. A key source of uncertainty which research funding bodies aim to address is uncertainty in relative treatment effects e.g. the odds ratio for treatment A vs treatment B. To avoid selection bias, resolving uncertainty in relative treatment effects often requires randomised controlled trials (RCTs) which are expensive and time consuming to carry out. Quantitative methods such as value of information (VOI) can be used to improve the transparency and accountability of the research prioritisation process. VOI provides an estimate of the expected health benefits gained each year if the uncertainty surrounding a parameter was resolved.

Exactly as for probabilistic sensitivity analysis (PSA), VOI requires that the degree of uncertainty about a parameter is expressed mathematically. In a Bayesian framework, this mathematical expression of uncertainty is known as a prior distribution. This means that to use VOI to understand the value of an RCT, a reasonable prior is required for the relative treatment effect. The typical method to define priors for VOI (and PSA) is to fit a parametric distribution to the confidence interval reported from a previous study or a meta-analysis.

Unfortunately, at the time of commissioning research there is very often no previous published data for relative treatment effects. This should not be surprising as research projects are often proposed because of a lack of data on relative treatment effects. This is fundamental issue in the use of VOI to aid research prioritisation. Despite its importance this issue has received little attention to date.

In this presentation we will outline the use of VOI in research prioritisation, conceptualise the task of research prioritisation in the absence of direct evidence on relative treatment effects, outline different approaches to addressing this problem and describe our ongoing work on a novel empirical approach (meta-epidemiology) to formulate reasonable priors in this context.