Elicitation

Overview
Publications

Overview

In informing decisions, expert elicitation can provide valuable information, particularly where evidence is missing, where it may not be as well developed (e.g. diagnostics, medical devices, early access to medicines scheme or public health) or limited (insufficient, not very relevant, contradictory and/or flawed). Here, formal method to elicit expert judgements are preferred to improve the accountability and transparency of the decision making process, additionally to the important role in reducing bias and the use of heuristics.

Elicitation is the process of transforming the subjective and implicit knowledge of experts into their quantifiable expressions. The use of expert elicitation in HTA has increased in recent years, however methodology is heterogeneous and there are no published guidelines for analysts to follow.

This project aims to establish a reference protocol for the elicitation of (distributions of) experts’ judgements to inform health care decision making, focussing on the defining characteristics of health care decisions making. This protocol will bring clarity over the methods for collecting and using experts’ judgements within the assessment of cost-effectiveness and will also demonstrate where alternative methodology may be required in a particular context and constraints such as time and funding.

Researchers at the University of York are involved in the ISPOR task force on Structured Expert Elicitation for Healthcare Decision Making

For resources for conducting structured expert elicitation, visit STEER

Publications

2022

Jankovic D, Payne K, Kanaan M, Bojke L. Eliciting uncertainty for complex parameters in model-based economic evaluations: quantifying a temporal change in the treatment effect. International Journal of Technology Assessment in Health Care 2022;18:38(1).

2021

Bojke L, Soares M, Claxton K, Colson A, Fox A, Jackson C, Jankovic D, Morton A, Sharples L, Taylor A. Reference case methods for expert elicitation in health care decision making. Medical Decision Making 2022;42(2):182-193.

Bojke L, Soares M, Claxton K, Colson A, Fox A, Jackson C, Jankovic D, Morton A, Sharples L, Taylor A. Developing a reference protocol for structured expert elicitation in health-care decision-making: a mixed-methods study. Health Technology Assessment 2021;25(37):1-124.

2020

Soares M, Sculpher M, Claxton K. Health opportunity costs: Assessing the implications of uncertainty using elicitation methods with experts. Medical Decision Making 2020;40(4):448-459.

Eliciting experts judgements in HTA. Report commissioned for NICE methods update (PDF , 1,411kb)

Supplementary material survey results (PDF , 2,586kb)

2018

Soares M, Bojke L. Expert elicitation to inform Health Technology Assessment. In Dias LC, Morton A, Quigley J (eds). Elicitation: The Science and Art of Structuring Judgement. Springer International Publishing 2018;Chapter 18:pp479-494.

Soares M, Sharples L, Morton A, Claxton K, Bojke L. Experiences of structured elicitation for model based cost-effectiveness analyses. Value in Health 2018;21(6):715-723.

2017

Bojke L, Grigore B, Jankovic D, Peters J, Soares M, Stein K. Informing reimbursement decisions using cost-effectiveness modelling: a guide to the process of generating elicited priors to capture model uncertainties. Pharmacoeconomics 2017;35(9):867-877.

2014

Bojke L, Soares M. Decision analysis: eliciting experts’ beliefs to characterise uncertainties. In:Culyer AJ (ed.) Encyclopedia of Health Economics, Vol 1. San Diego:Elsevier;2014:pp149-154.

Haakma W, Steuten LM, Bojke L, IJzerman MJ. Belief elicitation to populate health economic models of medical diagnostic devices in development. Applied Health Economics and Health Policy 2014;12(3):327-34. Available from springer

2011

Soares MO, Bojke L, Dumville J, Iglesias C, Callum N, Claxton K. Methods to elicit experts' beliefs over uncertain quantities: application to a cost effectiveness transition model of negative pressure wound therapy for severe pressure ulceration. Statistics in Medicine 2011;30(19):2363-2380. Available from wiley

2010

Bojke L, Claxton K, Bravo Vergel Y, Sculpher M, Palmer S, Abrams K. Eliciting distributions to populate decision analytic models. Value in Health 2010;13:557-64. Availalbe from wiley