Economic Evaluation seminar - Taking account of uncertainty in recommendations based on Network Meta-Analysis: Loss-adjusted Expected Value, and other methods
This event has now finished.
Event details
Abstract:
In theory, a risk-neutral decision maker should select the single treatment with the highest Expected Value (EV) on the objective function, regardless of uncertainty. In practice, guideline developers often recommend more than one treatment and sometimes take uncertainty into account, although in ad hoc and inconsistent ways.
New methods for forming recommendations based on probabilistic treatment rankings have been proposed by NMA methodologists, and the GRADE Working Group has also developed its own method. These proposals have the potential to recommend multiple treatments, and to penalise uncertainty, making them suitable for risk-averse decision-makers.
We introduce another method, Loss-adjusted Expected Value (LaEV), derived from Bayesian decision theory, and use a series of stylized examples to compare LaEV to the other methods discussed in the NMA literature. We define properties that a ranking system must have to be valid under uncertainty, and set out a number of other properties that metrics should have to make them useful for decision making. A two-stage process is proposed for both EV- and LaEV-based decisions: the first identifies treatments superior to the reference treatment; the second identifies those that are also within a Minimal Clinically Important Difference (MCID) of the best treatment. We apply EV, LaEV and GRADE methods to 10 NMAs used in NICE Guidelines.
Only LaEV reliably delivers valid rankings under uncertainty and has all the required properties. In 10 NMAs comparing between 5 and 41 treatments, an EV decision maker would recommend 4-14 treatments, and LaEV 0-3 (median 2) fewer. GRADE rules give rise to anomalies, and can privilege the most uncertainty treatments. Neither GRADE rules nor probabilistic rankings are able to generate rational decisions, as a choice between two treatments A and B could depend on the quality of a third treatment C.
A two-stage approach based on MCID ensures that EV- and LaEV-based rules recommend a clinically appropriate number of treatments. For a risk-averse decision maker, LaEV is conservative, simple to implement, and principled.
If you are not a member of University of York staff and are interested in attending a seminar, please contact alfredo.palacios
About the speaker
Tony Ades, Population Health Sciences, Bristol University Medical School
Tony’s first degree was in Experimental Psychology from University of Sussex. He went on to the Massachusetts Institute of Technology on a NATO studentship, and returned to the UK in 1974 with a PhD in speech perception. After a period at the Max Planck Institute in Nijmegen, working on theoretical linguistics, he decided to be a statistician in 1982. He worked with Catherine Peckham at the Institute of Child Health London for 15 years on infections of the mother, fetus and newborn, and on national surveillance programmes for HIV. Tony moved to Bristol in 2000, where he developed a programme of research on Multi-Parameter Evidence Synthesis (MPES) in epidemiology and decision making. He was a member of the NICE appraisals committees 2003-2013. In 2011 he received a lifetime achievement award from the Society of Research Synthesis Methods. Tony ‘retired’ in 2023. The MPES programme has expanded under Nicky Welton and Hayley Jones' leadership, with funding from MRC and NIHR. Tony continues to work with his former colleagues on evidence synthesis methods and decision making.