Methodological research 

Overview

Overview

TEEHTA undertakes a range of methodological research in economic evaluation.  Between 2004 and 2009 the team was part of the Health Services Research Collaboration funded by the Medical Research Council.   Methodological research has also been funded by project grants and individual fellowships.  Key themes in TEEHTA's methodological research include:

  • Developments in decision analytic modelling
  • Analytical issues is characterising uncertainty and assessing the value of additional research
  • Methodological research to support aspects of policy and decision making with respect to new medical technologies
  • Statistical issues in economic evaluation
  • Heterogeneity, generalisability and transferability in economic evaluation
  • Economic evaluation in public health

Modelling

Developments in decision analytic modelling 

Methods of extrapolating RCT evidence for economic evaluation 

  • Funder: Medical Research Council Methodology Research Programme (2010-12)
  • Collaborators: Stephen Palmer (York), Andrea Manca (York), Laura Bojke (York), Alan Brennan (Sheffield), Andrew Stevens (Sheffield), Paul Tappenden (Sheffield), Chris Jackson (Cambridge), Keith Abrams (Leicester) 

Economic evaluation is increasingly used to inform policy making decisions concerning the adoption of new healthcare technologies.  The National Institute for Health and Clinical Excellence (NICE) for England and Wales is an example of the type of agency that requires cost-effectiveness information to issue recommendations regarding the likely value for money of the health technologies undergoing the Institute’s review process.  One of the key elements of any such analyses is an appropriate time horizon, which in many circumstances will be the patient’s lifetime.   Most clinical studies though have durations that are shorter than the patient’s lifetime.  To produce cost effectiveness evidence to support healthcare policy decisions the analyst will have to use some form of modelling to extrapolate the trial data beyond the observed follow up period.  Estimation of longer-term or lifetime costs and benefits necessitates extrapolation of evidence beyond the duration of the trial, and involves making choices about the model(s) to use. This research will characterise and critically review methods of extrapolation, explore these using case studies, and recommend best-practice. An important aspect will be a classification of methods used in NICE appraisals and other HTAs, an activity informed by a series of reviews of methods used in health services research and the wider scientific literature. The reviews will help inform a series of case-studies designed to cover a range of issues. The combination of critical review of methods, further exploration and testing within case studies will be used to produce recommendations on best practice for methods of extrapolation, which will be of value to health economists, statisticians and decision makers both in the UK and internationally.  

Analytical issues with computationally expensive models 

Uncertainty

Analytical issues is characterising uncertainty and assessing the value of additional research  

Expected health benefits of additional evidence: principles, methods and applications

VAC therapy for pressure ulcers: value of information analysis and feasibility study 

  • Funder: Medical Research Council
  • Marta Soares, Karl Claxton
  • Publications:
    • Ashby R, Dumville J, Soares M, McGinnis E, Stubbs N, Torgerson D, Cullum N. A pilot randomised controlled trial of negative pressure wound therapy (NPWT) to treat grade III/IV pressure ulcers. Trials 2012;13(119). Download from trialsjournal
    • Soares MO, Dumville J, Ashby R, Iglesias C, Bojke L, Adderley U, McGinnis E, Stubbs N, Torgerson D, Claxton K, Cullum N. Methods to assess cost effectiveness and value of further research when data are sparse: negative pressure wound therapy for severe pressure ulcers. Medical Decision Making 2012;doi:10.1177/0272989X12451058. Download from sagepub  
    • Soares MO, Bojke L, Dumville J, Iglesias C, Callum N, Claxton K. Methods to elicit experts' beliefs over uncertain quantities: application to a cost effectiveness transition model of negative pressure wound therapy for severe pressure ulceration. Statistics in Medicine 2011;30(19):2363-2380. Download from Wiley

Optimal research portfolio design based upon value of information: a decision-theoretic approach 

  • Funder: Medical Research Council Health Services Research Collaboration (2004-9)
  • Stefano Conti, Karl Claxton 
  • Publication: Conti S, Claxton K. Dimensions of design space: a decision theoretic approach to optimal research portfolio design. Medical Decision Making 2009;29:643-60. Download from Sage Journals online 

VOI and sequencing 

The consequences of ignoring decision uncertainty 

The half-life of truth: appropriate time horizons for value of information analysis 

  • Funder: Medical Research Council Health Services Research Collaboration (2004-9)
  • Karl Claxton, Zoe Philips, Steve Palmer
  • Link to: Phillips Z, Claxton K, Palmer S. The half-life of truth? what is the appropriate time horizons for research decisions? Medical Decision Making. 2008;28:287-99. Download form Sage Journals 

Examining the value of information: pre- and post-CRASH (a large randomised control trial) 

The optimal development and efficient regulation of health care technologies: the role of value of information analysis 

An investigation of methods to incorporate irreversibility and timing of decisions into the economic evaluation of health care technologies 

The use of decision analysis as a framework for evidence synthesis 

  • Laura Bojke Doctoral Fellowship (2004-7)
  • Funder: Department of Health, Research Capacity Development Centre
  • Publications:
    • Bojke L, Claxton K, Bravo Vergel Y, Sculpher M, Palmer S, Abrams K. Eliciting distributions to populate decision analytic models. Value in Health 2010;13(5):557-64. Download from Wiley interscience 
    • Bojke L, Claxton K, Sculpher M, Palmer S. Characterizing structural uncertainty in decision-analytic models: a review and application of methods. Value in Health. 2009;12(5):739-49. Download from Wiley interscience
    • Bojke L, Claxton K, Sculpher M, Palmer S, Philips Z. The cost-effectiveness and value for information associated with repeat screening for age related macular degeneration. Medical Decision Making 2008;28:33-43. Download from Sage journals online  

Building a reference case for Bayesian applications to health economics and outcomes research 

Policy

Methodological research to support aspects of policy and decision making with respect to new medical technologies 

Methods for estimation of the NICE cost-effectiveness threshold

Study summary:

Cost-effectiveness analysis used by NICE is essentially an assessment of whether the health expected to be gained from the use of a new medical technology exceeds the health likely to be forgone as other NHS activities are displaced to accommodate the additional costs of the new technology. The cost-effectiveness threshold represents an estimate of the health forgone as services are displaced. Currently the threshold used by NICE has little empirical basis. The aim of this new research project is to develop and to demonstrate methods for threshold estimation which make best use of routinely available NHS data, allowing scrutiny by a range of stakeholders, improving accountability and predictability. This research will focus on complementary methods which can make best use of those data that are already available, where there are already plans to make data available or where additional data could feasibly be made available at reasonable cost.

A web cast of a presentation of this research can be found at https://vimeo.com/57156015

CHE Research Paper 81 (PDF , 3,944kb)

How much and what type of health is lost (MS Excel , 275kb)

Further details of project Threshold project - case for support (PDF , 128kb) and details of the funding call from the MRC ‌‌‎

Workshop held at Imperial College London 9th May 2011. Further details

Session at the International Health Economics Association meeting, Toronto, 13th July 2011. Further details

Draft Final Report (PDF , 1,235kb)

Appendix A (PDF , 404kb)

Appendix B (PDF , 2,714kb)

Appendix C (PDF , 1,811kb)

Informing a decision framework for when NICE should recommend the use of health technologies only in the context of an appropriately designed programme of evidence development 

Study summary: 

The general issue of balancing the value of evidence about the performance of a technology and the value of access to a technology can be seen as central to a number of policy questions. Establishing the key principles of what assessments are needed, as well as how they should be made, will enable them to be addressed in an explicit and transparent manner.  The aims of this research are to: i) Establish the key principles of what assessments are needed to inform an only in research (OIR) or Approval with Evidence Development (AED) recommendation. ii) Evaluate previous NICE guidance where OIR or AED recommendations were made or considered. iii) Establish what impact OIR and AED decisions may have had on subsequent research. iv) Evaluate a range of alternative options to establish the criteria, additional information and/or analysis which could be made available to help the assessment needed to inform an OIR or AED recommendation. v) Provide a series of final recommendations, with the involvement of key stakeholders, establishing both the key principles and associated criteria that might guide OIR and AED recommendations, identifying what, if any, additional information or analysis might be included in the Technology Appraisal process and how such recommendations might be more likely to be implemented through publically funded and sponsored research. To meet these aims the research is broadly structured as follows: i) development of a coherent conceptual framework to establish the key principles of what assessments need to be made and identify a set of explicit criteria (checklist of assessments) which could be applied; ii) review previous NICE guidance and develop a series of detailed case studies to evaluate a range of options for informing these assessments; iii) evaluate both the development of key principles and the range of options for informing assessments through a series of workshops involving a wide range of key stakeholders.

Informing choices about NICE's perspective for costs and effects 

The appropriate use of discounting in the appraisal of health technologies

  • Mike Paulden, Karl Claxton
  • Publication: Karl Claxton, Mike Paulden, Hugh Gravell, Werner Brouwer and Anthony J Culyer. Discounting and decision making in the economic evaluation of health-care technologies. Health Economics. 2010; DOI:10.1002/hec.1612 Download from Wiley online library 

Prioritising methodological research for the National Institute for Health and Clinical Excellence 

Coverage with evidence development, only in research, risk sharing or patient access scheme? A framework for coverage decisions 

Developing a mathematical programming approach for the optimal allocation of resources within and between different healthcare programmes 

  • Funder: Medical Research Council Methodology Research Programme
  • Claire McKenna, David Epstein, Karl Claxton, Zaid Chalabi (London School of Hygiene and Tropical Medicine)
  • Publications:
    • McKenna C, Chalabi Z, Epstein D, Claxton K. Budgetary policies and available actions: a generalisation of decision rules for allocation and research decisions. Journal of Health Economics 2010; 29: 170–181. Download from Science Direct
    • Chalabi Z, Epstein D, McKenna C, Claxton K. Uncertainty and value of information when allocating resources within and between healthcare programmes. European Journal of Operational Research. 2008;191(2):530-9. Download from Science Direct 

The role of third-party technology assessment to inform health systems’ decision-making about the funding of health technologies 

  • Funder: Medical Research Council Methodology Research Programme
  • Mark Sculpher, Marco Barbieri, Neil Hawkins
  • Publication: Barbieri M, Hawkins N, Sculpher MJ (2009).  Who does the numbers?  The role of third-party technology assessments to inform health systems’ decision-making about the funding of health technologies.  Value in Health 2009:12;193-201. Download from Wiley online library 

Self management interventions for chronic conditions: a methodological and applied programme of cost-effectiveness research in primary care 

  • Funder: Department of Health, Research Capacity Development Centre (2004-07)
  • Gerry Richardson's fellowship
  • Publications:
    • Richardson G, Bojke C, Kennedy A, Reeves D, Bower P, Lee V, et al. What outcomes are important to patients with long term conditions? A discrete choice experiment. Value in Health. 2009;12(2):331-9. 
    • Rogers A, Kennedy A, Reeves D, Bower P, Lee V, Gardner C, Gately and Richardson G. The UK Expert Patients Programme: results and implications from a national evaluation.  Medical Journal of Australia.  Accepted for publication Sept 2008.
    • Richardson G; Kennedy A; Reeves D; Bower P; Lee V; Middleton E; Gardner C; Gately C; Rogers A; Cost effectiveness of the Expert Patients Programme (EPP) for patients with chronic conditions Journal of  Epidemiology and Community Health 2008 ; 62 4 361 -367. 
    • Kennedy A, Reeves D, Bower P, Lee V, Middleton E, Richardson G, Gardner C,Gately C, Rogers A.  The effectiveness and cost effectiveness of a national lay led self care support programme for patients with long-term conditions: a pragmatic randomised controlled trial.  Journal of Epidemiology and Community Health 2007;61:254-61.
    • Richardson G, Gravelle H, Weatherly H and Ritchie G.  Cost-effectiveness of interventions to support self-care- a systematic review. International Journal of Technology Assessment in Health Care 2005;21(4) 423-432.

Statistical issues

Statistical issues in economic evaluation

Continuous time simulation and discretized models for cost-effectiveness analysis 

  • Marta Soares
  • Publication:
    • Soares MO, Canto e Castro L. Continuous time simulation and discretized models for cost-effectiveness analysis. Pharmacoeconomics 2012;30(12):1101-17. Download from adisonline

Analysis of health-related quality of life

This work developed a series of regression estimators based on features of the Beta distribution, to address idiosyncratic features of health-related quality of life (HRQoL) data, which include truncated support at both ends, positive or negative skewness, spikes at one or zero, and heteroskedasticity.   Firstly, we present both a single equation and a two-part model, and relevant estimation algorithms based on maximum-likelihood, quasi-likelihood and Bayesian Markov-chain Monte Carlo methods.  A novel Bayesian quasi-likelihood estimator is proposed.   Secondly, we present a simulation exercise to assess the performance of the proposed estimators against ordinary least squared (OLS) regression for a variety of HRQoL distributions that are encountered in practice.  Finally, using a real life example we assess the performance of the proposed estimators to quantify the treatment effect on QALYs in the EVALUATE hysterectomy trial.  Overall model fit is studied using several goodness of fit tests. Our simulation results reveal that the proposed methods are more robust in estimating covariate effects than OLS, especially when effects are large or there is a large spike at one. Quasi-likelihood techniques are more robust than maximum likelihood estimators.  In the EVALUATE trial, all but the Beta maximum likelihood estimator produce unbiased estimates of the treatment effect.

One and two-part Beta regression models provide flexible approaches to regress the mean of an outcome with truncated support such as HRQoL on covariates. We hope that this work will provide applied researchers with a practical set of tools to appropriately model outcomes in cost-effectiveness analysis.

  • Publications:
    • Basu A, Manca A. Regression estimators for generic health-related quality of life and quality-adjusted life years. Medical Decision Making 2011. In Press

Developing economic evaluation methods for decision making: the value of access to individual patient data 

  • NIHR Career Development Award (2010-2014)
  • Andrea Manca's fellowship
  • Study summary: 

Cost-effectiveness analysis assesses the value for money of a new technology versus current practice is assessed in terms of both health benefits and costs to the health care system.  A key requirement for any CEA is to consider all the available evidence relating to the clinical effectiveness, HRQoL and cost of the health technologies under scrutiny. Such an evidence base usually comprises a mix of aggregate (AD) and individual patient-level data (IPD). The overall aim of this project is to investigate the potential benefits of using IPD alongside AD when developing economic decision models.  This project will be a multifaceted exploration of the issues surrounding the use of such data.  It will consist of a mixture of (i) literature reviews; (ii) modelling methods development; (iii) case studies; and (iv) empirical evaluation of the performance of alternative approaches.   

Methodological issues relating to the analysis of aggregate and individual-level data for cost-effectiveness decision modelling 

This PhD aims at structuring and exploring how evidence has been used and analysed to inform decision models in economic evaluation in healthcare. The proposed framework will review the current practice of synthesizing evidence, highlighting related practical and methodological issues with a view to characterize, assess and draw guidance on the distinct ways evidence sources may be included in cost-effectiveness analysis. Furthermore, this PhD work will focus on methodological approaches related to the synthesis of data at aggregate and individual-level formats which will better inform decisions regarding resource allocation within a healthcare system. 

Multilevel modelling

  • Andrea Manca
  • Publications: 
    • Gauthier A, Manca A, Anton S. Bayesian modelling of healthcare resource use in multinational randomised clinical trials. Pharmacoeconomics. 2009;27(12):1017-29.
    • Manca A, Lambert PC, Sculpher M, Rice N. Cost-effectiveness analysis using data from multinational trials: the use of bivariate hierarchical modelling. Medical Decision Making 2007;27(4):471-90.
    • Manca A, Rice N, Sculpher MJ, Briggs AH. Assessing generalisability by location in trial based cost effectiveness analysis: the use of multilevel models. Health Economics 2005;14:471-485. 

Missing data 

  • Andrea Manca
  • Publication:
     
    • Manca A, Palmer S. Handling missing data in RCT-based cost-effectiveness analysis. Applied Health Economics and Health Policy 2005;4(2):65-75. 

Adjusting for baseline covariates 

  • Andrea Manca
  • Publications:
     
    • Austin PC, Manca A, Zwarenstein M, Juurlink DN, Stanbrook MB. A substantial and confusing variation exists in handling of baseline covariates in randomized controlled trials: a review of trials published in leading medical journals. Journal of Clinical Epidemiology 2010;63(2):142-53.
    • Manca A, Hawkins N, Sculpher MJ. Estimating mean QALYs in trial-based cost-effectiveness analysis: the importance of controlling for baseline utility. Health Economics 2005;14:487-496. 

Bayesian methods 

  • Andrea Manca
  • Publications:
     
    • Conigliani C, Tancredi A, Manca A. Bayesian methods for healthcare decisions (in Italian). Pharmacoeconomics (Italian Research Articles),2010;12(1):1-4.
    • Conigliani C, Manca A., Tancredi A. Statistical methods in health economic evaluation (Chapter) in  Faltin F, Kenett R, Ruggeri F. (eds) Statistical Methods in Healthcare: Planning, Delivering, and Monitoring Care,  Wiley (in progress).
    • Kharroubi SA, Manca A, Pennington M, et al. Bayesian statistical models to analyse willingness to pay data from contingent valuation studies (in progress). 

Bivariate random effects meta-analysis 

Heterogeneity

Heterogeneity, generalisability and transferability in economic evaluation 

Heterogeneity and sub-group analysis in NICE's technology appraisal
programme (2006-7) 

  • Funder: National Institute of Health and Clinical Excellence through the
    Decision Support Unit (2003- )
  • NICE's 2008 Methods Guide 
  • Publication: Sculpher MJ. Subgroups and heterogeneity in cost-effectiveness analysis. Pharmacoeconomics. 2008;26:799-806.  

Heterogeneity and subgroup analysis in cost-effectiveness 

The presence of patient heterogeneity in factors such as baseline risk, treatment effect, health-related quality of life and costs means that a  give treatment strategy may be cost-effective for some individuals but not for others.  In these circumstances, decisions based on the average cost-effectiveness estimator may be suboptimal for the society as a whole, since these decisions carry the risk to forego potential benefits for a given subset of the population.  This PhD research is aimed to further develop methods to account for heterogeneity and subgroup effects in cost-effectiveness analysis for decision making. 

The generalisability by location of the results of economic evaluation studies conducted alongside multinational and multicentre randomised controlled clinical trials collecting patient-level data (2004-7) 

An investigation of methods to increase the generalisability and transferability of economic evaluation studies  

  • Funder: NHS HTA programme 
  • Andrea Manca, Mark Sculpher, Mike Drummond 
  • Final report  
  • Publications:
     
    • Manca A, Rice N, Sculpher MJ, Briggs AH. Assessing generalisability by location in trial based cost effectiveness analysis: the use of multilevel models. Health Economics 2005;14:471-485.
    • Urdahl H, Manca A, Sculpher MJ.  Assessing generalisability in model-based economic evaluation studies: a structured review in osteoporosis, Pharmacoeconomics 2006; 24(12):1181-97.
    • Drummond MF, Manca A, Sculpher MJ. Increasing the generalisability of economic evaluations: recommendations for the design, analysis and reporting of studies. International Journal of Technology Assessment in Health Care 2005;21:165-171. 

Public Health

Economic evaluation in public health

Identifying appropriate methods to incorporate concerns about health inequalities into economic evaluations of health care programmes.

Workshop 12th March 2012

This study aims to explore options for extending CEA to appropriately and quantitatively incorporate concerns for unfair health inequality.  Drawing on the literature for measuring inequalities in health and the determinants of health it will identify the range of existing approaches which could be applied within CEA by examining their quantitative properties, practicality and underlying normative principles.  This requires reflection on existing relevant research, a distillation of essential features and the bringing together of expertise in each area.  The results will provide guidance on the alternative methods at each stage of the CEA of health programmes for a decision maker concerned with both efficiency and inequality.  Throughout, stylised examples will demonstrate the approach from the perspective of a UK decision maker.  The objective is to outline a conceptual and analytical framework which can be used in the economic evaluation of interventions in health care and public health which have potentially important effects on unfair health inequalities.

Academic Fellowship in Health Economics and Public Health

Methods for assessing the cost-effectiveness of public health interventions

  • Funder: Department of Health Policy Research Programme through the Public Health Research Consortium (2006-7)
  • Helen Weatherly, Mike Drummond, Mark Sculpher, Karl Claxton, Richard Cookson, Chris Godfrey, Brian Ferguson, Amanda Sowden
  • Publications:
    • Cookson R, Drummond M, Weatherly H. Explicit incorporation of equity considerations into economic evaluation of public health interventions. Health Economics Policy and Law. 2009;4(2):231-45.
    • Cookson R, Drummond M, Weatherly H. Explicit incorporation of equity considerations into economic evaluation of public health interventions. Reply to Richardson and Shiell. Health Economics Policy and Law. 2009;4(2):261-263.
    • Weatherly H, Drummond M, Claxton K, Cookson R, Ferguson B, Godfrey C, Sculpher M, et al. Methods for assessing the cost-effectiveness of public health interventions: key challenges and recommendations. Health Policy 2009;93:85-92.
    • Drummond M, Weatherly H, Ferguson B. Economic evaluation of health interventions: a broader perspective is needed that include potential cost and benefits for all stakeholders. British Medical Journal. 2008;337:a1204.

Workshop in methods of economic evaluation in public health