DIAGNOSTIC ACCURACY STUDIES

The development and validation of methods for assessing the quality and reporting of diagnostic studies

Background

This project aimed to review the literature on the concepts underlying diagnostic research and identify the main factors that can bias the results of diagnostic studies. An examination of how quality assessment has been used in systematic reviews and the existing methods or assessment tools that have been used to assess the quality of diagnostic research, together with any evidence on which they are based was undertaken. A new evidence-based assessment tool for the quality assessment of diagnostic studies was then developed and its reliability and validity was tested by examining the impact of 'low' versus 'high' quality studies on the results of meta-analyses of diagnostic studies.

Findings

The project produced an evidence-based quality assessment tool to be used in systematic reviews of diagnostic accuracy studies. Through the various stages of the project the current lack of such a tool and the need for a systematically developed validated tool were demonstrated. Further work to validate the tool continues beyond the scope of this project. The further development of the tool by the addition of design and topic specific criteria is proposed.

Conducted by: P Whiting1, AWS Rutjes2, J Dinnes3, JB Reitsma2, PMM Bossuyt2, J Kleijnen1

1. Centre for Reviews and Dissemination; 2. Department of Clinical Epidemiology and Biostatistics, Academic Medical Centre, University of Amsterdam; 3. Wessex Institute for Health Research and Development, University of Southampton

Further details

Project page on HTA Programme website

Publications

Westwood ME, Whiting PF, Kleijnen J. How does study quality affect the results of a diagnostic meta-analysis? BMC Med Res Methodol. 2005;5(1):20

Whiting P, Harbord R, Kleijnen J. No role for quality scores in systematic reviews of diagnostic accuracy studies. BMC Med Res Methodol. 2005; 5:19

Whiting P, Rutjes AWS, Dinnes J, Reitsma JB, Bossuyt PMM, Kleijnen J. Development and validation of methods for assessing the quality of diagnostic accuracy studies. Health Technol Assess. 2004;8(25):1-264

Wilby J, Kainth A, Hawkins N, Epstein D, McIntosh H, McDaid C, Mason A, Golder S, O'Meara S, Sculpher M, Drummond M, Forbes C. A rapid and systematic review of the clinical effectiveness, tolerability and cost effectiveness of newer drugs for epilepsy in adults. Health Technol Assess. 2005;9(15):1-172

Whiting P, Rutjes AWS, Reitsma JB, Bossuyt PMM, Kleijnen J. The development of QUADAS: a tool for the quality assessment of studies of diagnostic accuracy included in systematic reviews. BMC Med Res Methodol. 2003;3(1):25

Whiting P, Rutjes AWS, Reitsma JB, Glas AS , Bossuyt PMM, Kleijnen J. Sources of variation and bias in studies of diagnostic accuracy: a systematic review. Ann Intern Med. 2004;140(3):189-202

Whiting PF, Westwood ME, Rutjes AWS, Reitsma JB, Bossuyt PMM, Kleijnen J. Evaluation of QUADAS, a tool for the quality assessment of diagnostic accuracy studies. BMC Med Res Methodol. 2006;6:9

Posters

Westwood M, Whiting P. Should systematic reviews of diagnostic tests go beyond test accuracy? Cochrane Colloquium 2008. Freiburg, Germany, 3 - 7 October 2008

Glanville J, Whiting P, Sterne J, Westwood M, Burke M. Which databases should we search to identify diagnostic test accuracy studies? Health Technology Assessment International (HTAi) Annual Meeting; 2007 June; Barcelona, Spain

Ritchie G, Glanville J, Lefebvre C. Do published search filters to identify diagnostic test accuracy studies perform adequately? Health Technology Assessment International (HTAi) Annual Meeting; 2007 June; Barcelona, Spain

Whiting P, Harbord R, Kleijnen J. Scoring the quality of diagnostic accuracy studies: an example using QUADAS. 12th Cochrane Colloquium; 2004 October; Ottawa, Canada

Whiting P, Rutjes A, Reitsma H, Bossuyt P, Kleijnen J. The development of QUADAS: a tool for the quality assessment of studies of diagnostic accuracy included in systematic reviews. Health Technology Assessment International (HTAi) Annual Meeting; 2004 June; Krakow, Poland

Presentations

Glanville J. Finding non-RCT evidence - quality of life studies, diagnostic tests. 2nd InterTASC Workshop; 2002 May; London, UK

Glanville J. Outline of recent research into search filters to identify diagnostic tests. Health Technology Assessment International (HTAi) Annual Meeting; 2007 June; Barcelona, Spain

Glanville J. Searching for studies for reviews of diagnostic test accuracy - which databases, which filters? 12th Annual Meeting of the UK and Ireland Contributors to the Cochrane Collaboration; 2007 March; Oxford, UK

Rutjes A, Whiting P, Reitsma J, Vlassov V. Quality assessment of studies of diagnostic accuracy included in systematic reviews. 12th Cochrane Colloquium; 2004 October; Ottawa, Canada

Westwood M, Whiting P, Cooper J, Kleijnen J. Using the QUADAS tool in the conduct of systematic reviews of diagnostic tests. 11th Cochrane Colloquium; 2003 October; Barcelona, Spain

Whiting P, Rutjes A, Reitsma J, Bossuyt P, Kleijnen J. QUADAS: the development of a new tool for the quality assessment of diagnostic accuracy studies. 11th Cochrane Colloquium; 2003 October; Barcelona, Spain

Whiting P, Westwood M, Burke M, Sterne J, Glanville J. Is it necessary to search a wide range of databases to identify diagnostic test accuracy studies? 14th Cochrane Colloquium; 2006 October; Dublin, Ireland

Funding

Commissioned by the HTA Programme