Systematic reviews of adverse effects often include evidence from a variety of sources including RCTs, observational studies, case reports and case series. There are difficulties in synthesising disparate data sets and differences between studies have to be considered as a source of heterogeneity (whether narrative or quantitative). In observational studies the extent of drug exposure is not as certain as in RCTs. For example, in cohort studies many patients might have received an incomplete course of medication which may lead to the underestimation of the true rate or severity of adverse effects. Patients in the control group may have procured the medication during a generally prolonged follow-up period and this may lead to overestimation of the rate of adverse effects in the control group.73
Whether narrative or quantitative synthesis is used, researchers should try to explore any patterns identified across the results and discuss the possible factors that might explain variations in study findings (e.g. rate and severity of adverse effects). Attempts should be made to explore possible relationships between characteristics of included studies and their reported findings and also between the findings of different studies. Researchers should clearly indicate the populations addressed by the included studies and carefully assess the applicability to other populations.11
Exploring heterogeneity in study findings is especially important.74 Variations may be due to methodological differences and/or differences in the characteristics of the included studies. The possible effects of individual study quality indicators (e.g. follow-up period, methods used to identify adverse effects), study design, study size and funding sources in the analysis should be investigated and discussed.11 Subgroup, sensitivity or regression analyses may be helpful for explaining some of these variations and generating functional hypotheses.
Researchers should provide a detailed description of cases of unusual or not previously recorded adverse effects.48
There is little guidance about when and how to perform meta-analysis of adverse effects data. It is important, but not always easy to determine when and what data from multiple studies should be combined.75 No standard technique is available for meta-analysis of diverse and heterogeneous data, and selection of techniques depends on different factors including the aim of the review, characteristics of selected studies and type of outcomes.73 Although data from both observational studies and RCTs has been combined, for example to present a single estimate of mortality associated with chronic usage of non-steroidal anti-inflammatory drugs (NSAIDs),73 in some reviews it may only be appropriate to quantitatively combine results from one or some study designs (e.g. RCTs and cohort studies) and synthesise data from other types of studies (e.g. case series and case reports) using a narrative approach. As with efficacy data it may be appropriate to conduct subgroup analyses or, where data allow, use meta-regression to further explore the risk of adverse effects. For example the risk of bowel perforation with the cancer drug bevacizumab is thought to be higher in patients with ovarian cancer than in other cancers.76
Various Bayesian approaches to meta-analysis have been used77, 78, 79, 80 and when and how to use Bayesian approaches in reviews of adverse effects is a developing field. For example, a Bayesian approach has been used to combine evidence from case-control and prospective studies to estimate the absolute risk of developing ovarian cancer.81