Checking consistency in mixed treatment comparison meta-analysis

Abstract: Indirect and mixed treatment comparisons (MTC), also known as network meta-analysis, represent an important development in evidence synthesis, particularly in decision making contexts. Rather than pooling information on trials comparing treatments A and B, A and C, B and C etc separately, MTC combines data from randomised comparisons, A vs B, A vs C, A vs D, B vs D, and so on, to deliver an internally consistent set of estimates while respecting the randomisation in the evidence. MTC allows coherent judgements on which of several treatments is the most effective and produces estimates of the relative effects of each treatment compared to every other treatment in a network  even though some pairs of treatments may not have been directly compared. However, doubts have been expressed about the validity of MTC, particularly the assumption of consistency between "direct" and "indirect" evidence. Inconsistency can be thought of as a conflict between "direct" evidence on a comparison between treatments B and C, and "indirect" evidence gained from A vs C and A vs B trials. Like between-trial heterogeneity, inconsistency is caused by effect-modifiers, and specifically by an imbalance in the distribution of effect modifiers in the direct and indirect evidence.

Several methods have been put forward in the literature  node-split and inconsistency models, or calculating the difference between 'direct' and 'indirect' evidence  to attempt to formally or informally assess, test for and investigate the consistency assumption in different ways.

I will begin by defining inconsistency as a property of "loops" of evidence, and the Inconsistency Degrees of Freedom, as (approximately) the number of independent loops of evidence in a network. I will then explore different methods to assess whether there is inconsistency in the network and where it might be located, discussing the merits and drawbacks of each method using illustrative examples.