I read the Letter “Is it appropriate to compare the results from two clinical trials with one drug in common?” by Marrugat et al.,1 and published in Revista Española de Cardiología with great interest. The Letter discusses indirect comparisons of the effect of prasugrel vs ticagrelor in patients with acute coronary syndrome treated with percutaneous coronary interventions, citing as examples TRITON-TIMI 38 (TRial to Assess Improvement in Therapeutic Outcomes by Optimizing Platelet InhibitioN with Prasugrel-Thrombolysis In Myocardial Infarction; ClinicalTrials.gov: NCT00097591), which compared the efficacy of prasugrel vs clopidogrel, and the PLATO trial (A Comparison of Ticagrelor [AZD6140] and Clopidogrel in Patients With Acute Coronary Syndrome; ClinicalTrials.gov: NCT00391872), which evaluated the efficacy of ticagrelor vs clopidogrel. The authors claim that, in this type of analysis, indirect comparisons are inappropriate due to methodological differences between the studies or differences in patient characteristics and categorically conclude that such comparisons are “inappropriate, should be discouraged, and the scientific community should actively avoid this type of analysis”.1 Taking advantage of the invitation to debate implicit in the question posed by the authors in the title of their article, I would like to clarify certain issues concerning the use of indirect comparisons.
Firstly, indirect comparisons based on individual studies can lead to substantial selection bias that seriously places the validity of the results obtained in doubt. In fact, any indirect comparison should be established within the framework of rigorous systematic reviews and meta-analyses, bearing in mind the complete network of trials that guarantee their quality, a criterion that does not seem to have been met in the example cited by Marrugat el al.,1 but which has been fulfilled in a recent study performed by Steiner et al.,2 in which the authors identify a network of 14 clinical trials that serve to establish various treatment comparisons (high- vs standard-dose clopidogrel, 9 trials; prasugrel vs standard-dose clopidogrel, 2 trials; prasugrel vs. high-dose clopidogrel, 2 trials; ticagrelor vs standard-dose clopidogrel, 1 trial) (Figure).
Example of a network of evidence from clinical trials evaluating the efficacy of new antiplatelet treatments in acute coronary syndrome. Continuous lines represent direct comparisons and dotted lines represent indirect comparisons. Adapted from Steiner et al.2
In my opinion, it should be clarified that indirect comparisons and their extension in network meta-analyses can be useful when there are diverse treatment alternatives that have been compared with a common comparator (e.g., clopidogrel in the example given by Steiner et al.2) but when the information from direct comparisons is scarce or nonexistent. To interpret these indirect comparisons correctly, knowledge of the main assumptions adopted is also essential. Firstly, as with other evidence synthesis techniques (conventional meta-analyses), the validity of indirect comparisons depends on the quality of the studies, the variability among the studies (accepting a certain degree of heterogeneity) and the information biases. Secondly, indirect evidence is generally considered exploratory and observational and requires that a transitive property be postulated (that is, if drug A is superior to B, and B is superior to C, it is assumed that A is superior to C). In fact, indirect comparisons can indeed be established even when the studies differ in their characteristics and patients, etc., but are not advisable when there may be factors that could influence treatment effects. Some authors speak of transitivity when distinct trials are comparable in the sense that they do not differ in the distribution of the factors modifying the treatment effect (e.g. concomitant treatments, study design, patient severity, therapeutic indications, etc.). However, some of these factors can indeed be identified a priori and can be taken into account in these analyses by using more sophisticated techniques. In particular, meta-regression techniques allow indirect comparisons to be made by adjusting the measurements of the effect of the treatments (dependent variable) by several factors that could act as modifiers of that effect (independent variables), thus reducing sources of bias.3–5
Lastly, the use of indirect comparisons as a source of complementary, non-excluding and hypothesis-generating information could aid the application of research results in clinical practice. However, further knowledge and greater methodological development of techniques, with rigorous evaluation before their widespread use, would seem to be advisable.
Note
The opinions expressed in this letter are those of the author, and are not intended to reflect the views of the institutions in which he works.
.