On the misinterpretation of clinical trials

Corresponding Author:

Jeffrey W Moses
Department of Medicine, Division of Cardiology
New York Presbyterian Hospital/ Columbia University Medical Center
161 Fort Washington Ave., 5th Floor, NY 10032, USA
Tel: +1 212 305 7060
Fax: +1 212 342 3660
E-mail: jmoses@crf.org

Abstract

“...many trial investigators tend to overextend the generalizability of their trials. Such tendencies are a disservice to the medical community. The trials cannot be extended beyond the populations recruited.”

“Science ... warns me to be careful how I adopt a view which jumps with my preconceptions, and to require stronger evidence for such belief than for the one to which I was previously hostile …”

– Thomas H Huxley (1825–1895)

The design and execution of clinical trials constitute a complex interplay of framing relevant questions, statistical insight, fiscal limitations and the practical problems of patient recruitment and follow-up. The last issue is never more pertinent or complex than in the realm of trials comparing two alternative therapies. Thus, in the face of these nuances, it is quite concerning to this interventional cardiologist that several recently published ‘therapy versus therapy’ trials involving percutaneous coronary intervention (PCI) have been subjected to ‘sound bite’ interpretation, compounded by political agendas that lead to the loss of their core messages to those who actually render the care to the patient.

A florid example of the ‘sound bite’ approach appeared in the conclusions of a recent paper analyzing the differences in revascularization rates and the use of PCI in the Medicare populations across the USA. The authors (the two leads not even possessing medical degrees) are working in a highly complex and poorly understood arena; tabulating regional differences in procedure utilization. Nonetheless they rapidly jump to a conclusion that ‘given recent studies of medical versus interventional management of patients with stable coronary artery disease (CAD), patients living in high diagnostic regions may be getting more treatment than they want or is needed’, and ‘in low- and moderate-risk patients do we even want to know the anatomy?’ [1]. It is this type of ill-informed leap by a nonclinical ‘policy maker’ and population scientist that presents the greatest danger to depriving patients with CAD of optimal care. As an advocate of evidence-based medicine and having been involved in numerous multicenter trials exploring optimal treatments I share grave concerns regarding this trend. As our legal colleagues know well, ‘evidence’ can be subjected to many interpretations depending on the weight given to the various facts. There is no single conclusion derived from ‘evidence’.

A clinical trial can only be interpreted in the context of the population actually recruited and the therapies rendered. The extension of its findings to the inclusion/exclusion criteria despite the fact that a much narrower group of patients were actually included is a grave error. Extrapolating these findings beyond the populations potentially included in this trial, is in my mind, a scientific felony. Unfortunately, this has more often than not become the case. To emphasize this point I will review two recent clinical trials that highlight this trend toward overinterpreting a trial’s finding as opposed to focusing on the actual findings in the populations actually tested.

Example one: COURAGE

▪ Conclusion

‘As an initial management strategy in patients with stable CAD, PCI did not reduce the risk of death or myocardial infarction (MI) or the major cardiovascular events when added to medical therapy’ [2].

This trial has been touted to indicate that patients with so-called stable coronary disease have no improvement in outcome when treated with PCI on a background of modern medical therapy. This population of slightly less than 2300 patients was an ultra-low-risk group of patients. Two-fifths had virtually no angina and less than a third had significant ischemia on perfusion testing. These low-risk features are supported by the fact that the cardiac mortality was unprecedentedly low at 0.4% per year. When surveying centers that recruited to this trial, the vast majority systematically excluded high-risk patients on the basis of symptoms or anatomy, as patients were randomized only after coronary angiography.

▪ Findings

The trial only achieved approximately twothirds of its anticipated number of events due to this very-low-risk group. Yet in spite of this low ischemic burden and mildly symptomatic population, approximately a third crossed over to PCI, most by the end of the second year. There was no difference in death or MI although the numeric trends were in favor of PCI in reduction of spontaneous events. Importantly, medical treatment did nothing to reduce ischemia! This is a particularly important finding given the fact that the COURAGE nuclear substudy revealed that residual ischemia was the prime driver of death and MI [3]. A second remarkable finding was that in spite of the low overall symptomatic status of the patients, and the high crossover rate, quality of life was superior in the PCI group up to 3 years [4]. The equilibration in symptom status at that point was no doubt driven by the high crossover rate. Moreover, this symptomatic benefit extended over all three tertiles of symptoms, even though the bottom tertile had virtually no reported angina.

“A clinical trial can only be interpreted in the context of the population actually recruited and the therapies rendered ... Extrapolating these findings beyond the populations potentially included in this trial is, in my mind, a scientific felony.”

The patient-oriented conclusion: if you have significant ischemia, PCI provides far superior outcomes in the reduction of ischemia and potential for death and MI. If you have even modest symptoms, PCI provides immediate and superior symptomatic relief and quality of life compared with medical therapy alone with no incremental risk from the procedure itself. If neither is present then an initial trial of medical therapy is warranted if critical anatomy has been excluded.

Example two: OAT

▪ Conclusion

‘PCI did not reduce the occurrence of death due to infarction or heart failure and there is a trend towards excess MI over 4 years of follow-up in stable patients with occlusion of the infarct artery treated 28 days after MI’ [5].

The results of this trial have been used to indicate that mildly symptomatic postinfarction patients need only medical therapy and maybe not even ischemia assessment or anatomic definition.

This trial recruited slightly under 2200 patients approximately 1-week postinfarction with predominately untreated single- vessel right coronary occlusions and normal ejection fractions. Two-thirds of this population had akinesia and Q waves in the relevant territories. Of the 27% that had protocol-mandated stress tests because of lack of infarction pattern, almost 90% had no inducible ischemia.

▪ Findings

Given the predominance of single-vessel disease subserving infarcted regions, it is difficult to see how MI could be reduced in this population. With normal ejection fractions and single-vessel disease, congestive heart failure would be anticipated to be a very infrequent event and in fact it was at 4.5% over 4 years. The slight trend towards greater rates of infarction was predominately driven by periprocedural events.

What is particularly interesting in OAT is the quality of life study of over 950 patients [6]. In this population, only 23% had anginal symptoms and 87% were New York Heart Association (NYHA) Class I. Yet in spite of the minimal symptomatology in these patients, there was a 25% crossover to PCI at 2 years. Interestingly, this is quite similar to the rate seen in the COURAGE trial. Even with this crossover rate at 2 years, 48% of the medical patients had dyspnea or angina versus 37% treated with PCI, a statistically significant difference. Probably as a result of these findings: out-of-hospital medical costs in the medical group over 2 years far exceeded those of the PCI group at US$8000 versus $4900. One could speculate that on longer-term follow-up, this gap in medical costs would grow given the fact that it was well-demonstrated in the COURAGE trial there was significantly less use of antianginal medication in the PCI group as well.

Patient-oriented conclusion: even in a mostly ‘asymptomatic’ low-risk group with predominately infarcted myocardium, there is significant symptomatic benefit and reduction of out-of-hospital costs with no excess risks of mortality engendered by the PCI. This trial has no relevance to postinfarction patients with clear-cut symptoms, multivessel disease, depressed left ventricular function, ischemia or chronic total occlusions.

Conclusion

Thus, in this era of intense debate regarding the nature of healthcare reform in the USA, the interpretation of the ‘evidence’ and even the meaning of the word is critical.

“In spite of economic pressures and a political sweep that is demanding physicians look at the societal costs of treatment, it is our job as practitioners to advocate our patients’ best interest and base our medical care system on that premise.”

I cite here two examples of this in the realm of PCI that focused on ‘hard end points’. Yet, in both trials it is evident that predominately lowrisk populations were recruited as is the nature of such clinical trials, with rare exceptions such as the SYNTAX trial [7]. The key and consistent finding in these two trials is that symptomatic benefit from PCI accrued to the patients immediately and over several years compared with the medical therapy. In addition, it is clear that the small groups at higher risk that were recruited into these trials (specifically in COURAGE) had superior ‘hard’ outcomes with PCI.

From the clinician’s standpoint, discussions with the patient should make it clear that symptomatic relief is better with PCI and in those with evidence of even moderate ischemia, PCI is preferred. Two recent meta-analyses support these conclusions [8,9].

Nonetheless, many trial investigators tend to overextend the generalizability of their trials. Such tendencies are a disservice to the medical community. The trials cannot be extended beyond the populations recruited. For example, certain OAT investigators have claimed their trial’s findings apply to up to 100,000 PCIs a year. At Columbia, at most only 1.4% of PCIs fit the anatomic demographics of the OAT population, indicating that its findings apply to at most 11,000 PCIs a year in the USA. Such exaggerations misinform the lay public (media included) and general medical community. Even worse, they imply an unsupported gross overuse of the procedure in post-MI populations.

In spite of economic pressures and a political sweep that is demanding physicians look at the societal costs of treatment, it is our job as practitioners to advocate our patients’ best interest and base our medical care system on that premise. Clinical trials need to be viewed and interpreted in that context.

Financial & competing interests disclosure

Jeffrey Moses has minor speaking fees from Cordis and Abbott. The author has no other relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript apart from those disclosed.

No writing assistance was utilized in the production of this manuscript.

References

1. Lucas FL, Siewers AE, Malenka DJ, Wennberg DE: Diagnostic-therapeutic cascade revisited: coronary angiography, coronary artery bypass graft surgery, and percutaneous coronary intervention in the modern era. Circulation 118(25), 2797–2802 (2008).
2. Boden WE, O’Rourke RA, Teo KK et al.: Optimal medical therapy with or without PCI for stable coronary disease. N. Engl. J. Med. 356(15), 1503–1516 (2007).
3. Shaw LJ, Berman DS, Maron DJ et al.: Optimal medical therapy with or without percutaneous coronary intervention to reduce ischemic burden: results from the Clinical Outcomes Utilizing Revascularization and Aggressive Drug Evaluation (COURAGE) trial nuclear substudy. Circulation 117(10), 1283–1291 (2008).
4. Weintraub WS, Spertus JA, Kolm P et al.: Effect of PCI on quality of life in patients with stable coronary disease. N. Engl. J. Med. 359(7), 677–687 (2008).
5. Hochman JS, Lamas GA, Buller CE et al.: Coronary intervention for persistent occlusion after myocardial infarction. N. Engl. J. Med. 355(23), 2395–2407 (2006).
6. Mark DB, Pan W, Clapp-Channing NE et al.: Quality of life after late invasive therapy for occluded arteries. N. Engl. J. Med. 360(8), 774–783 (2009).
7. Serruys PW, Morice MC, Kappetein AP et al.: Percutaneous coronary intervention versus coronary-artery bypass grafting for severe coronary artery disease. N. Engl. J. Med. 360(10), 961–972 (2009).
8. Schomig A, Mehilli J, de Waha A et al.: A meta-analysis of 17 randomized trials of a percutaneous coronary intervention-based strategy in patients with stable coronary artery disease. J. Am. Coll. Cardiol. 52(11), 894–904 (2008).
9. Jeremias A, Kaul S, Rosengart TK, Gruberg L, Brown DL: The impact of revascularization on mortality in patients with nonacute coronary artery disease. Am. J. Med. 122(2), 152–161 (2009).