CAS, CREST and AHA guidelines for treating carotid stenosis: randomized controlled trials can be misleading

Corresponding Author:

Frank J Veith
Division of Vascular Surgery, New York University Medical Center
New York, NY, USA and Division of Vascular Surgery
The Cleveland Clinic, Cleveland, OH, USA
E-mail: fjvmd@msn.com

Abstract

Keywords

bias, carotid stenting, CREST, equivalence, flaws in randomized trials, SAPPHIRE

The CREST trial comparing carotid artery stenting (CAS) and carotid endarterectomy (CEA) for the treatment of high-grade carotid stenosis has been interpreted by many to demonstrate the equivalence of the two procedures [1]. This equivalence of CAS and CEA is supported by the recent American Heart Association (AHA) guideline for treating carotid stenosis, which has been endorsed by 13 other prestigious US organizations [2]. This article will examine this ‘equivalence’ and the nature and validity of the level one evidence, which is purported to support it.

Level one evidence and the randomized controlled trials (RCTs) that comprise it are widely considered to be the best basis for determining medical practice. This is particularly true when the RCTs are published in leading journals such as the New England Journal of Medicine or Lancet. Such trials are viewed by many as the ‘holy grail’ of medicine.

However, RCTs can have many flaws that render them obsolete, nonapplicable or overtly misleading. More importantly, RCTs can be spun or misinterpreted by their authors or others so that they exert an effect on practice trends or standards quite unjustified by their data.

Possible flaws in RCTs are of two types: first, are the timeliness flaws that can occur when progress is made in the treatment-underevaluation arm or the control arm of RCTs. Examples would be the early trials of CAS versus CEA. If progress in CAS technology or patient selection occurs, a trial such as EVA-3S, showing CAS inferiority becomes invalid [3]. By contrast, the landmark trials showing CEA to be superior to medical treatment in preventing strokes have become obsolete because dramatic recent progress has been made with medical treatment since patients were entered into these trials [4–6].

Second are the many design flaws that can also impair the validity of RCTs. These include patient selection flaws (e.g., in the SAPPHIRE trial, patients were selected for randomization only if they were high risk for CEA) [7]. SAPPHIRE also included 71% asymptomatic patients in whom the 30-day periprocedural stroke, death and myocardial infarction (MI) rates (~5 and ~10% for CAS and CEA, respectively) were so high that no invasive procedure was justified [7]. Good medical treatment would have served these patients better. CREST also had patient selection flaws. It was originally designed to compare CAS and CEA only in symptomatic patients. However, when adequate numbers of symptomatic patients could not be recruited, asymptomatic patients were added, thereby diluting the power of the study and impairing the statistical significance of some of its results (Table 1) [1].

Other design f laws include: questionable competence of operators in a trial (e.g., the CAS operators in the EVA-3S [3] and ICSS [8] trials); problems with randomization (e.g., SAPPHIRE in which only 10% of eligible patients were randomized [7]); and questionable applicability of RCT results to real-world practice (e.g., CAS operators in CREST were highly vetted and more skilled than many others performing the procedure) [1]. There are also idiosyncratic flaws, as in the EVAR 2 trial in patients unfit for open abdominal aortic aneurysm repair [9]. Although this trial, published in Lancet, showed endovascular aneurysm repair to have similar mortality to no treatment, half the deaths in the group randomized to endovascular aneurysm repair occurred from rupture during a lengthy (average 57 days) waiting period before treatment. Had these deaths been prevented by a more timely endovascular aneurysm repair, the results and conclusion of EVAR 2 might have been different.

Inappropriate or questionable primary end points in RCTs are another design flaw that can lead to misleading conclusions. An example is the inclusion of minor MIs with strokes and deaths as a composite end point in a CAS versus CEA trial (as in CREST [1]). The components of the primary end point within 30 days of the procedure in the CAS and CEA arms of CREST are shown in Table 1 [1]. Although some may argue otherwise, it is hard to understand how minor MIs could be considered an equivalent to strokes and deaths, yet only when MIs were included, were the adverse event rates within 4 years in the two groups similar (7.2% for CAS vs 6.8% for CEA; p = 0.051) [1].

Table 1. Some detailed data from CREST.

Aside from the flaws in RCTs, what about ‘good trials’ or those with no or only minor weaknesses? Even these can be misleading when the authors reach conclusions unjustified by their own data. The SAPPHIRE and CREST trials are two recent examples. Both were reported in the New England Journal of Medicine [1,7,10]. The conclusions reached were that “with high-risk patients CAS and CEA are equivalent treatments” (SAPPHIRE) [7,10], and “among patients with symptomatic and asymptomatic carotid stenosis, the risk of the composite primary end point … did not differ significantly in the group undergoing CAS and the group undergoing CEA” (CREST)[1].

Although the CREST authors did point out the higher incidence of stroke with stenting, others have used the CREST study to claim equivalence of CAS and CEA. Nowhere is this more apparent than in the recent AHA guideline on the management of patients with extracranial carotid and vertebral artery disease [2]. This influential document, which was also approved by 13 other important and relevant US organizations, stated that “CAS is indicated as an alternative to CEA for symptomatic patients at average or low risk of complications associated from endovascular interventions…” [2]. In Webster’s New World College Dictionary, one definition of ‘alternative’ is “a choice between two things” [101].

This clearly implies equivalence, and it has been so interpreted by many others, particularly those biased toward catheter-based treatment [11,102]. Of note, the AHA guideline appears to be based largely on CREST and did not even consider the findings of the ICSS trial [8], published in Lancet the same week as the main article reporting CREST. Although ICSS may also have flaws, it showed, in a large group of only symptomatic patients, that CAS produced significantly more strokes and diffusion-weighted MRI defects than CEA [8,12]. It is hard to understand why these ICSS results did not have more of an influence on the AHA Guideline.

“... even the results of good trials can be further misinterpreted by others to guide practice standards in a way unjustified by the data.”

Although my bias as a CAS enthusiast makes me believe that CAS will ultimately have a major role in the treatment of patients with carotid stenosis, and although results of CAS will ultimately improve as we develop better stents, brain protection devices and patient selection, that bias is not yet sufficient for me to use the current data to make believe we are now there. One has to wonder if bias more intense than mine was involved in the disputed ‘alternative’ conclusion reached in the AHA guideline.

Thus, it is apparent that misleading conclusions can be reached in articles reporting RCTs in leading journals. These can be the result of flaws in the RCTs and/or unrecognized author bias. More importantly, even the results of good trials can be further misinterpreted by others to guide practice standards in a way unjustified by the data. It is important for all to recognize the possible role of bias in these misinterpretations. By recognizing the possible flaws in RCTs and that physicians, like all other people, are influenced by bias, we can exercise the judgment to use RCTs fairly to help us treat individual patients optimally.

Financial & competing interests disclosure

The author has no relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript. This includes employment, consultancies, honoraria, stock ownership or options, expert testimony, grants or patents received or pending, or royalties.

No writing assistance was utilized in the production of this manuscript.

References

1. Brott TG, Hobson RW 2nd, Howard G et al.; CREST Investigators. Stenting versus endarterectomy for treatment of carotid artery stenosis. N. Engl. J. Med. 363, 11–23 (2010).
2. Brott TG, Halperin JL, Abbara S et al. 2011 ASA/ACCF/AHA/AANN/AANS/ ACR/ASNR/CNS/SAIP/SCAI/SIR/SNIS/ SVM/SVS guideline on the management of patients with extracranial carotid and vertebral artery disease: a report of the American College of Cardiology Foundation/ American Heart Association Task Force on practice guidelines, and the American Stroke Association, American Association of Neuroscience Nurses, American Association of Neurological Surgeons, American College of Radiology, American Society of Neuroradiology, Congress of Neurological Surgeons, Society of Atherosclerosis Imaging and Prevention, Society for Cardiovascular Angiography and Interventions, Society of Interventional Radiology, Society of Neurointerventional Surgery, Society for Vascular Medicine, and Society for Vascular Surgery developed in collaboration with the American Academy of Neurology and Society of Cardiovascular Computed Tomography. J. Am. Coll. Cardiol. 57(8), 1002–1044 (2011).
3. Mas JL, Chatellier G, Beyssen B et al. EVA-3S Investigators. Endarterectomy versus stenting in patients with symptomatic severe carotid stenosis. N. Engl. J. Med. 355, 1660–1671 (2006).
4. Endarterectomy for asymptomatic carotid artery stenosis. Executive Committee for the Asymptomatic Carotid Atherosclerosis Study. JAMA 273, 1421–1428, (1995).
5. Hobson RW 2nd, Weiss DG, Fields WS et al. Efficacy of carotid endarterectomy for asymptomatic carotid stenosis. The Veterans Affairs Cooperative Study Group. N. Engl. J. Med. 328, 221–227 (1993).
6. Halliday A, Harrison M, Hayter E et al. Asymptomatic Carotid Surgery Trial (ACST) Collaborative Group. 10‑year stroke prevention after successful carotid endarterectomy for asymptomatic patients (ACST-1): a multicentre randomised trial. Lancet 376, 1074–1084 (2010).
7. Yadav JS, Wholey MH, Kuntz RE et al.; Stenting and Angioplasty with Protection in Patients at High Risk for Endarterectomy Investigators. Protected carotid-artery stenting versus endarterectomy in high-risk patients. N. Engl. J. Med. 351, 1493–1501 (2004).
8. International Carotid Stenting Study Investigators, Ederle J, Dobson J, Featherstone RL et al. Carotid artery stenting compared with endarterectomy in patients with symptomatic carotid stenosis (International Carotid Stenting Study): an interim analysis of a randomised controlled trial. Lancet 375, 985–997 (2010).
9. EVAR trial participants. Endovascular aneurysm repair and outcome in patients unfit for open repair of abdominal aortic aneurysm (EVAR trial 2): randomised controlled trial. Lancet 365(9478), 2187–2192 (2005).
10. Gurm HS, Yadav JS, Fayad P et al. SAPPHIRE Investigators. Long-term results of carotid stenting versus endarterectomy in high-risk patients. N. Engl. J. Med. 358, 1572–1579 (2008).
11. Gray WA, MacDonald S, Schneider PA. Should Medicare reimbursement for carotid artery stenting be extended to standard and low risk symptomatic and asymptomatic patients with carotid stenosis? Vascular 20, 7–11 (2012).
12. Bonati LH, Jongen LM, Haller S et al. New ischaemic brain lesions on MRI after stenting or endarterectomy forsymptomatic carotid stenosis: a substudy of the International Carotid Stenting Study (ICSS). Lancet Neurol. 9(4), 353–362 (2010).

▪ Websites

101. Webster’s New World College Dictionary. www.yourdictionary.com/alternative
102. Vascular surgeons, interventionalists clash over dueling carotid guidelines. Heartwire. www.theheart.org/article/1365611.do