Development of objective performance goals for peripheral vascular interventions using real world data sources

Corresponding Author:

Daniel J. Bertges
Division of Vascular Surgery,
University of Vermont Medical Center,
Burlington,
VT,
USA,
E-mail: daniel.bertges@uvmhealth.org

Received date: July 22, 2021 Accepted date: August 05, 2021 Published date: August 12, 2021

Abstract

Objective Performance Goals (OPGs) and Objective Performance Criteria (OPCs) are intended to guide regulatory decisions of medical devices, inform clinical trial designs and may help inform clinical practice guidelines. In the Peripheral Vascular Intervention (PVI) space the wide variety of devices and evolving technologies have made OPC and OPG development challenging. Medical decision-making is complicated by the incremental evolution of devices marketed for the treatment of similar lesions. Difference in specialty training and individual physician bias further complicate treatment decisions including device selection. In addition, the lack of consistent definitions for both common and peripheral arterial disease specific covariates and outcomes has made it difficult to compare the safety and efficacy of devices. This has resulted in heterogeneous treatment pathways for different specialties, an inadequate evidence base for comparative effectiveness, and no clear consensus on the standard of care for peripheral arterial devices. This review discusses the current state of OPGs for PVI. The review focuses on the potential use of coordinated data networks such as clinical registries for new OPG development. The viewpoint proposes a new pathway for the development of “fit-for-purpose” OPGs that harnesses the power of real world data sources.

Keywords

Objective performance goals •Peripheral vascular intervention • Vascular surgery •Registry

Introduction

This review discusses the use of alternative data sources for the development of Objective Performance Goals (OPG) peripheral vascular interventions. The discussion highlights current limitations of randomized trials and Real World Data (RWD) sources in the generation of evidence for OPGs. The Superficial Femoral Artery-Popliteal EvidenceE Development (SPEED) OPGs are used to illustrate the utility of RWD.

Literature Review

Objective Performance Goals (OPGs) and Objective Performance Criteria (OPCs)

Objective Performance Goals (OPGs) and Objective Performance Criteria (OPCs) are intended to guide regulatory decisions of medical devices, inform clinical trial designs and may aid in the creation of clinical practice guidelines. OPCs are a target value, expressed as a point or range of numerical values that is established for the review and comparison of safety and effectiveness endpoints [1]. OPCs generally require are more stringent level of evidence and are developed only after a device technology has matured [1]. OPGs are less robust and intended for the assessment of evolving technologies particularly in situations where there is no equipoise on a satisfactory control group [1]. A detailed discussion of the regulatory uses of OPGs is beyond the scope of this review and is discussed in a variety of quality references [2,3].

Randomized Controlled Trials (RCTs) represent the most rigorous evidence for the construction of OPC/OPGs. However, the existing OPGs for PVI that were derived from trial data date back 12-14 years to publications by the Society of Vascular Surgery and the VIVA group [4,5]. These OPGs were developed from RCTs performed at specialized centers with relatively small populations and highly selected patient with defined anatomies that may not reflect the heterogeneity of patients and lesions encountered in everyday practice. In addition, plain balloon angioplasty, the standard at the time, was used as the comparator. Because of the paucity of quality randomized trials there is a clear need for new data to update obsolete OPGs. The expansion of clinical registries and other coordinated research networks provides an opportunity to meet the need for contemporary OPGs.

Registry Assessment of Peripheral Arterial Interventional Devices (RAPID)

The Registry Assessment of Peripheral arterial Interventional Devices (RAPID) initiative is a public-private partnership between vascular specialists, vascular registries, device manufacturers, and federal regulators which was established to advance the national evaluation of peripheral devices throughout the total product lifecycle [6]. The RAPID initiative emerged from the Predictable and Sustainable Implementation of National (PASSION) Registries for Cardiovascular Devices program of the Medical Device Epidemiology Network (MDEpiNet) [7]. RAPID is a demonstration project for the National Evaluation System for Health Technology coordinating center (NESTcc) [8].

To address the need for harmonious data elements and definition for the evaluation of PAD devices, the RAPID group first developed a set of common data elements [9]. RAPID proposed and published set of PAD-specific core data elements and mechanisms for recording detailed device data which was adopted by the Society for Vascular Surgery (SVS) Vascular Quality Initiative (VQI) Registry [10]. The SVS VQI is a national quality improvement registry designated as a Patient Safety Organization by the Agency for Healthcare Research and Quality. The VQI PVI Registry is one of 12 registries and includes 354 participating centers across the United States and Canada. The VQI is equally comprised of academic, teaching affiliated and community practices. A variety of specialists participate in the registry including vascular surgeons (46%), cardiologists (15%) and interventional radiologist (15%), general surgeons (7%) and others (17%). The VQI has a rich repository of real world date on over 275,000 PVI procedures. Lacking contemporary trial data, such Real-World Data (RWD) derived from registries provides value for fit-for-purpose OPGs. Thus the Superficial Femoral Artery-Popliteal EvidencE Development (SPEED) Study Group was conceived as a time and cost-efficient method to develop OPGs to address existing gaps in knowledge in keeping with the Food and Drug Administration’s (FDA) “least burdensome approach [11].”

SPEED OPGs

To this end RAPID recently published OPGs for femoral-popliteal peripheral vascular interventions [12]. SPEED utilized real-world data on 21,377 procedures from the SVS VQI to construct contemporary OPGs for femoral-popliteal PVI. OPGs were reported for target lesion revascularization at 1-year and major amputation and 4-year mortality (Table 1). The three OPGs were stratified by artery type (femoral vs. popliteal) and device classes including angioplasty, stenting, atherectomy and any treatment.

Table 1: Superficial Femoral Artery-Popliteal EvidencE Development (SPEED) objective performance goals for superficial femoral and popliteal arteries PVI according to treatment type.

The SPEED OPGs may be used for more efficient peripheral device clinical trial design to support regulatory decisions. For example they add to older, historical benchmarks of plain balloon angioplasty. The OPGs are intended to be dynamic in that they may be updated within the VQI registry or through other coordinated registry networks to assess long-term real-world device performance. The SPEED OPGs is a first effort at a registry-based PVI OPG. Recognizing this, the SPEED group intentionally used the term performance “goal” rather than performance “criteria”, which implies a higher evidentiary standard. Proposals intended for regulatory approval with the FDA should be discussed with regulators to refine the OPG to match the specific trial population. In particular the SPEED OPGs may serve as a basis for a propensity- matched, contemporaneous, control group.

Limitations of RWD OPGs

The limitations of real world datasets for clinical research are well known and apply to the generation of OPGs with RWD [13]. Missingness of data and incomplete follow-up is a primary concern. Similar concerns occur even in well-supported randomized controlled trials as demonstrated by the recent paclitaxel controversy in which unexpected, late mortality events were lost to follow-up and later required a concerted effort to rectify [14,15]. Historically the follow-up rate in VQI PVI registry is approximately 70% at one-year [16-18]. Importantly the baseline characteristics were examined and found to be similar between groups with and without follow suggesting random nature to the differential follow-up. To address follow-up the Vascular Implant Surveillance and Interventional Outcomes Network (VISION) has developed a validated process for matching Center for Medicare and Medicaid Services data to the VQI to improve late follow-up of key outcomes [19]. The chronic nature of peripheral arterial disease calls for such longitudinal follow-up of both open and percutaneous revascularizations. In the future linkage to claims will augment follow-up within the registry and further strengthen the data. Consistency of data elements and definitions is important for comparisons across data sources. SPEED utilized the minimum PAD-specific core data set of the RAPID to assure uniformity of patient and procedure characteristics [8].

The relevance of certain endpoints such as target lesion and target vessel revascularization has been questioned. While target lesion revascularization has been an important historical endpoint and a standard for device manufacturers, it does not reflect the full patient experience, particularly for those with intermittent claudication. In recent years the importance of shared decision making in device selection and patient reported outcomes has received more attention. The VQI is in the process of implementing patient reported outcomes measures for PVI using two health related quality of life surveys, the Vascu-QoL-6 and Euro-Qol 5D-5L. refs This effort is consistent with the FDAs strategic priorities for medical device assessment [20].

An important critique of the SPEED OPGs has been the lack of outcome differences across device classes. Device selection was not randomized for SPEED; rather treatment types were selected by physicians to match lesions. However, the purpose of SPEED was to provide a contemporary OPG to compare new devices of the same type. If such devices are analyzed within the same registry and with similar follow-up, then the methodology provides a valid alternative for device evaluation.

A better way forward

Randomized controlled trial remains the best evidence base for OPGs. However, a lack of trial data has hindered the development of OPGs for peripheral interventions. In addition, it is recognized that devices may perform differently in clinical practice outside of trial design [21].

These realities call for a more responsive system that can generate relevant, up-to-date OPGs.

Clinical registries are only one source of real world data that may meet this end. Administrative claims and electronic health records may provide suitable data provided they meet high standards such as those recommended by the FDA [1]. The construction of OPGs for peripheral devices need not be mutually exclusive to Randomized Controlled Trials (RCT), registries or other coordinated data networks. Instead, disparate data sources can be complimentary. Strengthening these alternative data sources will further enhance their utility beyond their primary purposes.

Unique opportunities are emerging in the world of vascular registries with the recent merger of the SVS VQI and American College of Cardiology’s National Cardiovascular Data Registry (NCDR) [22]. This collaboration presents an opportunity to bridge gaps between specialties through joint projects focusing on quality improvement and clinical research including OPG/OPCs and comparative effectiveness. Registries in particular are uniquely suited to study specific patient populations or disease severities that are under-represented in traditional clinical trials. OPGs demanding a higher level of evidence could be derived either from the existing RCT ecosystem or from RCTs embedded in registries. Stakeholders should collaborate to agree on the best source of data for specific OPGs.

Conclusion

Proponents of RCT-based and RWD-based OPGs should engage to improve both systems. Only in this spirit can we improve the national ecosystem of peripheral device evaluation, generate the comparative effectiveness data that are sorely missing and ultimately use this information to serve our patients.

References

1. U.S. Food and Drug Administration. Use of real world evidence to support regulatory decision making for medical devices. Guidance for industry and Food and Drug Administration staff. (2017).
2. Kumar A, Brooks SS, Cavanaugh K, et al. FDA perspective on objective performance goals and clinical trial design for evaluating catheter-based treatment of critical limb ischemia. J Vasc Surg. 50(6): 1474-6 (2009).
3. Pack LK, Buckley DC, Goode JL, et al. FDA perspective on the need for revised performance goals for evaluating stenting of femoropopliteal arteries. Catheter Cardiovasc Interv. 83(6): 984-6 (2014).
4. Conte MS, Geraghty PJ, Bradbury AW, et al. Suggested objective performance goals and clinical trial design for evaluating catheter-based treatment of critical limb ischemia. J Vasc Surg. 50(6): 1462-73.e1-3 (2009).
5. Rocha-Singh KJ, Jaff MR, Crabtree TR, et al. Performance goals and endpoint assessments for clinical trials of femoropopliteal bare nitinol stents in patients with symptomatic peripheral arterial disease. Catheter Cardiovasc Interv. 69(6): 910-9 (2007).
6. MDEpiNet Registry Assessment of Peripheral Interventional Devices (RAPID). (2015).
7. MDEpiNet. (2021).
8. Projects support NESTcc’s mission to establish functional and efficient pathways for medical device evidence generation. (2021).
9. Jones WS, Krucoff MW, Morales P, et al. Registry Assessment of Peripheral Interventional Devices (RAPID): Registry assessment of peripheral interventional devices core data elements. J Vasc Surg. 67(2): 637-644.e30 (2018).
10. Society for Vascular Surgery (SVS) Vascular Quality Initiative (VQI). (2021).
11. Administration USFDA. The Least Burdensome Provisions: Concept and Principles. (2019).
12. Bertges DJ, White R, Cheng YC, et al. Registry Assessment of Peripheral Interventional Devices (RAPID) and Superficial Femoral Artery-Popliteal EvidencE Development (SPEED) Study Group. Registry Assessment of Peripheral Interventional Devices objective performance goals for superficial femoral and popliteal artery peripheral vascular interventions. J Vasc Surg. 73(5): 1702-1714.e11 (2021).
13. Sherman RE, Anderson SA, Dal Pan GJ, et al. Real-World Evidence (RWE)-What is it and what can it tell us? N Engl J Med. 375(23): 2293-2297 (2016).
14. Lottes AE, Whatley EM, Royce SM, et al. Important considerations for trials for peripheral arterial disease: Lessons learned from the paclitaxel mortality signal: A report on behalf of the registry assessment for peripheral interventional Devices (RAPID) Paclitaxel Pathways Program. Am Heart J. 232: 71-83 (2021).
15. Rocha-Singh KJ, Duval S, Jaff MR, et al. Mortality and paclitaxel-coated devices: An individual patient data meta-analysis. Circulation. 141(23): 1859-1869 (2020).
16. Nordanstig J, Wann-Hansson C, Karlsson J, et al. Vascular Quality of Life Questionnaire-6 facilitates health-related quality of life assessment in peripheral arterial disease. J Vasc Surg. 59(3): 700-7 (2014).
17. EQ 5D-5L. About and main differences between the EQ-5D-5L and THE EQ-5D-3L. (2021).
18. Pickard AS, Law EH, Jiang R, et al. United States valuation of EQ-5D-5L health states using an international protocol. Value Health. 22(8): 931-941 (2019).
19. Vascular Implant Surveillance and Interventional Outcomes Network (VISION): SVS VQI-Medicare-Matched datasets. (2021).
20. Value and Use of Patient-Reported Outcomes (PROs) in assessing effects of medical devices CDRH Strategic Priorities 2016-2017. (2021).
21. Patel MR, Jones WS. Peripheral artery disease therapies may perform differently in practice than in randomized trials: The need for learning health systems. JACC Cardiovasc Interv. 9(7): 725-7 (2016).
22. ACC, SVS Join forces on single vascular registry. (2021).