Patient-level data: a paradigm shift in clinical trial transparency?

Corresponding Author:

Divaka Perera
Department of Cardiology, Cardiovascular Division
St Thomas’ Hospital Campus, King’s College
London, SE1 7EH, UK
E-mail: divaka.perera@kcl.ac.uk

Abstract

Published clinical trials have long formed the backbone of evidence-based medicine. The rigorous peer review process prior to publication of clinical trials supposedly ensures accurate and honest reporting of clinical outcomes. Regrettably, this is not always the case. Final publications often represent a distorted version of the trial with inaccurate descriptions of the design, conduct and results.

Keywords

clinical trials, evidence-based medicine, medical statistics, patient-level data, pharmaceutical industry, trial methodology

Published clinical trials have long formed the backbone of evidence-based medicine. The rigorous peer review process prior to publication of clinical trials supposedly ensures accurate and honest reporting of clinical outcomes. Regrettably, this is not always the case. Final publications often represent a distorted version of the trial with inaccurate descriptions of the design, conduct and results [1-4].

These published summaries of clinical trial outcomes form only a small part of available clinical data with many studies incomplete, abandoned or unpublished. Analyses of research protocols submitted to review boards, ethics committees and regulatory authorities have shown that less than 50% lead to publications [5-9]. As a result, the evidence that is widely available to the practicing physician and, indeed, guideline committees, represents only the tip of the evidence iceburg. Selective reporting of trials refers to researcher bias towards the reporting and submission of clinical trials with positive or interesting results; these trials being more than twice as likely to be reported than trials with nonsignificant outcomes [10]. Those trials thought to be most likely to lead to a change in clinical practice or indeed financial gain to a trial sponsor are also more likely to be put forward for publication [5].

Medical journals are also guilty of such bias, also being less likely to publish inconclusive or negative trials. As a result, a large body of clinical trial evidence essentially remains hidden, despite its obvious value.

In 2005, in a move to enhance transparency in clinical trials, the International Committee of Medical Journal Editors initiated a policy requiring investigators to deposit information about trial design into an accepted clinical trials registry before the onset of patient enrollment [101]. This document, which has gained widespread acceptance across a range of biomedical researchers and journals, proposed mandatory registration of all trials in a public trials registry before the start of patient recruitment, in order to be considered for publication. Prior to this, there was no robust method to identify what and how many trials were being conducted and by whom. Furthermore, there was no prospective public documentation of trial design or methodology. As a result, the accurate reporting of clinical trials was based on the trust of trial sponsors and representative. Prior to the International Committee of Medical Journal Editors policy, only 13,153 trials were registered on ClinicalTrials.gov, the largest registry at the time, but within a month this number had climbed to over 22,000 [11]. At the time of writing, over 150,000 studies are listed.

The compulsory prospective registration of clinical trials to allow publication in the major medical journals represented a big step towards greater transparency of clinical trials, allowing greater scrutiny of published design and methodology compared with the original study protocol documents. The obstacle of ensuring that clinical trial results are honestly and accurately reported has yet to be overcome. Furthermore, results of unpublished or abandoned clinical trials essentially belong to the trial sponsors and are not publicly available [12].

Clinical trials report summary data as the most digestible way to communicate results and perform statistical analysis. The transformation of patient-level data to summary data leads to a substantial loss of information. Protocols for summarizing data should be prespecified to minimize any subjectivity, as different methods for summarizing data can lead to significantly different conclusions [13]. To perform rigorous scrutiny of published clinical trials, or to examine results from unpublished or abandoned clinical trials, investigators need access to patient-level data. There has been a growing movement calling for just this: that patient-level data and complete study protocols, including amendments, are made available for public scrutiny from both industry- and nonindustry-funded research [12]. In 2012, the British Medical Journal declared that it would only publish trials of drugs and medical devices if the authors commit to making the relevant anonymized patient-level data available on reasonable request [14]. Both The Annals of Internal Medicine and PLoS Medicine have similar policies on data sharing [102,103]. This represented a significant step in ensuring the credibility of published clinical trials could be verified; however, it did not address the problem of unpublished and abandoned trials.

“The increasing emphasis and recognized importance of misreported, unpublished and abandoned clinical trials is gaining momentum and can be seen as a paradigm shift in clinical trial transparency...”

With respect to the public availability of patient-level data, several major steps toward this have recently been made. Following pressure from the Nordic Cochrane Centre (Copenhagen, Denmark) [15], the EMA has proposed a policy of making its patient-level data publically available for approved medications [16,17]. Perhaps the most significant of steps is the pledge of Glaxo- SmithKline (GSK; London, UK) to provide, on request, de-identified patient-level data for all clinical trials conducted since January 2007.

To obtain these data, investigators will need to submit a proposal to GSK describing their analysis plans, conflicts of interests and list of qualifications. These research proposals will then be reviewed by a panel of external experts appointed by GSK [18]. Roche (Basel, Switzerland) have since introduced a similar policy [104]. Although a progressive step, GSK and Roche will maintain overall control of who is given access to the data and for what purpose, and only time will tell whether data are made available to all suitably qualified research groups.

For some, the availability of patient-level data on request is not enough. A group led by Doshi and colleagues call for sponsors and investigators to publish or republish all unpublished, abandoned or misreported clinical trials within the next year [19]. They call the concept ‘restoring invisible and abandoned trials’. The group has access to over 178,000 pages of clinical trial reports made public through litigation battles and EMA. With these documents they plan to publish all previously unpublished data or reanalyze and publish any misreported data, if the sponsors fail to do so within 1 year; so-called restorative authorship. The article contains a list of abandoned and misreported clinical trials. If sponsors fail to publish/republish these clinical trials within 1 year, the group calls for the data to be made public and also calls for volunteer restorative authors to come forward to help with publication.

The increasing emphasis and recognized importance of misreported, unpublished and abandoned clinical trials is gaining momentum and can be seen as a paradigm shift in clinical trial transparency, which may well profoundly affect evidence-based medicine. Interventional cardiology is a specialty that is rich in clinical trial data, and will undoubtedly be impacted. It is likely that, in the near future, we will see greater numbers of restorative publications questioning previously held beliefs about the efficacy and safety of medications. In Doshi and colleagues call to ‘publish or be published’ [19], Bristol- Myers Squibb (NY, USA) have been requested to publish/republish or make publically available all clinical trial reports from its studies on clopidogrel, including the seminal CURE and CLARITY studies [20,21]. Is it possible that the role of one our most familiar and closely trusted medications could come into question? It is possible that further high-profile drugs and devices in interventional cardiology will come into question as the restoring invisible and abandoned trials movement gains momentum.

“Which clinical trial publications can we trust, or should we wait for the verification of results from restorative publications based on patient-level data?”

Greater transparency in the conduct of clinical trials and the availability of patient-level data can only be positive for patients and physicians. Although the majority of cardiologists will not be involved in the analysis of patient-level data, some will see it as an opportunity to become restorative authors.

Whether directly involved or not, the fact that major journals mandate the availability of patient-level data prior to publication will act as a quality-assurance process. It will mean that industry and nonindustry researchers will need to ensure that protocols for summarizing data and defining end points are prespecified and rigorous. We hope the considerable pressure on the pharmaceutical industry, as a whole, to be more transparent will lead to an increasing amount of data made available to the public and, importantly, the regulatory committees. In turn, it is possible that there will be an increasing number of trials published with a negative outcome.

It does, however, make the interpretation of trial data more difficult. Can we justify the prescription of medications until all the available data has been reviewed? Which clinical trial publications can we trust, or should we wait for the verification of results from restorative publications based on patient-level data? Although, at present, it has been large pharmaceutical companies that have been challenged to provide greater transparency, it is almost certain that device trials will undergo the same scrutiny, again unquestionably impacting the world of the interventional cardiologist.

Financial and competing interests disclosure

The authors have no relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript. This includes employment, consultancies, honoraria, stock ownership or options, expert testimony, grants or patents received or pending, or royalties.

No writing assistance was utilized in the production of this manuscript.

References

1. Song F, Parekh S, Hooper L et al. Dissemination and publication of research findings: an updated review of related biases. Health Technol. Assess. 14(8), iii, ix-xi, 1-193 (2010).
2. Turner EH, Matthews AM, Linardatos E, Tell RA, Rosenthal R. Selective publication of antidepressant trials and its influence on apparent efficacy. N. Engl. J. Med. 358(3), 252-260 (2008).
3. Melander H, Ahlqvist-Rastad J, Meijer G, Beermann B. Evidence b(i)ased medicine - selective reporting from studies sponsored by pharmaceutical industry: review of studies in new drug applications. BMJ 326(7400), 1171-1173 (2003).
4. McGauran N, Wieseler B, Kreis J, Schüler YB, Kölsch H, Kaiser T. Reporting bias in medical research - a narrative review. Trials 11, 37 (2010).
5. ee K, Bacchetti P, Sim I. Publication of clinical trials supporting successful new drug applications: a literature analysis. PLoS Med. 5(9), e191 (2008).
6. Pich J, Carné X, Arnaiz JA, Gómez B, Trilla A, Rodés J. Role of a research ethics committee in follow-up and publication of results. Lancet 361(9362), 1015-1016 (2003).
7. Stern JM, Simes RJ. Publication bias: evidence of delayed publication in a cohort study of clinical research projects. BMJ 315(7109), 640-645 (1997).
8. Dickersin K, Min YI, Meinert CL. Factors influencing publication of research results. Follow-up of applications submitted to two institutional review boards. JAMA 267(3), 374-378 (1992).
9. Blümle A, Antes G, Schumacher M, Just H, von Elm E. Clinical research projects at a German medical faculty: follow-up from ethical approval to publication and citation by others. J. Med. Ethics 34(9), e20 (2008).
10. Chan AW, Hróbjartsson A, Haahr MT, Gøtzsche PC, Altman DG. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA 291(20), 2457-2465 (2004).
11. Laine C, Horton R, De Angelis CD et al. Clinical trial registration: looking back and moving ahead. JAMA 298(1), 93-94 (2007).
12. Doshi P, Jefferson T, Del Mar C. The imperative to share clinical study reports: recommendations from the Tamiflu experience. PLoS Med. 9(4), e1001201 (2012).
13. Chan AW, Tetzlaff JM, Altman DG et al. SPIRIT 2013 statement: defining standard protocol items for clinical trials. Ann. Intern. Med. 158(3), 200-207 (2013).
14. Godlee F. Clinical trial data for all drugs in current use. BMJ 345, e7304 (2012).
15. Gøtzsche PC, Jørgensen AW. Opening up data at the European medicines agency. BMJ 342, d2686 (2011).
16. Doshi P, Jefferson T. The first 2 years of the European medicines agency’s policy on access to documents: secret no longer. JAMA Intern. Med. 173(5), 380-382 (2013).
17. Eichler HG, Abadie E, Breckenridge A, Leufkens H, Rasi G. Open clinical trial data for all? A view from regulators. PLoS Med. 9(4), e1001202 (2012).
18. Nisen P, Rockhold F. Access to patient-level data from glaxosmithkline clinical trials. N. Engl. J. Med. 475-478 (2013).
19. Doshi P, Dickersin K, Healy D, Vedula SS, Jefferson T. Restoring invisible and abandoned trials: a call for people to publish the findings. BMJ 346, f2865 (2013).
20. Yusuf S, Zhao F, Mehta SR, Chrolavicius S, Tognoni G, Fox KK. Effects of clopidogrel in addition to aspirin in patients with acute coronary syndromes without ST-segment elevation. N. Engl. J. Med. 345(7), 494-502 (2001).
21. Sabatine MS, Cannon CP, Gibson CM et al. Addition of clopidogrel to aspirin and fibrinolytic therapy for myocardial infarction with ST-segment elevation. N. Engl. J. Med. 352(12), 1179-1189 (2005).

▪ Websites

101. Editorial clinical trial registration: a statement from the International Committee of Medical Journal Editors. www.icmje.org/clin_trial.pdf
102. Annals of Internal Medicine. Information for authors: reproducible research. http://annals.org/public/authorsinfo. aspx#research-publication-ethics
103. PLoS Medicine. PLoS editorial and publishing policies: sharing of materials, methods, and data. www.plosmedicine.org/static/policies.actionjs essionid=0CD27C4DE9841485620ED21B9 E815D54#sharing
104. Roche global policy on sharing of clinical trials data. http://roche-trials.com/dataSharingPolicy. action