Analysis of the Modified Rankin Scale in Randomized Controlled Trials of Acute Ischemic Stroke: A Systematic Review

Laura Jake*

Department of Health Sciences, University of Leicester, Nottingham, UK

*Corresponding Author:

Laura Jake Department of Health Sciences, University of Leicester, Nottingham, UKE-mail: Jakelaura597@edu.uk

Received: 04-Jul-2022, Manuscript No. JESTM-22-40114; Editor assigned: 07-Jul-2022, PreQC No. JESTM-22-40114 (PQ); Reviewed: 21-Jul-2022, QC No. JESTM-22-40114; Revised: 27-Jul-2022, Manuscript No. JESTM-22-40114 (R); Published: 31-Jul-2022, DOI: 10.37532/ jestm.2022.14(4).79-84

Abstract

Abstract

Historically, trials in acute stroke have largely been unable to show benefit of new interventions. Trials have previously favored dichotomous analysis of outcome measures employing an ordinal scale, such as the Modified Rankin Scale (mRS). In 2007, the OAST Collaboration showed that preserving the ordinal nature of these scales increased statistical power, recommending the use of ordinal logistic regression where proportional odds could be assumed. A systematic review of trials and protocols published since 2007 was conducted to re-evaluate statistical methods used and assess whether practice has changed.

Methods

Searches of electronic databases identified trials published between Jan 2007 and July 2014 in acute ischemic stroke using an ordinal measure of dependency as the primary outcome. Published protocols were also identified to evaluate proposed statistical analyses.

Results

Forty-two RCT results publications were identified. The majority of studies used a dichotomous analysis (25, 59.5%), eight (21.4%) retained the ordinal scale and nine (19.0%) used another type of analysis (two of which used a sliding dichotomy). Sixteen published protocols were identified, nine (56.3%) of which intend to use an ordinal method of analysis. Six (37.5%) intend to use a dichotomous analysis and one trial a sliding dichotomy (6.3%).

 

Conclusions

Trials published since 2007 still favour dichotomous analyses over ordinal. Assessment of ongoing trial protocols shows that ordinal analyses are being incorporated more often, although trials with a published protocol may reflect a biased sample of all trials. Stroke trials, where appropriate, should retain the ordinal nature of dependency scales.

Introduction

The modified Rankin Scale (mRS) is a 7-level ordered categorical scale capturing levels of patient functional independence following a stroke, with scores ranging from 0 (fully independent) to 6 (dead). The mRS has been reported to be a valid and reliable endpoint in randomized clinical trials and as such it is a common and recommended outcome measure in acute ischaemic stroke studies [1].

Historically, clinical trials in acute ischaemic stroke have largely been unable to show statistical benefit of therapy over control. This failure has been attributed to multiple causes, including the relevance of laboratory findings to clinical stroke, inadequate sample size, choice of primary outcome, and its statistical analysis. The majority of trials have previously favoured dichotomous analysis of outcome measures that employ an ordinal scale. However, previous reviews of stroke outcomes have suggested that the choice of analytical methods has been less than optimal [2]. The OAST collaboration published a reanalysis of stroke outcomes using alternative statistical methods in 2007 and showed that methods preserving the ordinal nature of the original data were the most optimal. Ordinal logistic regression (OLR) was shown to provide the most statistically efficient analysis of ordinal outcome scales when the proportional odds assumption was met, permitting trial sample size to be reduced compared to dichotomous analysis. This along with other related works led the European Stroke Organisation Outcomes Working Group to recommend that trialists move away from dichotomous outcomes and chose an analysis approach based on the type of patients to be recruited and the likely mechanism of the intervention to be tested [3].

The primary objective of this systematic review is to provide an updated evaluation of statistical methods used in the analysis of the mRS in clinical trials of acute ischaemic stroke published from 2007 to 2014. Given the recommendations made by the OAST collaboration in 2007, it is pertinent to assess whether these findings have influenced more recent trends in analysis of ordinal outcomes in acute stroke studies.

Materials and Methods

Overlapping search strategies were conducted in order to identify a complete list of trials for systematic review. National Centre for Biotechnology Information (NCBI) PubMed, Ovid MEDLINE, and Cochrane Collaboration Trials electronic databases were accessed in July 2014. Publications citing the OAST collaboration findings were also reviewed to detect potentially eligible studies. Care was taken to record only the original publication of trial results, and subsequent publications and subgroup analyses were not included.

Keywords “stroke”, “ischemic”, “randomized”, and “Rankin” were used, accounting for differences in spelling and combination depending on the database used. The systematic review sought to include prospective, randomized, phase III studies in acute ischaemic stroke using the mRS in the primary outcome of the trial. Trials using the Oxford Handicap Scale (OHS), a very close variant of the mRS, were also included. The search was further restricted to studies published in English, from the year 2007 until July 2014. Studies of stroke prevention, haemorrhagic stroke, and those that did not involve the mRS in the primary outcome were excluded from the review [4].

Titles and abstracts of studies were screened in order to identify potentially eligible studies. The full texts of relevant publications were subsequently obtained and reviewed to finalise the complete list of eligible studies, excluding those that did not meet the full inclusion criteria.

Data for the primary objective of the review was collected from the full text of each publication and included the trial name, year of publication, number of randomized participants, intervention tested, and followup time. Additionally, the named method of analysis used in evaluation of the primary outcome measure, definition of favourable mRS outcome where applicable [5], and statement of the study result were also recorded

Results

A total of 192 publications were identified using the search methods after removal of duplicates. Screening of the study abstracts identified 76 potentially relevant clinical trials in ischaemic stroke using the mRS in the primary outcome. Eighteen studies were excluded as being nonrandomized, observational, retrospective, or pilot studies, originally published prior to 2007; trials in stroke prevention; or those not using mRS in the primary outcome.

A total of 42 clinical trial publications were eligible, incorporating a total of 32,432 participants, with studies ranging in size from 37 to 4,071 randomized individuals (Table 1). Nine (21.4%) trials were positive, while the vast majority of studies (31 studies, 73.8%) were unable to show benefit of the studied intervention over control. Two trials (4.8%) evaluating candesartan and statin withdrawal showed evidence of harmful intervention. Neuroprotective or neurotrophic compounds comprised a large proportion of studied interventions in 17 (40.5%) published clinical trials. Antiplatelet or thrombolytic therapies were observed in 11 (26.2%) studies, while five (11.9%) trials sought to ameliorate physical symptoms with blood pressure management or by controlling body temperature and fever. Three (7.1%) studies investigated endovascular therapy or catheter device, while two (4.8%) sequential studies evaluated transcranial laser therapy. Three (7.1%) studies concerned the benefit of stroke rehabilitation initiatives, while one (2.4%) study examined the effect of electrical scalp acupuncture treatment [6].

Table 1.Phase III trials in acute ischemic stroke using mRS as primary outcome

Primary outcome measures differed widely across the published studies. Use of the mRS alone was observed in over half of the included studies (24 studies, 57.1%). Thirteen (31.0%) clinical trials used the mRS (or OHS) alongside other outcome measures including the Barthel Index (BI), NIH Stroke Scale (NIHSS), Quality of Life measures EQ- 5D and SF36, Glasgow Outcome Scale (GOS), Gulhane Aphasia Test (GAT), or Primary Stroke Centre (PCS) time. Five (11.9%) studies used a composite endpoint incorporating the mRS plus BI, NIHSS, or GOS scores, with three of these five studies describing a global endpoint with a threshold of result to be achieved on multiple scales.

Outcome was deemed favourable for mRS scores of 0-1 and 0–2 in equal numbers of studies, 10 (23.8%) for each. Only one (2.4%) study defined a favourable outcome to be a mRS score of 0–3 [7]. Three (7.1%) further trials defined favourable outcome scores that differed depending on baseline NIHSS score that is using a sliding dichotomy. Eighteen (35.7%) studies did not specify a desired outcome.

Discussion

Over half of reported studies in acute ischaemic stroke employed dichotomous analysis of an ordinal scale with wide disagreement in the threshold of favourable outcome. This result is similar to the finding by the OAST collaboration in 2007 that almost half of the 55 identified studies used a dichotomous analysis (49%), indicating that dichotomous analyses are still the prevailing choice for analysis of an ordinal scale. Conversely, the OAST collaboration found around 45% of studies to employ analyses of mean or median, compared to a much smaller percentage using the same analyses in this more recent review (9.5%). Merely a fifth of studies showed significant benefit of intervention over control in this review, whereas Duncan et al. (2000) reported a systematic review of 51 studies in which a much higher percentage of studies achieved significant benefit (21 studies, 41%), although none were seen to subsequently influence clinical practice [8]. Less than a quarter of clinical trials chose to utilise analyses appropriate for an ordinal scale; however, a third of trials reported using ordinal analyses in secondary and sensitivity analyses, indicating that trial investigators were aware of these methods. Only two studies reported the NNT alongside the main trial result, despite the OAST recommendation that this measure aids clinical interpretation of the main trial result. One possible explanation for this finding is how regulatory authorities, such as the FDA, authors, and journals, view ordinal analyses. The FDA has only recently accepted no dichotomous approaches for the analysis of ordinal scales. Therefore, trialists may have been reluctant to change their analysis plans while the FDA was reluctant to accept such approaches. There is anecdotal evidence to suggest that people find it hard to interpret results from ordinal analyses in terms of the clinical importance, which may also lead to hesitancy to implement these methods. Finally support for using such methods may increase as larger scale trials using such methods are published. Since the completion of this review a number of trials using an ordinal method of analysis have been published, which may encourage uptake where appropriate [9]. Although not shown here, we also conducted a brief scoping search of published study protocols of ongoing stroke trials. Of the published papers assessed 56% propose using an analysis preserving the ordinal scale, with six studies specifically stating that the analysis of primary outcome will be OLR, which is already numerically greater than the three published studies observed during the systematic review. Although this is a highly selective sample, it may suggest that prevalence of such methods is increasing. Since the publication of the OAST study in 2007, there is continued interest in both developing and testing novel methods for the analysis of ordinal stroke outcomes. Use of the OLR method relies on the proportional odds assumption being met; that is, there is a common shift across cut points. Researchers should use data from previous studies to assess whether it is reasonable to assume this for the intervention being assessed. This assumption may not be met for some stroke treatments; for example, thrombolysis increases the odds of a good outcome but may, in certain circumstances, increase the odds of death. In these situations the partial proportional odds model has been advocated, where the proportional odds assumption is relaxed [10]. This method has been shown to have some advantages over OLR when compared using data from the NINDS thrombolysis trial. Assumption free alternatives have also been suggested, such as the permutation method. Some have argued that another limitation of moving to an ordinal method of analysis is the interpretability of a common odds ratio. Therefore alterative measures of treatment effect have been proposed, although these have had limited uptake. The NNT is a well-recognised measure of absolute treatment effect; an extension of this method for ordinal data has been suggested which may overcome this issue. A limitation of these studies is that they tend to reanalyze data from one study, which makes generalizations to wider stroke trials difficult. Future research should concentrate on consolidating the extensive evidence to date on a large number of diverse trials, such as the OAST data set. Although this review has concentrated on trials in stroke, similar work and findings have been reported in other areas, such as traumatic brain injury and cancer. Although based on different outcome scales the findings from the traumatic brain injury and cancer studies have generally echoed those seen in stroke. To date there has not been a review of practice in trials in these areas to assess whether there has been uptake to the methods proposed. There are some limitations to the work presented here. Firstly, it is advised that a systematic review be conducted and data collected by two independent authors, followed by crosschecking and resolution of disagreement. This review was conducted by a sole author under the supervision of a senior statistician and so it does not benefit from such validation by a second independent author. Secondly, non-English language publications were excluded from the review and as such may limit the generalizability of the findings [11]. However, only eight non- English language papers were identified in the original list of 192 search results, and work by Morrison et al. Found no evidence of systematic bias in language-restricted meta-analyses; thus it is unlikely that limiting the search to English publications will have introduced bias in this review. We only included the results of published trials in this systematic review. A more comprehensive search could have also included data from completed but unpublished studies by searching trial registries such as ClinicalTrials. gov and ISRCTN. There is data to suggest that published studies tend to be larger and show a greater treatment effect than those which are unpublished. Therefore the studies included here may not be representative of all trials conducted during this time, and the results should be viewed with some caution [12,13].

Conclusions

The findings of this systematic review do not indicate a dramatic shift in the analysis of primary functional outcomes following acute ischaemic stroke despite the OAST recommendations; however, there appears to be awareness of the use of these methods and there may be an emerging trend towards more ordinal-appropriate analyses in ongoing and future studies.

Competing Interests

The authors declare that there are no competing interests regarding the publication of this paper

References

1. Rankin J Cerebral vascular accidents in patients over the age of 60: II. Prognosis. Scott Med J. 2, 200–215 (1957).

Indexed at, Google Scholar, Crossref

2. Quinn TJ, Dawson J, Walters MR et al. Functional outcome measures in contemporary stroke trials. Int J Stroke. 4, 200–205 (2009).

Indexed at, Google Scholar, Crossref

3. Lees KR, Bath PMW, Schellinger PD et al. Contemporary outcome measures in acute stroke research: choice of primary outcome measure. Stroke. 43, 1163–1170 (2012).

Indexed at, Google Scholar, Crossref

4. Sulter G, Steen C, Keyser JD et al. Use of the Barthel index and modified Rankin scale in acute stroke trials. Stroke. 30, 1538–1541 (1999).

Indexed at, Google Scholar, Crossref

5.  Grotta J Why do all drugs work in animals but none in stroke patients? 2 Neuroprotective therapy. J Intern Med. 237, 89–94 (1995).

Indexed at, Google Scholar, Crossref

6.  Weaver CS, Leonardi-Bee J, Bath-Hextall et al. Sample size calculations in acute stroke trials: a systematic review of their reporting, characteristics, and relationship with outcome. Stroke. 35, 1216–1224 (2004).

Indexed at, Google Scholar, Crossref

7.  Weaver CS, Leonardi-Bee J, Bath-Hextall et al. Can we improve the statistical analysis of stroke trials? Statistical reanalysis of functional outcomes in stroke trials Stroke. 38, 1911–1915 (1911).

Indexed at, Google Scholar, Crossref

8. Duncan PW, Jorgensen HS, Wade DT et al. Outcome measures in acute stroke trials: a systematic review and some recommendations to improve practice. Stroke. 31, 1429–1438 (2000).

Indexed at, Google Scholar, Crossref

9.  Grotta J Calculation of sample size for stroke trials assessing functional outcome: comparison of binary and ordinal approaches. Int J Stroke. 3, 78–84 (2008).

Indexed at, Google Scholar, Crossref

10.  Bath PMW, Lees KR, Schellinger PD et al. Statistical analysis of the primary outcome in acute stroke trials. Stroke. 43, 1171–1178 (2012).

Indexed at, Google Scholar, Crossref

11.  Bath P, Hogg C, Tracy M et al. Calculation of numbersneeded-to-treat in parallel group trials assessing ordinal outcomes: case examples from acute stroke and stroke prevention. Int J Stroke. 6, 472–479 (2011).

Indexed at, Google Scholar, Crossref

12.  Berkhemer OA, Fransen PSSS, Beumer D et al. A randomized trial of intraarterial treatment for acute ischemic stroke. N Engl J Med. 372, 11–20 (2015).

Indexed at, Google Scholar, Crossref

13. M. Goyal, A. M. Demchuk, B. K. Menon et al. Randomized assessment of rapid endovascular treatment of ischemic stroke. N Engl J Med. 372, 1019–1030 (2015).

Indexed at, Google Scholar, Crossref