Brain injury clinical trials: new agents or new statistics?
Critical Care volume 15, Article number: 188 (2011)
Failure of the vast majority of clinical trials evaluating recovery after severe brain injury from stroke or trauma has triggered interest in novel statistical techniques that are more powerful than conventional dichotomized outcomes. A retrospective analysis of data from a large international trial evaluating high-dose steroids for severe traumatic brain injury found that analysis of a wide range of outcome levels by using an ordinal scale with proportional odds regression or a sliding dichotomy was more likely to detect a treatment effect than the single-dichotomy approach. These techniques may soon become commonplace as critical care research increasingly focuses on patient-centered functional outcomes.
The results of clinical trials for brain injury, be it stroke or trauma, have tended to follow the same pattern: animal model, preclinical, and phase II studies all show great promise only to be followed by the disappointment of a negative phase III trial. Dozens of negative clinical trials for stroke and traumatic brain injury (TBI) have been published, and we are left with few, if any, pharmaco-logical treatment options, particularly for neuroprotection.
Several hypotheses have been proposed to explain why the majority of neuroprotective agents have failed to show benefit in clinical trials. These include lack of a meaningful biological treatment effect, unrealistically prolonged windows of time, lack of sufficient statistical power, or the choice of an inadequate or inappropriate outcome or statistical analysis plan. Roozenbeek and colleagues , in a previous issue of Critical Care, addressed the last of these concerns. The authors examined data from the Corticosteroid Randomization After Significant Head Injury (CRASH) trial [2, 3] to illustrate how alternative methodological and statistical analyses increase statistical power, reduce sample size requirements, and allow efficacy to be detected.
Like those who have conducted analyses for other agents for TBI  and stroke , the authors evaluated two different statistical methods: proportional odds logistic regression (a type of polytomous logistic regression) and the sliding-dichotomy approach. Both methods have been used in epidemiological studies  and increasingly in clinical trials [7–9]. They have common strengths, including accounting for floor and ceiling effects in a chosen outcome, improved statistical power, and accounting for variation in the quality and range of recovery after brain injury. A lot of clinically relevant information is lost when clinical efficacy is evaluated as a simple yes/no outcome.
In proportional odds logistic regression, outcomes are not treated as binary but as distributions across an ordered ordinal scale. The analysis thus yields an odds ratio (OR) that can be interpreted as the odds of improving one level of outcome across the total range of the scale. In this approach, it is common to combine death with severe impairment (that is, bedbound and completely dependent) because decisions to withhold or withdraw support can influence which group a patient lands in and because there is no consensus that one fate is clearly worse than the other.
In the case of the CRASH trial, the authors collapsed the Glasgow Outcome Scale into four categories, combining vegetative state and severe disability, and noted a common OR of 1.15 (P = 0.0007), suggesting that the observed treatment effect affected survival and recovery across the entire range of outcome categories. The authors also demonstrated improved statistical power with this method. This approach, however, may have some important pitfalls. First, trials need to specify a priori effects across an ordinal scale, and the proportional odds assumption needs to be formally tested. If the likelihood of improving one level is not similar across the entire range of defined outcomes, interpretation of the common OR is highly problematic. Second, care is required in the clinical interpretation of a common OR, particularly when discussing treatment with a family. For instance, for some patients, it may be relevant only whether they become fully independent or not. Third, ordinal scales cannot be considered truly linear, even though the points assigned are. For example, it is impossible to know whether the 'meaning' of transitioning from moderate disability to good recovery is the same as that of transitioning from death or a vegetative state to severe disability.
The authors also examined the sliding-dichotomy approach by using logistic regression, in which the OR can be interpreted as the odds of having a better outcome than would have been predicted. This approach was even more statistically efficient than the proportional odds analysis and has obvious clinical interpretability and is a concept that may be easy to convey to families in discussions on treatment. In this study, the authors included pupil reactivity, Glasgow Coma Scale, age, and systemic trauma to derive the pretreatment odds of an outcome and classified study participants into that band. The major pitfall with this Bayesian type of approach is that it requires a robust and well-validated predictive model for long-term outcome, and this may be the case for stroke with the National Institutes of Health stroke scale or the APACHE II (Acute Physiology and Chronic Health Evaluation II) scale for critical care, but not for other diseases.
Finally, it is important to note that increased power for detecting differences in outcome is crucial not only for detecting a therapeutic benefit but also for detecting harm. We often think of intensivists and trauma surgeons as people who save lives, but a clever statistician can do so as well. This is particularly relevant to the CRASH trial, in which the observed treatment effect was increased mortality in the active treatment arm. Statistical analysis plans that combine a more powerful set of techniques for detecting between-group differences with interim analyses can help us in our obligation to optimize patient safety. In other words, the sooner we can be confident that a treatment such as high-dose steroids for TBI actually kills patients, the sooner the study can be stopped and the greater the number of lives that will be saved. In the future, trials on brain injury or other critical care conditions such as sepsis, fulminant liver failure, or acute respiratory distress syndrome should consider these increasingly creative and powerful methodological and statistical approaches.
Corticosteroid Randomization After Significant Head Injury
traumatic brain injury.
Roozenbeek B, Lingsma HF, Perel P, Edwards P, Roberts I, Murray GD, Maas AI, Steyerberg EW, the IMPACT (International Mission on Prognosis and Clinical Trial Design in Traumatic Brain Injury) Study Group and the CRASH (Corticosteroid Randomisation After Significant Head Injury) Trial Collaborators: The added value of ordinal analysis in clinical trials: an example in traumatic brain injury. Crit Care 2011, 15: R127. 10.1186/cc10240
Edwards P, Arango M, Balica L, Cottingham R, El-Sayed H, Farrell B, Fernandes J, Gogichaisvili T, Golden N, Hartzenberg B, Husain M, Ulloa MI, Jerbi Z, Khamis H, Komolafe E, Laloë V, Lomas G, Ludwig S, Mazairac G, Muñoz Sanchéz Mde L, Nasi L, Olldashi F, Plunkett P, Roberts I, Sandercock P, Shakur H, Soler C, Stocker R, Svoboda P, Trenkler S, Venkataramana NK, Wasserberg J, Yates D, Yutthakasemsunt S, CRASH trial collaborators: Final results of MRC CRASH, a randomised placebo-controlled trial of intravenous corticosteroid in adults with head injury-outcomes at 6 months. Lancet 2005, 365: 1957-1959.
Roberts I, Yates D, Sandercock P, Farrell B, Wasserberg J, Lomas G, Cottingham R, Svoboda P, Brayley N, Mazairac G, Laloë V, Muñoz-Sánchez A, Arango M, Hartzenberg B, Khamis H, Yutthakasemsunt S, Komolafe E, Olldashi F, Yadav Y, Murillo-Cabezas F, Shakur H, Edwards P, CRASH trial collaborators: Effect of intravenous corticosteroids on death within 14 days in 10008 adults with clinically significant head injury (MRC CRASH trial): randomised placebo-controlled trial. Lancet 2004, 364: 1321-1328.
Murray GD, Barer D, Choi S, Fernandes H, Gregson B, Lees KR, Maas AI, Marmarou A, Mendelow AD, Steyerberg EW, Taylor GS, Teasdale GM, Weir CJ: Design and analysis of phase III trials with ordered outcome scales: the concept of the sliding dichotomy. J Neurotrauma 2005, 22: 511-517. 10.1089/neu.2005.22.511
Bath PM, Gray LJ, Collier T, Pocock S, Carpenter J: Can we improve the statistical analysis of stroke trials? Statistical reanalysis of functional outcomes in stroke trials. Stroke 2007, 38: 1911-1915.
Willey JZ, Disla N, Moon YP, Paik MC, Sacco RL, Boden-Albala B, Elkind MS, Wright CB: Early depressed mood after stroke predicts long-term disability: the Northern Manhattan Stroke Study (NOMASS). Stroke 2010, 41: 1896-1900. 10.1161/STROKEAHA.110.583997
den Hertog HM, van der Worp HB, van Gemert HM, Algra A, Kappelle LJ, van Gijn J, Koudstaal PJ, Dippel DW, PAIS Investigators: The Paracetamol (Acetaminophen) In Stroke (PAIS) trial: a multicentre, randomised, placebo-controlled, phase III trial. Lancet Neurol 2009, 8: 434-440. 10.1016/S1474-4422(09)70051-1
Mendelow AD, Gregson BA, Fernandes HM, Murray GD, Teasdale GM, Hope DT, Karimi A, Shaw MD, Barer DH, STICH investigators: Early surgery versus initial conservative treatment in patients with spontaneous supratentorial intracerebral haematomas in the International Surgical Trial in Intracerebral Haemorrhage (STICH): a randomised trial. Lancet 2005, 365: 387-397.
Sacco RL, DeRosa JT, Haley EC Jr, Levin B, Ordronneau P, Phillips SJ, Rundek T, Snipes RG, Thompson JL, Glycine Antagonist in Neuroprotection Americas Investigators: Glycine antagonist in neuroprotection for patients with acute stroke: GAIN Americas: a randomized controlled trial. JAMA 2001, 285: 1719-1728. 10.1001/jama.285.13.1719
The authors declare that they have no competing interests.
About this article
Cite this article
Willey, J.Z., Mayer, S.A. Brain injury clinical trials: new agents or new statistics?. Crit Care 15, 188 (2011). https://doi.org/10.1186/cc10350
- Traumatic Brain Injury
- Ordinal Scale
- Severe Traumatic Brain Injury
- Glasgow Outcome Scale
- Statistical Analysis Plan