Brain injury clinical trials: new agents or new statistics?

Abstract

Failure of the vast majority of clinical trials evaluating recovery after severe brain injury from stroke or trauma has triggered interest in novel statistical techniques that are more powerful than conventional dichotomized outcomes. A retrospective analysis of data from a large international trial evaluating high-dose steroids for severe traumatic brain injury found that analysis of a wide range of outcome levels by using an ordinal scale with proportional odds regression or a sliding dichotomy was more likely to detect a treatment effect than the single-dichotomy approach. These techniques may soon become commonplace as critical care research increasingly focuses on patient-centered functional outcomes.

The results of clinical trials for brain injury, be it stroke or trauma, have tended to follow the same pattern: animal model, preclinical, and phase II studies all show great promise only to be followed by the disappointment of a negative phase III trial. Dozens of negative clinical trials for stroke and traumatic brain injury (TBI) have been published, and we are left with few, if any, pharmaco-logical treatment options, particularly for neuroprotection.

Several hypotheses have been proposed to explain why the majority of neuroprotective agents have failed to show benefit in clinical trials. These include lack of a meaningful biological treatment effect, unrealistically prolonged windows of time, lack of sufficient statistical power, or the choice of an inadequate or inappropriate outcome or statistical analysis plan. Roozenbeek and colleagues [1], in a previous issue of Critical Care, addressed the last of these concerns. The authors examined data from the Corticosteroid Randomization After Significant Head Injury (CRASH) trial [2, 3] to illustrate how alternative methodological and statistical analyses increase statistical power, reduce sample size requirements, and allow efficacy to be detected.

Like those who have conducted analyses for other agents for TBI [4] and stroke [5], the authors evaluated two different statistical methods: proportional odds logistic regression (a type of polytomous logistic regression) and the sliding-dichotomy approach. Both methods have been used in epidemiological studies [6] and increasingly in clinical trials [7–9]. They have common strengths, including accounting for floor and ceiling effects in a chosen outcome, improved statistical power, and accounting for variation in the quality and range of recovery after brain injury. A lot of clinically relevant information is lost when clinical efficacy is evaluated as a simple yes/no outcome.

In proportional odds logistic regression, outcomes are not treated as binary but as distributions across an ordered ordinal scale. The analysis thus yields an odds ratio (OR) that can be interpreted as the odds of improving one level of outcome across the total range of the scale. In this approach, it is common to combine death with severe impairment (that is, bedbound and completely dependent) because decisions to withhold or withdraw support can influence which group a patient lands in and because there is no consensus that one fate is clearly worse than the other.

In the case of the CRASH trial, the authors collapsed the Glasgow Outcome Scale into four categories, combining vegetative state and severe disability, and noted a common OR of 1.15 (P = 0.0007), suggesting that the observed treatment effect affected survival and recovery across the entire range of outcome categories. The authors also demonstrated improved statistical power with this method. This approach, however, may have some important pitfalls. First, trials need to specify a priori effects across an ordinal scale, and the proportional odds assumption needs to be formally tested. If the likelihood of improving one level is not similar across the entire range of defined outcomes, interpretation of the common OR is highly problematic. Second, care is required in the clinical interpretation of a common OR, particularly when discussing treatment with a family. For instance, for some patients, it may be relevant only whether they become fully independent or not. Third, ordinal scales cannot be considered truly linear, even though the points assigned are. For example, it is impossible to know whether the 'meaning' of transitioning from moderate disability to good recovery is the same as that of transitioning from death or a vegetative state to severe disability.

The authors also examined the sliding-dichotomy approach by using logistic regression, in which the OR can be interpreted as the odds of having a better outcome than would have been predicted. This approach was even more statistically efficient than the proportional odds analysis and has obvious clinical interpretability and is a concept that may be easy to convey to families in discussions on treatment. In this study, the authors included pupil reactivity, Glasgow Coma Scale, age, and systemic trauma to derive the pretreatment odds of an outcome and classified study participants into that band. The major pitfall with this Bayesian type of approach is that it requires a robust and well-validated predictive model for long-term outcome, and this may be the case for stroke with the National Institutes of Health stroke scale or the APACHE II (Acute Physiology and Chronic Health Evaluation II) scale for critical care, but not for other diseases.

Finally, it is important to note that increased power for detecting differences in outcome is crucial not only for detecting a therapeutic benefit but also for detecting harm. We often think of intensivists and trauma surgeons as people who save lives, but a clever statistician can do so as well. This is particularly relevant to the CRASH trial, in which the observed treatment effect was increased mortality in the active treatment arm. Statistical analysis plans that combine a more powerful set of techniques for detecting between-group differences with interim analyses can help us in our obligation to optimize patient safety. In other words, the sooner we can be confident that a treatment such as high-dose steroids for TBI actually kills patients, the sooner the study can be stopped and the greater the number of lives that will be saved. In the future, trials on brain injury or other critical care conditions such as sepsis, fulminant liver failure, or acute respiratory distress syndrome should consider these increasingly creative and powerful methodological and statistical approaches.