Skip to main content

Severe sepsis: are PROWESS and PROWESS-SHOCK trials comparable? A clinical and statistical heterogeneity analysis

Abstract

Despite the same manufacturer, the same drotrecogin alfa activated dose, and the same placebo-controlled design, the negative result from the PROWESS-SHOCK trial contradicted the survival benefit observed in the PROWESS trial. We hypothesize that the different results were due to factors other than the experimental therapy and performed an analysis of the clinical heterogeneity (differences related to the trials' clinical aspects) and the statistical heterogeneity (differences related to the trials' statistical aspects) between these trials. Baseline characteristics and co-interventions were analyzed by chi-square testing and mortality was analyzed by random-effects modeling and I2. Our findings show that clinical variables presented significant heterogeneity, and that up to 90% of the mortality differences between both trials were not due to chance. These results demonstrate that PROWESS and PROWESS-SHOCK are not comparable trials due to the highly significant clinical and statistical heterogeneity. We propose a new and pragmatic solution.

Drotrecogin alfa activated (DAA) was approved for treatment of patients with severe sepsis in 2001 based on a large randomized double-blind clinical trial, PROWESS [1]. A second randomized clinical trial, PROWESS-SHOCK [2], was recently completed, but the survival benefits observed with the original trial were not reproduced in the second trial. The results from PROWESS-SHOCK culminated in the removal of this drug from the market in 2011. Both trials were multicenter, randomized, and double-blinded, and both used the same drug manufacturer. Which trial should we believe in? What should be done while one-third of our patients with severe sepsis are still dying despite the best standard of care? Our paper aims to explore the reasons for this discrepancy and offer new solutions.

A total of 3,370 patients with severe sepsis were enrolled in both trials, for which we performed an analysis of the clinical heterogeneity (differences related to the trials' clinical aspects) and the statistical heterogeneity (differences related to the trials' statistical aspects) between these trials. Baseline characteristics, infection etiologies and sites, and co-interventions were compared by chi-square testing for the clinical analysis, while random-effects modeling and I2 were performed for the statistical analysis. All results are shown in Tables 1 and 2. Our clinical findings demonstrate that infection sites, etiology, co-interventions, and geographic enrollment were all significantly different between the two trials. Moreover, the use of appropriate antibiotics, low-dose steroids, and heparin were all significantly different. Based on 28-day mortality, we also found a highly significant statistical heterogeneity: up to 90% of the mortality differences between the trials were not due to chance. This heterogeneity remained consistently high even when the analysis was done by shock status, number of organ failures, or APACHE II (Acute Physiology and Chronic Health Evaluation II) scores.

Table 1 Clinical heterogeneity: PROWESS versus PROWESS-SHOCK - trial characteristics
Table 2 Statistical heterogeneity: PROWESS versus PROWESS-SHOCK - 28-day mortality

A recent study by Levy and colleagues [3] showed that the absolute mortality of severe sepsis is different between the US and EU; hence, the very different rates of geographic enrollment between PROWESS and PROWESS-SHOCK may explain, in part, the mortality differences. Infection site is known to be a major determinant of survival in patients with severe sepsis, so the significant differences we found regarding these sites provide more corroborating evidence of clinical heterogeneity. The diversity of microbiological etiologies between studies also provides evidence for the clinical differences; similarly, the rate of appropriate use of antibiotics was not comparable. Let alone the fact that co-interventions (for example, heparin and low-dose steroids) were significantly different between trials.

The statistical heterogeneity analysis demonstrates that the vast majority (80 to 90%) of the detected heterogeneity regarding survival outcomes between these trials could not be explained by chance. This is quite remarkable because it points out that the reasons for this large heterogeneity derive from differences related to the trials themselves, in this case, patient population, baseline infection, and co-interventions. Moreover, even after we stratified the survival outcome analysis by disease severity, the elevated heterogeneity did not change. We conjecture that the different clinical characteristics and co-interventions were most likely the cause for this irreversible statistical heterogeneity.

Another complicating factor is that the PROWESS-SHOCK trial was substantially underpowered: 42% probability of false-negative results. Although a frequent question since the completion of PROWESS-SHOCK has been 'which trial should we believe in?', we propose that this is not the 64 million-dollar question; the one that is begging for an answer is 'will we ever be able to replicate the design of the PROWESS trial?' If we aim for a control mortality of 35%, the answer is 'no' because recent phase III trials [2, 4] have shown that the mortality associated with severe sepsis now ranges from 24 to 28%. If we slightly modify the question to 'can we perform another phase III trial on DAA with adequate statistical power?', the answer is yes on two accounts: 1) a large sample size (N = 2,500 to 3,000) would fulfill the frequentist (classical) statistical approach; and 2) a smaller sample size (500 to 1,000) would fulfill the adaptive Bayesian statistical approach, as we explained in a previous manuscript [5]. What about financial support? The financial and logistic challenges would be enormous for the frequentist approach, but definitely more feasible for the Bayesian approach. Would it be ethical to perform a third trial? Yes, a study we published recently [6] demonstrated that, in real-life application outside phase III trials, DAA significantly reduced in-hospital mortality by 18% (95% confidence interval 13 to 22%) in patients with severe sepsis (N = 41,401 patients). How would this trial be designed? First, an individual-patient data meta-analysis to combine all randomized trials at the patient-level would provide the most accurate and statistically powerful way to reduce the current scientific uncertainty; second, the concomitant use of both frequentist and Bayesian methodologies [7] would maximize the opportunity to gather the most valuable scientific information on the efficacy of DAA; and third, the findings from this new analysis would provide the necessary tools to optimize the design of the next randomized trial. Thus, it is our responsibility to not stop our scientific investigation here, especially considering that the 3,370 patients who gave their consent to participate in these clinical trials were assured that their information would be fully utilized for fostering progress in medical science and for the betterment of future patients afflicted by severe sepsis.

Conclusion

PROWESS and PROWES-SHOCK trials are not comparable based on both clinical and statistical heterogeneity. Hence, the true effect of DAA in patients with severe sepsis remains to be defined. Unless the totality of the available evidence is thoroughly evaluated through an individual-patient data meta-analysis, and an adaptive Bayesian clinical trial is performed, we will continue treating our patients with the appalling sensation that we are not improving their survival due to our own inability to advance the quality of clinical research in the sepsis field.

Abbreviations

DAA:

drotrecogin alfa activated.

References

  1. 1.

    Bernard GR, Vincent JL, Laterre PF, LaRosa SP, Dhainaut JF, Lopez-Rodriguez A, Steingrub JS, Garber GE, Helterbrand JD, Ely EW: Efficacy and safety of recombinant human activated protein C for severe sepsis. N Engl J Med 2001, 344: 699-709. 10.1056/NEJM200103083441001

    PubMed  CAS  Article  Google Scholar 

  2. 2.

    Ranieri VM, Thompson BT, Barie PS, Dhainaut JF, Douglas IS, Finfer S, Gardlund B, Marshall JC, Rhodes A, Artigas A, Payen D, Tenhunen J, Al-Khalidi HR, Thompson V, Janes J, Macias WL, Vangerow B, Williams MD, PROWESS-SHOCK Study Group: Drotrecogin alfa (activated) in adults with septic shock. N Engl J Med 2012, 366: 2055-2064. 10.1056/NEJMoa1202290

    PubMed  CAS  Article  Google Scholar 

  3. 3.

    Levy MM, Artigas A, Phillips GS, Rhodes A, Beale R, Osborn T, Vincent JL, Townsend S, Lemeshow S, Dellinger RP: Outcomes of the Surviving Sepsis Campaign in intensive care units in the USA and Europe: a prospective cohort study. Lancet Infect Dis 2012, 12: 919-924. 10.1016/S1473-3099(12)70239-6

    PubMed  Article  Google Scholar 

  4. 4.

    Opal SM, Laterre PF, Francois B, LaRosa SP, Angus DC, Mira JP, Wittebole X, Dugernier T, Perrotin D, Tidswell M, Jauregui L, Krell K, Pachl J, Takahashi T, Peckelsen C, Cordasco E, Chang CS, Oeyen S, Aikawa N, Maruyama T, Schein R, Kalil AC, Van Nuffelen M, Lynn M, Rossignol DP, Gogate J, Roberts MB, Wheeler JL, Vincent JL, ACCESS Study Group: Effect of eritoran, an antagonist of MD2-TLR4, on mortality in patients with severe sepsis: the ACCESS randomized trial. JAMA 2013, 309: 1154-1162. 10.1001/jama.2013.2194

    PubMed  CAS  Article  Google Scholar 

  5. 5.

    Kalil AC, Sun J: How many patients with severe sepsis are needed to confirm the efficacy of drotrecogin alfa activated? A Bayesian design. Intensive Care Med 2008, 34: 1804-1811. 10.1007/s00134-008-1159-8

    PubMed  Article  Google Scholar 

  6. 6.

    Kalil AC, LaRosa SP: Effectiveness and safety of drotrecogin alfa (activated) for severe sepsis: a meta-analysis and metaregression. Lancet Infect Dis 2012, 12: 678-686. 10.1016/S1473-3099(12)70157-3

    PubMed  CAS  Article  Google Scholar 

  7. 7.

    Kalil AC, Sun J: Why are clinicians not embracing the results from pivotal clinical trials in severe sepsis? A bayesian analysis. PLoS ONE 2008, 3: e2291. 10.1371/journal.pone.0002291

    PubMed  PubMed Central  Article  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Andre C Kalil.

Additional information

Competing interests

ACK has received research grant support from Eisai, AstraZeneca, Agennix, and Asahi Kasei. DFF has no competing interests.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Kalil, A.C., Florescu, D.F. Severe sepsis: are PROWESS and PROWESS-SHOCK trials comparable? A clinical and statistical heterogeneity analysis. Crit Care 17, 167 (2013). https://doi.org/10.1186/cc12752

Download citation

Keywords

  • Severe Sepsis
  • Statistical Heterogeneity
  • Clinical Heterogeneity
  • Drotrecogin Alfa
  • Mortality Difference