Skip to main content

Validation of survival prediction models for ECMO in Sars-CoV-2-related acute respiratory distress syndrome


To the editor

Predictive survival scores have been proposed for patient candidates for ECMO in the context of non-COVID-19 ARDS [1,2,3]. Indeed, better survival prediction in these patients may improve resource utilization, allow risk-adjusted comparison of centre-specific outcomes, and help clinicians target patients most likely to benefit from ECMO. It could be of utmost importance in the context of a pandemic with a shortage of resources and ICU beds and evolving mortality of the most severe forms with unclear long-term outcomes. However, the performance of these scores in patients with COVID-19 is currently unknown. Based on an ancillary analysis of patients proposed for ECMO consideration at the ECMO–COVID-19 hub in Paris between March 8, 2020, and June 3, 2020 [1], we aimed to validate and compare the performance of the main predictive survival models in that population of COVID-19 patients considered for ECMO.

This study followed TRIPOD recommendations for Prediction Model Development. Ninety-day survival status was prospectively collected for all patients of whom an ECMO was discussed. Briefly, contraindications for ECMO were age > 70 years (case-by-case discussion for those aged between 65 and 70 years), serious comorbidities (including immunosuppression and chronic lung diseases), multiple organ failure, and ongoing mechanical ventilation for > 10 days. Detailed indications and contraindications for ECMO during this period have been listed elsewhere [1]. We computed the Respiratory ECMO Survival Prediction score (RESP) [2], PRedicting dEath for SEvere ARDS on VV-ECMO (PRESERVE) [3], Roch [4], and Sequential Organ Failure Assessment (SOFA) scores (ranging from 3 to 12 pre-ECMO items) in each patient at the time of ECMO consideration. Because the peak pressure was not systematically collected, the plateau pressure was used instead in the RESP score. The discriminative abilities of each score to predict 90-day survival was assessed by the area under the receiver-operating characteristics curves (AUC) and compared to each other using the De Long test. To test whether the observed 90-day survival matched expected mortality in our population, we used the Hosmer–Lemeshow test. Similarly, calibration was tested by the Brier score. The lower the Brier score the more calibrated the prediction.

Among the 575 cases submitted to the ECMO-COVID-19 hub, 302 (56%) patients met eligibility criteria and received ECMO (Fig. 1). The remaining patients were denied ECMO either because of contraindications or because the criteria for ECMO were not met yet. Patients’ characteristics and items of the RESP, PRESERVE, and Roch Scores according to ECMO decision are reported in Table 1. Overall 90-day mortality was 62.6%, whereas it was 54.3% and 73.3% in patients who received ECMO and those denied ECMO (i.e.ECMO contraindications or ECMO criteria not met), respectively.

Fig. 1
figure 1

Study flow chart. ECMO extracorporeal membrane oxygenation, MV mechanical ventilation, PEEP positive end-expiratory pressure

Table 1 Patients’ characteristics and items of the RESP, PRESERVE, and Roch Scores according to ECMO decision

External validation of the RESP-score in this COVID-19 population demonstrated reasonable discrimination (c = 0.74 [95% CI 0.70–0.78]) and good calibration with a Hosmer–Lemeshow C-statistic of 1.56 (p = 0.99) in contrast to poorer discrimination of the PRESERVE (c = 0.64 [95% CI 0.60–0.70]; p < 0.001), Roch (c = 0.64 [95% CI 0.60–0.69]; p < 0.001), and SOFA scores (c = 0.65 [95% CI 0.60–0.69]; p = 0.003). Lastly, ninety-day survival was much lower in risk class III and IV (i.e.RESP score ≤  − 2) than in risk class I, II (i.e.RESP score ≥  − 1) (p < 0.001) (Fig. 2).

Fig. 2
figure 2

Comparison of A) the receiver-operating curves, B) Hosmer–Lemeshow C-statistic, and Brier score of the RESP, PRESERVE, Roch, et SOFA scores and C) Kaplan–Meier estimates of cumulative probabilities of survival according to the RESP class in a COVID-19 population candidate for ECMO (n = 535). ECMO extracorporeal membrane oxygenation, RESP Respiratory ECMO Survival Prediction score, PRESERVE PRedicting dEath for SEvere ARDS on VV-ECMO, SOFA Sequential Organ Failure Assessment

To our knowledge, this is the first validation of predictive survival models in a population of patients with severe COVID-19-related ARDS proposed for VV-ECMO. The RESP score exhibited acceptable discrimination and a good calibration which was consistently better than the PRESERVE, Roch, and SOFA scores. Predicting outcomes of COVID patients on ECMO is challenging, as evolving mortality has been reported over the pandemic with changes in the management of the pre-ECMO period and new variants. We confirm the poor discriminant accuracy of the SOFA score to predict mortality of patients with COVID-19, even when combined with age as in the Roch score. The RESP score may offer an additional tool to help clinicians select appropriate COVID-19 candidates for ECMO and improve resource utilization, but it should not be used as a substitute for clinicians’ judgment.

Our study has limitations, the PRESERVE score was initially built to predict 6-month survival [3] and the Roch score was created for patients with influenza-related ARDS [4]. We considered all these cases as “viral pneumonia” in the calculation of the scores although it is likely that some bacterial pulmonary superinfection could have precipitated the need for ECMO. We do not think that it would have changed our results as the distinction between bacterial and viral pneumonia is proposed only in the RESP score and both pneumonia etiologies are finally weighted similarly in that score [2]. Further studies are now warranted to reassess the performance of the RESP score as the pandemic evolves and the expected mortality of patients treated with ECMO is higher. Further adaptation of the RESP score to this specific population could be needed.

Abbreviations

ARDS:

Acute respiratory distress syndrome

AUC:

Receiver-operating characteristics curves

COVID-19:

Coronavirus disease 2019

ICU:

Intensive care unit

RESP:

Respiratory ECMO Survival Prediction score

PRESERVE:

PRedicting dEath for SEvere ARDS on VV-ECMO

SOFA:

Sequential Organ Failure Assessment

VV-ECMO:

Venovenous-extracorporeal membrane oxygenation

References

  1. Lebreton G, Schmidt M, Ponnaiah M, Folliguet T, Para M, Guihaire J, et al. Extracorporeal membrane oxygenation network organisation and clinical outcomes during the COVID-19 pandemic in Greater Paris, France: a multicentre cohort study. Lancet Respir Med. 2021;9:851–62.

    CAS  Article  Google Scholar 

  2. Schmidt M, Bailey M, Sheldrake J, Hodgson C, Aubron C, Rycus PT, et al. Predicting survival after extracorporeal membrane oxygenation for severe acute respiratory failure. The Respiratory Extracorporeal Membrane Oxygenation Survival Prediction (RESP) Score. Am J Respir Crit Care Med. 2014;189:1374–82.

    Article  Google Scholar 

  3. Schmidt M, Zogheib E, Rozé H, Repesse X, Lebreton G, Luyt C-E, et al. The PRESERVE mortality risk score and analysis of long-term outcomes after extracorporeal membrane oxygenation for severe acute respiratory distress syndrome. Intensive Care Med. 2013;39:1704–13.

    CAS  Article  Google Scholar 

  4. Roch A, Hraiech S, Masson E, Grisoli D, Forel J-M, Boucekine M, et al. Outcome of acute respiratory distress syndrome patients treated with extracorporeal membrane oxygenation and brought to a referral center. Intensive Care Med. 2014;40:74–83.

    CAS  Article  Google Scholar 

Download references

Acknowledgements

None.

Funding

None.

Author information

Authors and Affiliations

Authors

Contributions

QM conceived of the study, participated in its design, data collection, coordination, performed the statistics, and draft the manuscript; MPC participated in its design, data collection, and revised the manuscript; GL participated in its design, data collection, and revised the manuscript; HC participated in data collection and revised the manuscript; AC participated in its design, data collection and revised the manuscript; MS conceived of the study, participated in its design, data collection, coordination, performed the statistics and draft the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Matthieu Schmidt.

Ethics declarations

Ethics approval and consent to participate

According to French research methodology MR004, this work was considered a non-interventional retrospective study and therefore only needed information from the survivors. The study was approved by the local ethical committee, Comité d’Ethique de la Recherche of Sorbonne University (#CER-SU-2020-69).

Consent for publication

Not applicable.

Availability of data and materials

The datasets used during the current study are available from the corresponding author on reasonable request.

Competing interests

Pr Combes reports grants from Getinge, and personal fees from Getinge, Baxter, and Xenios outside the submitted work. Pr Schmidt reports receiving personal fees from Getinge, Drager, and Xenios, outside the submitted work. No other disclosures were reported.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Moyon, Q., Pineton de Chambrun, M., Lebreton, G. et al. Validation of survival prediction models for ECMO in Sars-CoV-2-related acute respiratory distress syndrome. Crit Care 26, 187 (2022). https://doi.org/10.1186/s13054-022-04039-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13054-022-04039-4