- Correspondence
- Open Access
- Published:
Validation of survival prediction models for ECMO in Sars-CoV-2-related acute respiratory distress syndrome
Critical Care volume 26, Article number: 187 (2022)
To the editor
Predictive survival scores have been proposed for patient candidates for ECMO in the context of non-COVID-19 ARDS [1,2,3]. Indeed, better survival prediction in these patients may improve resource utilization, allow risk-adjusted comparison of centre-specific outcomes, and help clinicians target patients most likely to benefit from ECMO. It could be of utmost importance in the context of a pandemic with a shortage of resources and ICU beds and evolving mortality of the most severe forms with unclear long-term outcomes. However, the performance of these scores in patients with COVID-19 is currently unknown. Based on an ancillary analysis of patients proposed for ECMO consideration at the ECMO–COVID-19 hub in Paris between March 8, 2020, and June 3, 2020 [1], we aimed to validate and compare the performance of the main predictive survival models in that population of COVID-19 patients considered for ECMO.
This study followed TRIPOD recommendations for Prediction Model Development. Ninety-day survival status was prospectively collected for all patients of whom an ECMO was discussed. Briefly, contraindications for ECMO were age > 70 years (case-by-case discussion for those aged between 65 and 70 years), serious comorbidities (including immunosuppression and chronic lung diseases), multiple organ failure, and ongoing mechanical ventilation for > 10 days. Detailed indications and contraindications for ECMO during this period have been listed elsewhere [1]. We computed the Respiratory ECMO Survival Prediction score (RESP) [2], PRedicting dEath for SEvere ARDS on VV-ECMO (PRESERVE) [3], Roch [4], and Sequential Organ Failure Assessment (SOFA) scores (ranging from 3 to 12 pre-ECMO items) in each patient at the time of ECMO consideration. Because the peak pressure was not systematically collected, the plateau pressure was used instead in the RESP score. The discriminative abilities of each score to predict 90-day survival was assessed by the area under the receiver-operating characteristics curves (AUC) and compared to each other using the De Long test. To test whether the observed 90-day survival matched expected mortality in our population, we used the Hosmer–Lemeshow test. Similarly, calibration was tested by the Brier score. The lower the Brier score the more calibrated the prediction.
Among the 575 cases submitted to the ECMO-COVID-19 hub, 302 (56%) patients met eligibility criteria and received ECMO (Fig. 1). The remaining patients were denied ECMO either because of contraindications or because the criteria for ECMO were not met yet. Patients’ characteristics and items of the RESP, PRESERVE, and Roch Scores according to ECMO decision are reported in Table 1. Overall 90-day mortality was 62.6%, whereas it was 54.3% and 73.3% in patients who received ECMO and those denied ECMO (i.e.ECMO contraindications or ECMO criteria not met), respectively.
External validation of the RESP-score in this COVID-19 population demonstrated reasonable discrimination (c = 0.74 [95% CI 0.70–0.78]) and good calibration with a Hosmer–Lemeshow C-statistic of 1.56 (p = 0.99) in contrast to poorer discrimination of the PRESERVE (c = 0.64 [95% CI 0.60–0.70]; p < 0.001), Roch (c = 0.64 [95% CI 0.60–0.69]; p < 0.001), and SOFA scores (c = 0.65 [95% CI 0.60–0.69]; p = 0.003). Lastly, ninety-day survival was much lower in risk class III and IV (i.e.RESP score ≤ − 2) than in risk class I, II (i.e.RESP score ≥ − 1) (p < 0.001) (Fig. 2).
Comparison of A) the receiver-operating curves, B) Hosmer–Lemeshow C-statistic, and Brier score of the RESP, PRESERVE, Roch, et SOFA scores and C) Kaplan–Meier estimates of cumulative probabilities of survival according to the RESP class in a COVID-19 population candidate for ECMO (n = 535). ECMO extracorporeal membrane oxygenation, RESP Respiratory ECMO Survival Prediction score, PRESERVE PRedicting dEath for SEvere ARDS on VV-ECMO, SOFA Sequential Organ Failure Assessment
To our knowledge, this is the first validation of predictive survival models in a population of patients with severe COVID-19-related ARDS proposed for VV-ECMO. The RESP score exhibited acceptable discrimination and a good calibration which was consistently better than the PRESERVE, Roch, and SOFA scores. Predicting outcomes of COVID patients on ECMO is challenging, as evolving mortality has been reported over the pandemic with changes in the management of the pre-ECMO period and new variants. We confirm the poor discriminant accuracy of the SOFA score to predict mortality of patients with COVID-19, even when combined with age as in the Roch score. The RESP score may offer an additional tool to help clinicians select appropriate COVID-19 candidates for ECMO and improve resource utilization, but it should not be used as a substitute for clinicians’ judgment.
Our study has limitations, the PRESERVE score was initially built to predict 6-month survival [3] and the Roch score was created for patients with influenza-related ARDS [4]. We considered all these cases as “viral pneumonia” in the calculation of the scores although it is likely that some bacterial pulmonary superinfection could have precipitated the need for ECMO. We do not think that it would have changed our results as the distinction between bacterial and viral pneumonia is proposed only in the RESP score and both pneumonia etiologies are finally weighted similarly in that score [2]. Further studies are now warranted to reassess the performance of the RESP score as the pandemic evolves and the expected mortality of patients treated with ECMO is higher. Further adaptation of the RESP score to this specific population could be needed.
Abbreviations
- ARDS:
-
Acute respiratory distress syndrome
- AUC:
-
Receiver-operating characteristics curves
- COVID-19:
-
Coronavirus disease 2019
- ICU:
-
Intensive care unit
- RESP:
-
Respiratory ECMO Survival Prediction score
- PRESERVE:
-
PRedicting dEath for SEvere ARDS on VV-ECMO
- SOFA:
-
Sequential Organ Failure Assessment
- VV-ECMO:
-
Venovenous-extracorporeal membrane oxygenation
References
Lebreton G, Schmidt M, Ponnaiah M, Folliguet T, Para M, Guihaire J, et al. Extracorporeal membrane oxygenation network organisation and clinical outcomes during the COVID-19 pandemic in Greater Paris, France: a multicentre cohort study. Lancet Respir Med. 2021;9:851–62.
Schmidt M, Bailey M, Sheldrake J, Hodgson C, Aubron C, Rycus PT, et al. Predicting survival after extracorporeal membrane oxygenation for severe acute respiratory failure. The Respiratory Extracorporeal Membrane Oxygenation Survival Prediction (RESP) Score. Am J Respir Crit Care Med. 2014;189:1374–82.
Schmidt M, Zogheib E, Rozé H, Repesse X, Lebreton G, Luyt C-E, et al. The PRESERVE mortality risk score and analysis of long-term outcomes after extracorporeal membrane oxygenation for severe acute respiratory distress syndrome. Intensive Care Med. 2013;39:1704–13.
Roch A, Hraiech S, Masson E, Grisoli D, Forel J-M, Boucekine M, et al. Outcome of acute respiratory distress syndrome patients treated with extracorporeal membrane oxygenation and brought to a referral center. Intensive Care Med. 2014;40:74–83.
Acknowledgements
None.
Funding
None.
Author information
Authors and Affiliations
Contributions
QM conceived of the study, participated in its design, data collection, coordination, performed the statistics, and draft the manuscript; MPC participated in its design, data collection, and revised the manuscript; GL participated in its design, data collection, and revised the manuscript; HC participated in data collection and revised the manuscript; AC participated in its design, data collection and revised the manuscript; MS conceived of the study, participated in its design, data collection, coordination, performed the statistics and draft the manuscript. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Ethics approval and consent to participate
According to French research methodology MR004, this work was considered a non-interventional retrospective study and therefore only needed information from the survivors. The study was approved by the local ethical committee, Comité d’Ethique de la Recherche of Sorbonne University (#CER-SU-2020-69).
Consent for publication
Not applicable.
Availability of data and materials
The datasets used during the current study are available from the corresponding author on reasonable request.
Competing interests
Pr Combes reports grants from Getinge, and personal fees from Getinge, Baxter, and Xenios outside the submitted work. Pr Schmidt reports receiving personal fees from Getinge, Drager, and Xenios, outside the submitted work. No other disclosures were reported.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
About this article
Cite this article
Moyon, Q., Pineton de Chambrun, M., Lebreton, G. et al. Validation of survival prediction models for ECMO in Sars-CoV-2-related acute respiratory distress syndrome. Crit Care 26, 187 (2022). https://doi.org/10.1186/s13054-022-04039-4
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s13054-022-04039-4