Validation of survival prediction models for ECMO in Sars-CoV-2-related acute respiratory distress syndrome
Critical Care volume 26, Article number: 187 (2022)
To the editor
Predictive survival scores have been proposed for patient candidates for ECMO in the context of non-COVID-19 ARDS [1,2,3]. Indeed, better survival prediction in these patients may improve resource utilization, allow risk-adjusted comparison of centre-specific outcomes, and help clinicians target patients most likely to benefit from ECMO. It could be of utmost importance in the context of a pandemic with a shortage of resources and ICU beds and evolving mortality of the most severe forms with unclear long-term outcomes. However, the performance of these scores in patients with COVID-19 is currently unknown. Based on an ancillary analysis of patients proposed for ECMO consideration at the ECMO–COVID-19 hub in Paris between March 8, 2020, and June 3, 2020 , we aimed to validate and compare the performance of the main predictive survival models in that population of COVID-19 patients considered for ECMO.
This study followed TRIPOD recommendations for Prediction Model Development. Ninety-day survival status was prospectively collected for all patients of whom an ECMO was discussed. Briefly, contraindications for ECMO were age > 70 years (case-by-case discussion for those aged between 65 and 70 years), serious comorbidities (including immunosuppression and chronic lung diseases), multiple organ failure, and ongoing mechanical ventilation for > 10 days. Detailed indications and contraindications for ECMO during this period have been listed elsewhere . We computed the Respiratory ECMO Survival Prediction score (RESP) , PRedicting dEath for SEvere ARDS on VV-ECMO (PRESERVE) , Roch , and Sequential Organ Failure Assessment (SOFA) scores (ranging from 3 to 12 pre-ECMO items) in each patient at the time of ECMO consideration. Because the peak pressure was not systematically collected, the plateau pressure was used instead in the RESP score. The discriminative abilities of each score to predict 90-day survival was assessed by the area under the receiver-operating characteristics curves (AUC) and compared to each other using the De Long test. To test whether the observed 90-day survival matched expected mortality in our population, we used the Hosmer–Lemeshow test. Similarly, calibration was tested by the Brier score. The lower the Brier score the more calibrated the prediction.
Among the 575 cases submitted to the ECMO-COVID-19 hub, 302 (56%) patients met eligibility criteria and received ECMO (Fig. 1). The remaining patients were denied ECMO either because of contraindications or because the criteria for ECMO were not met yet. Patients’ characteristics and items of the RESP, PRESERVE, and Roch Scores according to ECMO decision are reported in Table 1. Overall 90-day mortality was 62.6%, whereas it was 54.3% and 73.3% in patients who received ECMO and those denied ECMO (i.e.ECMO contraindications or ECMO criteria not met), respectively.
External validation of the RESP-score in this COVID-19 population demonstrated reasonable discrimination (c = 0.74 [95% CI 0.70–0.78]) and good calibration with a Hosmer–Lemeshow C-statistic of 1.56 (p = 0.99) in contrast to poorer discrimination of the PRESERVE (c = 0.64 [95% CI 0.60–0.70]; p < 0.001), Roch (c = 0.64 [95% CI 0.60–0.69]; p < 0.001), and SOFA scores (c = 0.65 [95% CI 0.60–0.69]; p = 0.003). Lastly, ninety-day survival was much lower in risk class III and IV (i.e.RESP score ≤ − 2) than in risk class I, II (i.e.RESP score ≥ − 1) (p < 0.001) (Fig. 2).
To our knowledge, this is the first validation of predictive survival models in a population of patients with severe COVID-19-related ARDS proposed for VV-ECMO. The RESP score exhibited acceptable discrimination and a good calibration which was consistently better than the PRESERVE, Roch, and SOFA scores. Predicting outcomes of COVID patients on ECMO is challenging, as evolving mortality has been reported over the pandemic with changes in the management of the pre-ECMO period and new variants. We confirm the poor discriminant accuracy of the SOFA score to predict mortality of patients with COVID-19, even when combined with age as in the Roch score. The RESP score may offer an additional tool to help clinicians select appropriate COVID-19 candidates for ECMO and improve resource utilization, but it should not be used as a substitute for clinicians’ judgment.
Our study has limitations, the PRESERVE score was initially built to predict 6-month survival  and the Roch score was created for patients with influenza-related ARDS . We considered all these cases as “viral pneumonia” in the calculation of the scores although it is likely that some bacterial pulmonary superinfection could have precipitated the need for ECMO. We do not think that it would have changed our results as the distinction between bacterial and viral pneumonia is proposed only in the RESP score and both pneumonia etiologies are finally weighted similarly in that score . Further studies are now warranted to reassess the performance of the RESP score as the pandemic evolves and the expected mortality of patients treated with ECMO is higher. Further adaptation of the RESP score to this specific population could be needed.
Acute respiratory distress syndrome
Receiver-operating characteristics curves
Coronavirus disease 2019
Intensive care unit
Respiratory ECMO Survival Prediction score
PRedicting dEath for SEvere ARDS on VV-ECMO
Sequential Organ Failure Assessment
Venovenous-extracorporeal membrane oxygenation
Lebreton G, Schmidt M, Ponnaiah M, Folliguet T, Para M, Guihaire J, et al. Extracorporeal membrane oxygenation network organisation and clinical outcomes during the COVID-19 pandemic in Greater Paris, France: a multicentre cohort study. Lancet Respir Med. 2021;9:851–62.
Schmidt M, Bailey M, Sheldrake J, Hodgson C, Aubron C, Rycus PT, et al. Predicting survival after extracorporeal membrane oxygenation for severe acute respiratory failure. The Respiratory Extracorporeal Membrane Oxygenation Survival Prediction (RESP) Score. Am J Respir Crit Care Med. 2014;189:1374–82.
Schmidt M, Zogheib E, Rozé H, Repesse X, Lebreton G, Luyt C-E, et al. The PRESERVE mortality risk score and analysis of long-term outcomes after extracorporeal membrane oxygenation for severe acute respiratory distress syndrome. Intensive Care Med. 2013;39:1704–13.
Roch A, Hraiech S, Masson E, Grisoli D, Forel J-M, Boucekine M, et al. Outcome of acute respiratory distress syndrome patients treated with extracorporeal membrane oxygenation and brought to a referral center. Intensive Care Med. 2014;40:74–83.
Ethics approval and consent to participate
According to French research methodology MR004, this work was considered a non-interventional retrospective study and therefore only needed information from the survivors. The study was approved by the local ethical committee, Comité d’Ethique de la Recherche of Sorbonne University (#CER-SU-2020-69).
Consent for publication
Availability of data and materials
The datasets used during the current study are available from the corresponding author on reasonable request.
Pr Combes reports grants from Getinge, and personal fees from Getinge, Baxter, and Xenios outside the submitted work. Pr Schmidt reports receiving personal fees from Getinge, Drager, and Xenios, outside the submitted work. No other disclosures were reported.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Moyon, Q., Pineton de Chambrun, M., Lebreton, G. et al. Validation of survival prediction models for ECMO in Sars-CoV-2-related acute respiratory distress syndrome. Crit Care 26, 187 (2022). https://doi.org/10.1186/s13054-022-04039-4