Skip to main content

Demystifying machine learning for mortality prediction

The Original Article was published on 08 September 2021

With interest, we read the article by Banoei et al. [1] on machine learning (ML) models to predict mortality among COVID-19 patients. They refer to other studies that failed to predict mortality using ‘conventional statistical analysis’, after which they present a linear ML model as a better suited method for such complex medical problems. We feel such a claim creates an image around ML as an alternative technique that offers solutions where statistical modeling fails. However, ML and statistical modeling are tightly interwoven. Indeed, there is no consensus on whether or how to differentiate between the two [2, 3], e.g., the approach Banoei and colleagues present (Partial Least Squares) could easily be considered a ‘statistical analysis’ as well. A recent systematic review [4] differentiates ML from statistical models based on how ‘automatically’ these models learn and found no difference in discriminative performance. This raises the question why efforts are made to differentiate between statistical modeling and ML, as it does not provide insights into which prediction models work for which kind of problems. We advocate it is more important to demystify ML by emphasizing its connections to statistical models most clinicians are already familiar with, as it may help in setting reasonable expectations for the potential clinical benefit ML could bring. Towards demystifying ML, good reporting of methodology and findings is essential. Due to unclear or incomplete reporting, one may draw wrong conclusions and miss out on opportunities to learn. For instance, Banoei and colleagues are unclear on how frequently measured predictors (like SpO2) were aggregated to one value. We encourage to follow a unified approach for reporting, e.g. the TRIPOD guidelines [5]. Moreover, it is unfortunately still common that the intended use of prediction models is unclear, whereas it has implications for choices in the model development. Likewise, Banoei and colleagues present several models (for mortality prediction and patient clustering) and suggest that these can ‘aid in clinical decision making and resource allocation’, without further specification. We strongly encourage to develop models with an explicit intended use in mind, enabling fair judgement of their clinical relevance. Altogether, we join Banoei and colleagues in their belief that ML models hold a lot of promise as valuable tools in the modern ICU. However, we advise against differentiating them from statistical models and advocate proper reporting about the methodology and intended use.

Authors’ response

Machine learning-based methods vs. conventional statistical methods for studying mortality outcome

In response to the Letter to the Editor by J.M. Smit et al., we certainly agree that machine learning (ML)-based methods can be used interchangeably with multivariate data analysis (MVA) and multivariate prediction models (MPMs) [6]. Although ML approaches like partial least square (PLS), statistically inspired modification of PLS (SIMPLS), random forest (RF), Support Vector Machine (SVM), and artificial neural network (ANN) are considered statistical methods, they are notably different from conventional statistical methods (CSM) [7]. Considering the advantages of ML methods, ML approaches have contributed significantly to the early detection, tracing, diagnosis, prognosis and clinical trials of COVID-19 that have been more functional to support researchers in confronting the coronavirus pandemic [8]. Several studies previously have shown that ML can be more appropriate than CSM for the clinical datasets. ML algorithms have proven better able to stratify COVID-19 patients and mortality risk [9], identify high-risk patients with COVID-19 [10].

It is generally believed that there is no single ML method superior to others. The SIMPLS method was the only prediction model used in our current study. SIMPLS has remarkable advantages including a lower risk of overfitting, a high level of interpretation, a high level of variable selection, easy implementation when compared with RF, SVM and ANN methods [6]. SIMPLS can easily account for batch processing and a high degree of correlation (multicollinearity) between and among variables of large datasets. SIMPLS fit the outcome responses with nominal, continuous, and polynomial data type, interaction and categorial effects, and provides strong visualization when compared with other ML and CSM. In our study, SIMPLS was successfully applied to recognize the most differentiating variables involved in the prediction of COVID-19 mortality. Other statistical methods that are not considered prediction models, such as principal component analysis (PCA) and latent class analysis (LCA) were established based on the findings of SIMPLS to identify patients at the highest risk of dying.

In our study, the importance of ML-based model was its ability to predict patient mortality using variables measured at the time of admission, although these variables were frequently measured during the patients’ hospitalization for other purposes.

Our study mainly focused on the importance of ML in clinical application, and the recognition of the most important variables contributing to mortality from COVID-19, instead of describing the details of ML that may be extraneous for clinicians.

Abbreviations

ML:

Machine learning

ICU:

Intensive care unit

COVID-19:

Coronavirus disease 2019

TRIPOD:

Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis

References

  1. 1.

    Banoei MM, Dinparastisaleh R, Zadeh AV, Mirsaeidi M. Machine-learning-based COVID-19 mortality prediction model and identification of patients at low and high risk of dying. Crit Care (Lond Engl). 2021;25(1):328. https://doi.org/10.1186/s13054-021-03749-5.

    Article  Google Scholar 

  2. 2.

    Breiman L. Statistical modeling: the two cultures. Qual Eng. 2001;48:81–2.

    Google Scholar 

  3. 3.

    Beam AL, Kohane IS. Big data and machine learning in health care. JAMA. 2018;319(13):1317–8. https://doi.org/10.1001/jama.2017.18391.

    Article  PubMed  Google Scholar 

  4. 4.

    Christodoulou E, Ma J, Collins GS, Steyerberg EW, Verbakel JY, Calster BV. Review: a systematic review shows no performance benefit of machine learning over logistic regression for clinical prediction models. J Clin Epidemiol. 2019;110:12–22. https://doi.org/10.1016/j.jclinepi.2019.02.004.

    Article  PubMed  Google Scholar 

  5. 5.

    Collins GS, Reitsma JB, Altman DG, Moons KGM. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD Statement. BMC Med. 2015;13(1):1. https://doi.org/10.1186/s12916-014-0241-z.

    Article  PubMed  PubMed Central  Google Scholar 

  6. 6.

    Liebal UW, et al. Machine learning applications for mass spectrometry-based metabolomics. Metabolites. 2020;10(6):243.

    CAS  Article  Google Scholar 

  7. 7.

    Rajula HSR, et al. Comparison of conventional statistical methods with machine learning in medicine: diagnosis, drug development, and treatment. Medicina (Kaunas). 2020;56(9):455.

    Article  Google Scholar 

  8. 8.

    Rahman MM, et al. A comprehensive study of artificial intelligence and machine learning approaches in confronting the coronavirus (COVID-19) pandemic. Int J Health Serv. 2021;51(4):446–61.

    Article  Google Scholar 

  9. 9.

    Halasz G, et al. A machine learning approach for mortality prediction in COVID-19 pneumonia: development and evaluation of the Piacenza score. J Med Internet Res. 2021;23(5):e29058.

    Article  Google Scholar 

  10. 10.

    Quiroz-Juárez MA, et al. Identification of high-risk COVID-19 patients using machine learning. PLoS ONE. 2021;16(9):e0257234.

    Article  Google Scholar 

Download references

Author information

Affiliations

Authors

Contributions

JS drafted the manuscript. MvG, JK, MR and JvB critically reviewed the manuscript. All authors read and approved the final manuscript.

Corresponding authors

Correspondence to J. M. Smit or Mehdi Mirsaeidi.

Ethics declarations

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Smit, J.M., van Genderen, M.E., Reinders, M.J.T. et al. Demystifying machine learning for mortality prediction. Crit Care 25, 447 (2021). https://doi.org/10.1186/s13054-021-03868-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13054-021-03868-z