Skip to main content

The simple observational critical care studies: estimations by students, nurses, and physicians of in-hospital and 6-month mortality

Abstract

Background

Prognostic assessments of the mortality of critically ill patients are frequently performed in daily clinical practice and provide prognostic guidance in treatment decisions. In contrast to several sophisticated tools, prognostic estimations made by healthcare providers are always available and accessible, are performed daily, and might have an additive value to guide clinical decision-making. The aim of this study was to evaluate the accuracy of students’, nurses’, and physicians’ estimations and the association of their combined estimations with in-hospital mortality and 6-month follow-up.

Methods

The Simple Observational Critical Care Studies is a prospective observational single-center study in a tertiary teaching hospital in the Netherlands. All patients acutely admitted to the intensive care unit were included. Within 3 h of admission to the intensive care unit, a medical or nursing student, a nurse, and a physician independently predicted in-hospital and 6-month mortality. Logistic regression was used to assess the associations between predictions and the actual outcome; the area under the receiver operating characteristics (AUROC) was calculated to estimate the discriminative accuracy of the students, nurses, and physicians.

Results

In 827 out of 1,010 patients, in-hospital mortality rates were predicted to be 11%, 15%, and 17% by medical students, nurses, and physicians, respectively. The estimations of students, nurses, and physicians were all associated with in-hospital mortality (OR 5.8, 95% CI [3.7, 9.2], OR 4.7, 95% CI [3.0, 7.3], and OR 7.7 95% CI [4.7, 12.8], respectively). Discriminative accuracy was moderate for all students, nurses, and physicians (between 0.58 and 0.68). When more estimations were of non-survival, the odds of non-survival increased (OR 2.4 95% CI [1.9, 3.1]) per additional estimate, AUROC 0.70 (0.65, 0.76). For 6-month mortality predictions, similar results were observed.

Conclusions

Based on the initial examination, students, nurses, and physicians can only moderately predict in-hospital and 6-month mortality in critically ill patients. Combined estimations led to more accurate predictions and may serve as an example of the benefit of multidisciplinary clinical care and future research efforts.

Background

Patients admitted to intensive care units (ICUs) may suffer from various illnesses and comorbidities. ICU patients are known to have high mortality rates, and survivors may suffer from long-term impairments in both cognitive and overall function, leading to a reduced quality of life [1]. Although all patients admitted to the ICUs are in an acute state of critical illness, they are often remarkably different in terms of characteristics and prognosis. Every day, healthcare providers judge clinical expectations, including possible health outcomes, and provide such information to patients and their relatives. Accurate estimates of survival are essential, as these may influence clinical decisions, especially in resource-limited situations. In an ICU setting, physicians and nurses typically apply a combination of deductive and inductive reasoning, along with intuitive (non-methodical reasoning) analysis. Discussing mortality estimations in an inter-disciplinary setting can therefore offer new insights and improvements in the reasoning process and improved prediction [2, 3].

During the training of physicians and nurses, various theories are used to teach them how to actively combine acquired clinical knowledge, acquired diagnostic experience, and personal intuition. This approach is needed to make as accurate prognostic statements as possible, including statements about the likelihood that the applied treatment will lead to survival [4].

One intriguing study evaluated the discriminative accuracy of physicians and nurses’ estimations of 303 critically ill patients' clinical outcomes and found that the estimations of physicians and nurses added significantly to the discriminative accuracy of existing prediction models [5]. However, the results of this study have not been validated in a sufficiently large population [6], and research in other settings has shown contrasting results [7]. Nevertheless, the estimation would be valuable in the early stages of critical diseases and should be made shortly after ICU admission. Furthermore, the accuracy of estimations can evolve and differ for different healthcare providers [8,9,10].

The ability of healthcare providers to predict which patients will survive is relevant for clinical decision-making. However, intuition alone may not always lead to correct diagnostic judgments, and both managing this intuition and clinical experience are prerequisites for reliable estimates of the outcome. The estimations of experienced physicians and nurses may be substantially more accurate than those of students. Whether this assumption is correct is unknown, however. This study aimed to explore the accuracy of students’, nurses’, and physicians’ estimations of mortality and to evaluate the association of combined estimations by students, nurses, and physicians with the outcome.

Methods

Design and setting

The Simple Observational Critical Care Studies (SOCCS, NCT03553069) is a single-center, prospective observational study designed to evaluate the diagnostic and prognostic value of clinical examination on admission and serves as the first time point for repeated clinical examination in critically ill patients as part of the Simple Intensive Care Studies II (SICS-II, NCT03577405) study [11, 12]. The local institutional review board approved the study (2018/203).

Participants

This study included all acutely admitted critically ill patients over 18 years of age, with an expected stay of at least 24 h. Patients were excluded if they had previously been included in this study (e.g., in cases of readmission), if strict isolation rules limited access, or if no informed consent was obtained.

Variables

Patient characteristics were registered, including age, gender and body mass index were registered at admission, and outcome was estimated according to our protocol [11]. All acutely admitted patients were included as soon as possible after acute admission during the day- and nighttime, with a maximum of 3 h after ICU admission. A selected team of about thirty medical and nursing students was available for shifts to support research in our ICU voluntarily. All medical and nursing students had completed at least one year of medical or nursing school and were trained to perform a structured one-time clinical examination. This type of examination was familiar to all students from their medical and nursing education, including skills.

When a patient was acutely admitted to the ICU, the SOCCS team was called by the study coordinator (daytime) or physician (nighttime). The student first performed a structured clinical examination, assessing basic vitals and signs of shock. Subsequently, the medical student, an ICU nurse, and a physician independently answered the following questions: (1) "Do you think the patient will survive this hospital admission?"; (2) "Do you think the patient will be alive in 6 months?" [13, 14]. The physicians and nurses received no additional training but were all informed about the study. Clinical risk scores were calculated after the termination of the study and were not available for the researcher at the time of estimation. The students were not involved in patient care and were instructed not to share their findings with the attending physicians and nurses. Instead, mortality data were gathered from the electronic health records.

Statistics

Continuous variables were reported as means (with standard deviations) or medians (with interquartile ranges (IQRs)) depending on distributions. Categorical data were presented in proportions. Associations were calculated as odds ratios with 95% confidence intervals (CI). Student's t tests, Mann–Whitney U tests, or the Chi-square tests were used as appropriate. In addition, the clinical characteristics of patients were explored.

The univariate association between single estimations and in-hospital mortality and the association of cumulative estimations and in-hospital mortality were explored using univariate logistic regression analysis. To test for marginal homogeneity of estimations between groups, a McNemar test was performed. Area under the receiving operator characteristic (AUROC) was calculated for each estimation to assess discriminative accuracy. Values of p <0.05 were considered statistically significant. Analyses were performed using Stata version 16 (StataCorp, College Station, TX, USA).

Results

Between May 14, 2018, and July 10, 2019, a total of 3357 ICU admissions were assessed for eligibility. In total, 1,104 patients fulfilled the inclusion criteria for clinical examination. Data were not obtained for 94 patients: 45 patients died before inclusion, continuous resuscitation efforts were made for 26 patients, and there was no access to 23 patients due to logistic reasons. In total, 1,010 patients were included in the SICS-II cohort. For 183 patients (18%), estimations were not available, resulting in 827 patients (82%) being included in the SOCCS cohort (Fig. 1; Table 1). Median time from ICU admission to clinical examination was 1.8 h (IQR 1.1–2.7). In total, 178 patients (22%) died during the hospital stay. By the 6-month follow-up, 238 patients (29%) had died. Ten patients were lost to follow-up.

Fig. 1
figure1

Flowchart of study inclusion

Table 1 Clinical characteristics of the SOCCS cohort

Students’, nurses’, and physicians’ estimations

Estimations concerning in-hospital mortality were obtained from medical students in all cases, from the nurses in 709 cases (86%), and from the physicians in 507 cases (61%). There were no significant differences between the characteristics and severity of the disease of patients who had and patients who did not have an estimation by the physician (Additional file 1: Table S1).

In-hospital mortality

Medical and nursing students predicted in-hospital mortality of 11%, nurses 15%, and physicians 17%. There was a fair agreement for all students, nurses, and physicians estimating in-hospital mortality (range 87.4–90.8%) (Table 2).

Table 2 Agreement between students, nurses, and physicians for estimating in-hospital mortality

Estimations of all students, nurses, and physicians were available in 481 cases, and all were correct in 71% of cases. There were no significant differences in the proportions of correct estimates between groups (all p values > 0.2). The estimations of students, nurses, and physicians were all associated with in-hospital mortality (OR 5.8, CI [3.6, 9.4], OR 4.7, [2.9, 7.5], and OR 7.7 [4.5, 13.2], respectively). Discriminative accuracy was moderate for students, nurses, and physicians, with AUROC ranging from 0.61 to 0.68. When more estimates were of non-survival, odds of non-survival increased (OR 2.4, CI [1.9, 3.1] per additional estimate, AUROC 0.70 CI [0.65, 0.76] (Table 3).

Table 3 Univariate associations between estimations and in-hospital mortality

Mortality at 6 months

Six-month mortality was predicted to be 18% by students, 20% by nurses, and 21% by physicians. The estimations of students, nurses, and physicians were all associated with 6-month mortality (OR 4.4, CI [3.1, 6.4], OR 6.3, CI [4.2, 9.4] and OR 7.1, CI [4.4, 11.2], respectively), and clinical characteristics were different when mortality was estimated (Additional file 2: Table S2a-c). When more estimates were of non-survival, odds of non-survival increased (OR 2.5, CI [2.0, 3.1]) per additional estimate). In total, estimations of all students, nurses, and physicians for 6-month mortality were available in 479 cases. There were 41 (9%) patients for whom all students, nurses, and physicians predicted non-survival at 6 months. These patients more often received vasoactive medication, had a lower temperature, and had higher APACHE IV and SAPS II scores (Additional file 3: Table S3). Discriminative accuracy was moderate for all students, nurses, and physicians, with AUROC ranging from 0.62 to 0.68, but was fair when estimations were combined (0.72, CI [0.67, 0.76]) (Table 4).

Table 4 Univariate associations between estimations and six-month mortality

Discussion

In this study, the estimations of medical and nursing students, nurses, and physicians of both in-hospital and 6-month mortality within 3 h after admission to the ICU were found to have moderate discriminative accuracy. In addition, the variation in accuracy between students, nurses, and physicians was not statistically different. The most valuable observation was that, in case where more students, nurses, and physicians estimated non-survival, both the risk of death and the discriminative accuracy were significantly higher, suggesting the benefits of multidisciplinary clinical care and prognostication.

Several studies have evaluated the accuracy of physicians and nurses’ estimations of clinical outcomes in critically ill patients [5, 15,16,17,18,19]. Overall, these studies show an additional value of physicians and nurses’ estimations compared or added to existing clinical prediction models. Below, we present the previous studies most relevant to the current study. Detsky et al. investigated the discriminative accuracy of both nurses' and physicians’ estimations and showed that, of multiple outcomes estimated, nurses could best estimate in-hospital mortality in critically ill patients [5]. In our study, physicians were most accurate in estimating in-hospital mortality, followed by students and nurses. No previous study has evaluated the accuracy of medical students’ estimations of patient mortality. Buehler et al. showed that the overall accuracy of predictions made by ICU physicians was moderate [20]. However, their study was a literature review comparing several observational studies that were limited to physicians.

We are unaware of any studies investigating the cumulative estimation of multiple healthcare providers considering mortality. However, one study showed that predictions considering ineffective treatment were more valuable when based on collaborative decision-making [21]. Another study focusing on detecting sepsis showed that the chance of severe sepsis was highest when estimated by two healthcare providers [22].

The most significant difference between our study and previous research efforts is that we intended to assess patients at the earliest ICU admission stage. In previous studies, the estimations were made within 24 h after ICU admission, leaving time for interventions and treatment responses that which might influence outcome predictions, and were mainly made by fewer physicians [23]. Treatment decisions and interventions influence prognostication only minimally in the first few hours after admission. Thereafter, many other factors, such as deterioration or improvement of organ function, logically influence the (prediction of the) outcome. To eventually guide clinical decision-making, it is essential to estimate mortality as soon as possible after, or even at, ICU admission to trigger reconsideration of ICU admittance and treatment restrictions.

The moderate accuracy of clinical predictions by individuals has been well established [24]. It is assumed that many supportive technical innovations in clinical decision-making will exponentially become available [25]. The challenge is to find and especially use the best possible algorithm to predict patient outcomes [26]. Nevertheless, healthcare providers can take into account improbable events and take variables that are currently unavailable for algorithms. It remains essential to keep an eye on the patient at the bedside, as previous studies have shown that often simple variables can predict adverse outcomes [27, 28]. Furthermore, the healthcare providers' “gut feeling,” or “clinical gestalt,” will probably never be fully understood but may be of crucial additive value. For example, Ferreyro et al. investigated physician- and patient-level factors influencing physicians' predictions of 6-month mortality in critically ill patients based on variables available at admission [29]. The authors found that older age, malignancy, and higher APACHE III scores were associated with the physicians’ predictions [29]. Although we did not investigate students, nurses, and physicians factors, we report similar observations for patient-level factors such as the SAPS II and APACHE IV score, the use of vasopressors, prolonged capillary refill time, and lower level of consciousness in our study. Using data from studies like from Ferreyro et al. or our study using Bayesian network analysis may further unravel how, why, and based on what variables healthcare providers make their predictions [30, 31]. Eventually, insight into the drivers of clinical predictions could be valuable for developing and evaluating decision algorithms and may inform future healthcare providers' education [32].

Implications and generalizability

This was a single-center study; however, outcome estimations are free, available worldwide, and do not seem to depend on ICU facilities. Our study design is widely applicable, and collaboration with other centers and other ICU departments might increase generalizability. It is important to note that these findings result from univariate analysis, as mortality prediction models are widely available [33], and developing a new one lies beyond the scope of this study. Currently, we do not know which factors help healthcare providers make their “educated guess.” Future research should focus on evaluating the conditional dependencies between the estimations and variables obtained from clinical examination to assess how healthcare providers reach their estimations (e.g., using Bayesian networks) [30]. This may lead to a better understanding of how these factors influence clinical decision-making and improve outcome prediction [34]. In the current study, we included nurses and medical students and their predictions, which led to improved collaborative mortality prediction. We showed that multidisciplinary research, including various healthcare providers with different experience levels, can lead to a successful observational study. We have learned that this experience—both participating in research and learning to perform a structured clinical examination of a critically ill patient—has benefitted the junior researchers involved during their medical or research career. In addition, it would be interesting to assess the rate of withdrawal of care during the ICU stay in future studies.

Limitations

Several limitations of this study must be acknowledged. First, students’, nurses', and physicians’ estimations are subjective and may be influenced by personal experience, beliefs, or specialty training. Second, unlike other studies, we did not report any characteristics of the students, nurses, and physicians, who provided the estimations of mortality [5, 23]. Furthermore, we reported the estimations based on a binary scale, using “Yes” and “No.” Physicians may be more likely to make mortality predictions on a continuous scale. Using a rating scale based on likelihood and providing more characteristics of the healthcare providers could have helped further characterize our results. Third, the results of the estimations of the nurses and physicians need to be interpreted carefully since the a priori chance of a good outcome is, in general, relatively high and might therefore influence the overall accuracy. Moreover, the students were instructed to complete their assessments first, and the physicians and nurses were also instructed to let the student make their estimates first. However, the students were not completely blinded to the estimations of the physicians and nurses, which could have influenced their estimations. Furthermore, there might have been a selection bias due to the relatively high number of missing estimations by physicians. These estimations, and those of others, may be missing because the physician or nurse was absent or occupied at the time students made their estimations. However, there was no significant clinical difference between patients with and without an estimation of mortality by the physician, and as we included a large cohort of unselected patients, we deem it unlikely that this could have influenced our results.

Conclusion

Based on the initial examination, students, nurses, and physicians can only moderately predict in-hospital and 6-month mortality in critically ill patients. Combined estimations led to more accurate predictions and may serve as an example of the benefit of multidisciplinary clinical care and future research efforts.

Abbreviations

ICU:

Intensive Care Unit

SICS-I:

Simple Intensive Care Studies-I

SICS-II:

Simple Intensive Care Studies-II

SOCCS:

Simple Observational Critical Care Studies

UMCG:

University Medical Center Groningen

APACHE IV:

Acute Physiology and Chronic Health Evaluation

AUROC:

Area under the receiver operating curve

CI:

Confidence interval

OR:

Odds ratio

References

  1. 1.

    Vincent JL, Marshall JC, Ñamendys-Silva SA, et al. Assessment of the worldwide burden of critical illness: the intensive care over nations (ICON) audit. Lancet Respir Med. 2014. https://doi.org/10.1016/S2213-2600(14)70061-X.

    Article  PubMed  Google Scholar 

  2. 2.

    Connors GR, Siner JM. Clinical reasoning and risk in the intensive care unit. Clin Chest Med. 2015;36(3):449–59. https://doi.org/10.1016/j.ccm.2015.05.016.

    Article  PubMed  Google Scholar 

  3. 3.

    Standing M. Clinical judgement and decision-making in nursing—nine modes of practice in a revised cognitive continuum. J Adv Nurs. 2008;62(1):124–34. https://doi.org/10.1111/j.1365-2648.2007.04583.x.

    Article  PubMed  Google Scholar 

  4. 4.

    Hayes MM, Chatterjee S, Schwartzstein RM. Critical thinking in critical care: five strategies to improve teaching and learning in the intensive care unit. Ann Am Thorac Soc. 2017;14(4):569–75. https://doi.org/10.1513/AnnalsATS.201612-1009AS.

    Article  PubMed  PubMed Central  Google Scholar 

  5. 5.

    Detsky ME, Harhay MO, Bayard DF, et al. Discriminative accuracy of physician and nurse predictions for survival and functional outcomes 6 months after an ICU admission. JAMA. 2017;317(21):2187. https://doi.org/10.1001/jama.2017.4078.

    Article  PubMed  PubMed Central  Google Scholar 

  6. 6.

    Adibi A, Sadatsafavi M, Ioannidis JPA. Validation and utility testing of clinical prediction models. JAMA. 2020;324(3):235. https://doi.org/10.1001/jama.2020.1230.

    Article  PubMed  Google Scholar 

  7. 7.

    Kahneman D, Sibony O, Sunstein CR. Noise: a flaw in human judgement; 2021.

  8. 8.

    King L, Clark JM. Intuition and the development of expertise in surgical ward and intensive care nurses. J Adv Nurs. 2002;37(4):322–9. https://doi.org/10.1046/j.1365-2648.2002.02105.x.

    Article  PubMed  Google Scholar 

  9. 9.

    Marino MA, Andrews K, Ward J. Clinical decision making at the bedside. Nurs Clin North Am. 2020;55(1):29–37. https://doi.org/10.1016/j.cnur.2019.10.003.

    Article  PubMed  Google Scholar 

  10. 10.

    Copeland-Fields L, Griffin T, Jenkins T, Buckley M, Wise LC. Comparison of outcome predictions made by physicians, by nurses, and by using the mortality prediction model. Am J Crit Care. 2001;10(5):313–9. https://doi.org/10.4037/ajcc2001.10.5.313.

    CAS  Article  PubMed  Google Scholar 

  11. 11.

    Wiersema R, Castela Forte JN, Kaufmann T, et al. Observational study protocol for repeated clinical examination and critical care ultrasonography within the simple intensive care studies. J Vis Exp. 2019;143: e58802. https://doi.org/10.3791/58802.

    Article  Google Scholar 

  12. 12.

    Wiersema R, Jukarainen S, Eck RJ, et al. Different applications of the KDIGO criteria for AKI lead to different incidences in critically ill patients: a post hoc analysis from the prospective observational SICS-II study. Crit Care. 2020;24(1):1–8. https://doi.org/10.1186/s13054-020-02886-7.

    Article  Google Scholar 

  13. 13.

    White N, Kupeli N, Vickerstaff V, Stone P. How accurate is the ‘surprise question’ at identifying patients at the end of life? A systematic review and meta-analysis. BMC Med. 2017;15(1):139. https://doi.org/10.1186/s12916-017-0907-4.

    Article  PubMed  PubMed Central  Google Scholar 

  14. 14.

    Downar J, Goldman R, Pinto R, Englesakis M, Adhikari NKJ. The “surprise question” for predicting death in seriously ill patients: a systematic review and meta-analysis. Can Med Assoc J. 2017;189(13):E484–93. https://doi.org/10.1503/cmaj.160775.

    Article  Google Scholar 

  15. 15.

    Scholz N, Bäsler K, Saur P, Burchardi H, Felder S. Outcome prediction in critical care: physicians’ prognoses vs. scoring systems. Eur J Anaesthesiol. 2004;21(8):606–611. http://www.ncbi.nlm.nih.gov/pubmed/15473614. Accessed 21 Aug 2019.

  16. 16.

    Rocker G, Cook D, Sjokvist P, et al. Clinician predictions of intensive care unit mortality. Crit Care Med. 2004;32(5):1149–54. https://doi.org/10.1097/01.CCM.0000126402.51524.52.

    Article  PubMed  Google Scholar 

  17. 17.

    Radtke A, Pfister R, Kuhr K, Kochanek M, Michels G. Is ‘gut feeling’ by medical staff better than validated scores in estimation of mortality in a medical intensive care unit? The prospective FEELING-ON-ICU study. J Crit Care. 2017;41:204–8. https://doi.org/10.1016/j.jcrc.2017.05.029.

    Article  PubMed  Google Scholar 

  18. 18.

    Hadique S, Culp S, Sangani RG, et al. Derivation and validation of a prognostic model to predict 6-month mortality in an intensive care unit population. Ann Am Thorac Soc. 2017;14(10):1556–61. https://doi.org/10.1513/AnnalsATS.201702-159OC.

    Article  PubMed  Google Scholar 

  19. 19.

    Lipson AR, Miano SJ, Daly BJ, Douglas SL. The accuracy of nurses’ predictions for clinical outcomes in the chronically critically ill. Res Rev J Nurs Heal Sci. 2017;3(2):35–8.

    Google Scholar 

  20. 20.

    Buehler AE, Ciuffetelli IV, Delman AM, et al. Contributors to intensive care unit clinicians’ predictions of patient outcomes: a qualitative analysis. Am J Crit Care. 2018;27(6):445. https://doi.org/10.4037/ajcc2018100.

    Article  PubMed  Google Scholar 

  21. 21.

    Neville TH, Wiley JF, Yamamoto MC, et al. Concordance of nurses and physicians on whether critical care patients are receiving futile treatment. Am J Crit Care. 2015;24(5):403–10. https://doi.org/10.4037/ajcc2015476.

    Article  PubMed  Google Scholar 

  22. 22.

    Bhattacharjee P, Churpek MM, Snyder A, Howell MD, Edelson DP. Detecting sepsis: are two opinions better than one? J Hosp Med. 2017;12(4):256–8. https://doi.org/10.12788/jhm.2721.

    Article  PubMed  Google Scholar 

  23. 23.

    Sinuff T, Adhikari NKJ, Cook DJ, et al. Mortality predictions in the intensive care unit: comparing physicians with scoring systems. Crit Care Med. 2006;34(3):878–85. https://doi.org/10.1097/01.CCM.0000201881.58644.41.

    Article  PubMed  Google Scholar 

  24. 24.

    Ægisdóttir S, White MJ, Spengler PM, et al. The meta-analysis of clinical judgment project: fifty-six years of accumulated research on clinical versus statistical prediction. Couns Psychol. 2006;34(3):341–82. https://doi.org/10.1177/0011000005285875.

    Article  Google Scholar 

  25. 25.

    Zellner M, Abbas AE, Budescu DV, Galstyan A. A survey of human judgement and quantitative forecasting methods. R Soc Open Sci. 2021;8(2):201187. https://doi.org/10.1098/rsos.201187.

    Article  PubMed  PubMed Central  Google Scholar 

  26. 26.

    Dietvorst BJ, Bharti S. People reject algorithms in uncertain decision domains because they have diminishing sensitivity to forecasting error. Psychol Sci. 2020;31(10):1302–14. https://doi.org/10.1177/0956797620948841.

    Article  PubMed  Google Scholar 

  27. 27.

    Hiemstra B, Eck RJ, Wiersema R, et al. Clinical examination for the prediction of mortality in the critically ill. Crit Care Med. 2019;47(10):1301–9. https://doi.org/10.1097/CCM.0000000000003897.

    Article  PubMed  Google Scholar 

  28. 28.

    Wiersema R, Koeze J, Eck RJ, et al. Clinical examination findings as predictors of acute kidney injury in critically ill patients. Acta Anaesthesiol Scand. 2020;64(1):69–74. https://doi.org/10.1111/aas.13465.

    CAS  Article  PubMed  Google Scholar 

  29. 29.

    Ferreyro BL, Harhay MO, Detsky ME. Factors associated with physicians’ predictions of six-month mortality in critically ill patients. J Intensive Care Soc. 2020;21(3):202–9. https://doi.org/10.1177/1751143719859761.

    Article  PubMed  Google Scholar 

  30. 30.

    Kaufmann T, Castela Forte J, Hiemstra B, et al. A bayesian network analysis of the diagnostic process and its accuracy to determine how clinicians estimate cardiac function in critically ill patients: prospective observational cohort study. JMIR Med Inform. 2019;7(4): e15358. https://doi.org/10.2196/15358.

    Article  PubMed  PubMed Central  Google Scholar 

  31. 31.

    Yarnell CJ, Jewell LM, Astell A, et al. Observational study of agreement between attending and trainee physicians on the surprise question: “Would you be surprised if this patient died in the next 12 months?” Hsieh JC-H, ed. PLoS ONE. 2021;16(2): e0247571. https://doi.org/10.1371/journal.pone.0247571.

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  32. 32.

    Neumann M, Hengeveld M, Niessen ASM, Tendeiro JN, Meijer RR. Education increases decision-rule use: an investigation of education and incentives to improve decision making. J Exp Psychol Appl. 2021. https://doi.org/10.1037/xap0000372.

    Article  PubMed  Google Scholar 

  33. 33.

    Keuning BE, Kaufmann T, Wiersema R, et al. Mortality prediction models in the adult critically ill: a scoping review. Acta Anaesthesiol Scand. 2020;64(4):424–42. https://doi.org/10.1111/aas.13527.

    Article  PubMed  Google Scholar 

  34. 34.

    Beldhuis IE, Marapin RS, Jiang YY, et al. Cognitive biases, environmental, patient and personal factors associated with critical care decision making: a scoping review. J Crit Care. 2021;64:144–53. https://doi.org/10.1016/j.jcrc.2021.04.012.

    Article  PubMed  Google Scholar 

  35. 35.

    Ait-Oufella H, Lemoinne S, Boelle PY, et al. Mottling score predicts survival in septic shock. Intensive Care Med. 2011;37(5):801–7. https://doi.org/10.1007/s00134-011-2163-y.

    CAS  Article  PubMed  Google Scholar 

Download references

Acknowledgements

We would like to thank all involved researchers, coordinators, and students from the SICS Study Group and SOCCS Student Team: Tamar van der Aart, Amila Azdahic, Diede Benjamins, Anke Bergsma, Yorick Bergsma, Jeroen Boekema, Vera Braun, Berend Bremer, Jan-Paul de Bruin, Elisa Chen, Chantal Dankfort, Karin Delfsma, Isabel Dieleman, Allysa Dijkstra, Alma Dijkstra, Elsa Doornbos, Ruben Eck, Rebecca van Elst, Dajana Erceg, Koen Geertzen, Chris Govaerts, Dion Groothof, Elisabeth Hagenauw, Xantia Heeres, Bart Hiemstra, Jildou Hoogland, Gerben Horstink, Maike Huizenga, Alexander Irk, Astrid Jansen, Thibault de Jonge, Thomas Kaufmann, Britt Keuning, Eric Keus, Charlotte Koerts, Evelyn van der Kooi, Femia Koopmans, Lisa Koops, Geert Koster, Menno de Leeuw, Lothar Mastenbroek, Ahra Meetsma, Gwen Miedema, Hidde Pelsma, George Pijpstra, Sarah Pruckl, Arlinde Roelofs, Maaike Schagen, Carine Schilte, Kim Selles, Elma Sluiter, Anna Smit, Lou van der Starre, Paul van Stee, Tim Takkenkamp, Melanie Theunis, Leonie Tijsma, Kabir Tombat, Nymke Trouwborst, Fennie Uiterwijk, Nynke van der Veen, Dorje Meilink, Bart Vinke, Erin Anne Visser, Brenda Wiersma, Marleen Wijma, Hinse Wiltingh, and Jelle Wolters

Funding

This research received no specific grant from any funding agency from any sector.

Author information

Affiliations

Authors

Consortia

Contributions

EC, IvdH, and RW drafted the manuscript and EC and RW conducted the analyses. IvdH and WD created the idea of the study. MO, MV, and RW coordinated the SOCCS Student Team. WP and JK critically reviewed the manuscript for important intellectual content. All authors critically reviewed the manuscript and agreed with the final version and findings.

Corresponding author

Correspondence to Eline G. M. Cox.

Ethics declarations

Ethics approval and consent to participate:

This study was approved by Medisch Ethische Toetsingscommissie, University Medical Center Groningen; 2017/507.

Consent for publication

Not applicable.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: Table S1

. Clinical characteristics of patients with and without the physicians' estimation.

Additional file 2: Table S2a

. Clinical characteristics of patients when the student estimated survival or non-survival. Table S2b. Clinical characteristics of patients when the nurse estimated survival or non-survival. Table S2c. Clinical characteristics of patients when the physician estimated survival or non-survival.

Additional file 3: Table S3

. Clinical characteristics of patients when all students, nurses, and physicians estimated non-survival.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Cox, E.G.M., Onrust, M., Vos, M.E. et al. The simple observational critical care studies: estimations by students, nurses, and physicians of in-hospital and 6-month mortality. Crit Care 25, 393 (2021). https://doi.org/10.1186/s13054-021-03809-w

Download citation

Keywords

  • Critically ill patients
  • Prognostic
  • Estimations
  • Students
  • Nurses
  • Physicians
  • Mortality prediction