Model calibration and discriminatory ability: a comparison of four derived variables from the SOFA score and the SAPS II

Introduction

We sought to compare four derived variables from the SOFA score and the SAPS II in ICU patients in terms of discriminatory ability and model calibration.

Patients and methods

Four hundred and fourteen patients were included; they were evaluated on admission and every 48 hours thereafter until ICU discharge or death. Readmission and patients with an ICU stay shorter than 48 hours were excluded. The TMS score was calculated by summing the worst scores for each of the organ systems. Organ failure was defined by a SOFA score ≥3. ΔSOFA was defined by TMS minus admission SOFA (SOFAi). The maximum SOFA was defined by the worst SOFA value during the ICU stay. Logistic regression modeling techniques were used to describe the association of derived SOFA variables and SAPS II with mortality. ROC curves were used to assess the model's discriminatory ability and we examined the model calibration using the Hosmer–Lemeshow goodness-of-fit test. P < 0.05 was considered significant.

Results

Diagnostic categories were: trauma 21.3%, postoperative 19% and medical 59.7%. Global mortality was 34.3%. Survivors had lower average SAPS II (28.1 ± 14 against 48.6 ± 19, P < 0.01), SOFAi score (3.7 ± 3 against 7.2 ± 4, P < 0.01), SOFAmax score (4.6 ± 4 against 10.8 ± 3, P < 0.01), ΔSOFA (1.6 ± 6 against 4.2 ± 3, P < 0.01), DoMAX (1.6 ± 6 against 4.2 ± 3, P < 0.01) and TMS (5 ± 3 against 11.4 ± 4, P < 0.01), and the difference was statistically significant. Results regarding model calibration and discriminatory ability are presented in Figure 1.

Figure 1

*Classified as + if predicted mortality ≥0.5. Domax, the maximum number of organ failures during ICU stay.

Conclusion

The SOFAmax score had the best model calibration and could be used to compare different patient populations in terms of mortality.