External validation of six mortality prediction systems for intensive care unit patients

Introduction

This study investigates the prognostic quality of six prediction models (APACHE II score, SAPS II, SAPS II (Expanded), SAPS 3, MPM and MPM II0) for assessment in an adult surgical ICU in the Netherlands.

Methods

Of all single admissions to the surgical ICU of the St Elisabeth Hospital between 1995 and 2000, data to calculate the results of six prediction models were prospectively documented. To evaluate discrimination and calibration, receiver operating characteristic (ROC) curves, area under the characteristic (AUC) curve and the Hosmer-Lemeshow goodness-of-fit test were performed.

Results

The data of 1,821 patients were applied to all six models. Accurate overall mortality prediction was found for the APACHE II, SAPS II, SAPS 3 and MPM models. Discrimination was best for the SAPS 3 model and worst for the APACHE II and MPM models with AUC of 0.81, 0.77 and 0.77, respectively (Figure 1). Calibration was poor for the six prediction systems, varying between 23 (SAPS 3) and 233 (SAPS II (Expanded)).

Figure 1

Receiver operating characteristic curves for all six prediction models.

Conclusions

The SAPS 3 prediction mortality model is the best validated model for a Dutch surgical ICU population of six models tested. The other general prognostic models for ICU patients underestimate the risk of dying.