Comparison of three severity of illness scoring systems for intensive care unit (ICU) patients

Background

Scoring systems have been proposed to assist in assessing prognosis, to compare ICU performance and to stratify patients for clinical trials. Three different models of severity of illness scoring systems (APACHE II, SAPS II and MPM 24) have been widely used to evaluate critically ill patients but which one is better to measure severity of illness and to predict hospital outcome?

Study objective

To compare the performance of these three scoring systems in the same cohort of patients.

Methods

Data was prospectivelly collected for each ICU admission. In order to strictly follow the models rules, patients who stayed less than 24 h at the ICU or were younger than 18 years or were burn, coronary care or cardiac surgery patients were excluded. The outcome measure was vital status at hospital discharge. The discrimination was evaluated using ROC curve area and for the calibration was used the Hosmer-Lemeshow goodness-of-fit test

Results

Out of 283 consecutives ICU admissions, there were 172 patients who were eligible by the criteria and had full sets of data. There were 69.2% male and 30.8% female patients; age was 45 ± 18.5 (61% had less than 50 years old) and postoperative care took up 99 (57.6%) cases, of which 84 (85%) were emergency surgery. Trauma was the admission cause for 65 (37.8%) patients. APACHE II was 17.6 ± 8.3 and SAPS II was 33.2 ± 16.1. ICU mortality rate was 34.3% and hospital mortality rate was 43.6%.

Conclusions

The truest assessment of adequacy of a predictive model is through goodness-of-fit test that compares expected with observed frequencies. It is possible for a method to have a high ROC curve but to not fit an observed set of data well. At this study, all three models showed good discrimination power, that is, they were able to distinguish patients who lived from patients who died. Nevertheless, the calibration was very poor, that is, the predictions did not correlate with the actual outcome across the entire range of risk. This finding may be due to meaningful differences between this study casemix and the original development populations (too many emergency surgery and trauma patients in this study). Furthermore, resource utilization, type of treatment and quality of care should be reviewed and considered when evaluating hospital mortality.

Table