Inter-rater reliability of APACHE II scores in the medical ICU

Introduction

The aim of this study was to determine inter-rater reliability of APACHE II scores between an ICU specialist and a well-trained ICU fellow.

Methods

In a prospective observational study, two raters collect APACHE II scores on 50 consecutive patients in a medical ICU respectively. Intraclass correlation coefficients were calculated for the APACHE II total score, the APACHE II component scores, and the Glasgow coma scale components. Concordance correlation coefficients for each chosen value of the APACHE II component were assessed.

Results

Mean (standard deviation) APACHE II scores were 21.6 (8.2) for the specialist, and 21.7 (7.1) for the fellow. The intraclass correlation coefficient was 0.848 for the APACHE II total score. Within the score components, the inter-rater reliabilities of acute physiology score, age, and chronic health evaluation were 0.860, 0.987, and 0.645, respectively. See Table 1. Further analysis of each chosen value of the APACHE II component found the lowest reliability for the mean artery pressure, which was 0.482. Further investigation discovered that the most common reason for inter-rater difference is data dropout.

Table 1 Overall agreement

Conclusion

The agreements of the APACHE II score among the two collectors was good, but there were more differences in collecting the value of mean artery pressure, for which the original data could not be read directly from medical chart.