Is Paediatric Index of Mortality 2 already out of date?

Aim

The Paediatric Index of Mortality (PIM) is the most widely used mortality-risk score in UK paediatric intensive care units (PICU). A recalibrated version, PIM 2, has recently been published. Although released in 2003, PIM 2 was derived from data collected between 1998 and 1999. We aimed to assess temporal changes in performance of both of these scores. Specifically, we hypothesised that the 6-year period between commencement of data collection (1998) and publication (2003) of the revised PIM 2 score may have allowed for significant decalibration.

Methods

A prospective data collection from a single, 20-bed tertiary PICU over 5 years (1999–2003). The standardised mortality ratio (SMR) was calculated using the standard formula, discrimination assessed via the area under the receiver–operating characteristic curve (ROC), and calibration using the Hosmer–Lemeshow goodness of fit test.

Results

Scores were calculated for 4183 patient episodes (average 800–900 admissions/year). There was no significant temporal variation in either case mix (cardiac surgery 27–31%) or disease severity (data not shown). Both scores discriminated well, consistently yielding an area under the ROC curve > 0.75 (Fig. 1 top). Not surprisingly, PIM demonstrated a loss of calibration (Hosmer–Lemeshow χ² > 15.5, P > 0.05) from 2000 onwards. PIM 2 also showed a temporal trend towards decalibration that became apparent by 2003. This trend was mirrored by a progressive reduction in SMR (Fig. 1 bottom), such that the upper confidence limit for the PIM 2 SMR was less than 1.00 by 2002. Interestingly, the SMR derived from either score decreased at a similar rate, 0.07–0.08 per year.

Figure 1

Conclusions

Both scores continue to discriminate well between survival and nonsurvival. As expected, PIM 2 is better calibrated than PIM, although both appear to be decalibrating at a similar rate. Because PIM 2 reflects the 1998–1999 standard of care, many PICUs will currently exhibit fewer deaths than expected (SMR < 1.00). More frequent calibration appears necessary.