Table 8 New model development
Study | Ahluwalia et al. (1999) [29] | Riviello et al. (2016) [43] | Riviello et al. (2016) [43] | Nimgaonkar et al. (2004) [35] | Nimgaonkar et al. (2004) [35] |
|---|---|---|---|---|---|
Model | New score | Rwanda MPM (R-MPM) | Simplified R-MPM | Artificial Neural Network (ANN 22) | Artificial Neural Network (ANN 15) |
Source | Prospective cohort | Prospective cohort | Prospective cohort | Prospective cohort | Prospective cohort |
Participants | Consecutive admissions (>13 years) to eight-bed medical ICU, India; inclusion period NR; participant age range 13–80, mean = 46 | Consecutive patients (>15 years) admitted to two ICUs in different hospitals; exclusion criteria: not specified; August 2013–October 2014; participant age range 34 years (IQR 25–47) (median) | Consecutive patients (>15 years) admitted to two ICUs in different hospitals; exclusion criteria: not specified; August 2013–October 2014; participant age range 34 years (IQR 25–47) (median) | All consecutive patients (>12 years) admitted to 17-bed medical–neurological ICU, tertiary referral hospital, India; January 1996–May 1998 | All consecutive patients (>12 years) admitted to 17-bed medical–neurological ICU, tertiary referral hospital, India; January 1996–May 1998 |
Outcomes | Hospital mortality | Hospital mortality | Hospital mortality | Hospital mortality | Hospital mortality |
Predictors | 1. pH (at admission); 2. serum albumin (at admission); 3. heart rate (at 48Â hours); 4. GCS (at 48Â hours); 5. bilirubin (at 48Â hours) | Only the following five variables were included: 1. age; 2.confirmed or suspected infection within 24Â hours of ICU admission; 3. hypotension or shock as a reason for ICU admission; 4. heart rate at ICU admission; 5. GSC at time of admission | Altered mental status on ICU admission (present vs not present) used in place of the GCS score in the R-MPM (see previous model) | 22 APACHE II variables | 15 APACHE II variables with the highest information gain (measured by calculation of entropy) |
Sample size | 79 | 427 | 427 | 2962 | 2962 |
Missing data | Not reported | Normal values attributed as in original study; two patients excluded due to lack of discharge status | Normal values attributed as in original study; two patients excluded due to lack of discharge status | Not reported | Not reported |
Model development | Based on APACHE II (Knaus et al. 1985 [10]) and 11 other clinical and laboratory parameters. Backward step method used to remove non-significant (p > 0.05) variables (of univariate analysis) | Based on the 16 MPM III (initial) and additional variables. Variables for inclusion in model selected from the univariate analyses, based on their predictive power (as determined by p < 0.05) as well as their ease of capture based on experience, the proportion of missing values in the dataset, and their clinical significance | Based on the 16 MPM III (initial) and additional variables. Variables for inclusion in model selected from the univariate analyses, based on their predictive power (as determined by p < 0.05) as well as their ease of capture based on experience, the proportion of missing values in the dataset, and their clinical significance | Artificial Neural Network trained on an Indian patient dataset using the 22 APACHE II variables | Artificial Neural Network trained on an Indian patient dataset using the 15 APACHE II variables with the highest information gain (measured by calculation of entropy) |
Model performance | Discrimination measured in terms of AUROC, sensitivity and specificity. Multivariate and univariate regression | Discrimination measured in terms of AUROC. Calibration measured as Hosmer–Lemeshow. Multivariate and univariate regression | Discrimination measured in terms of AUROC. Calibration measured as Hosmer–Lemeshow. Multivariate and univariate regression | Discrimination measured as AUROC. Calibration measured as Hosmer–Lemeshow | Discrimination measured as AUROC. Calibration measured as Hosmer–Lemeshow |
Model evaluation | Developmental dataset only, no further evaluation (compared with APACHE II at 48Â hours) | Internal validation with bootstrapping (compared with MPM III (initial)) | Internal validation with bootstrapping (compared with MPM III (initial)) | Data from 1962 patients were used to train the neural network using a back-propagation algorithm. Data from the remaining 1000 patients were used for testing this model and comparing it with APACHE II | Data from 1962 patients were used to train the neural network using a back-propagation algorithm. Data from the remaining 1000 patients were used for testing this model and comparing it with APACHE II |
Results | New score ROC: 0.90, sensitivity: 98.2%, specificity: 66.6%. APACHE II (after 48 hours) ROC: 0.74, sensitivity: 92.8%, specificity: 23.6% | Rwanda MPM (R-MPM) AUROC: 0.81 (0.77–0.86), HL: χ2 = 11.94 (p = 0.154). MPM III (initial) AUROC: 0.72, HL: χ2 = 17.66 (p = 0.024) | Simplified R-MPM AUROC: 0.76, HL: χ2 = 11.46 (p = 0.177). MPM III (initial) AUROC: 0.72, HL: χ2 = 17.66 (p = 0.024) | ANN 22 AUROC: 0.87, HL H statistic: χ2 = 22.4 (p < 0.05). APACHE II AUROC: 0.77, HL H statistic: χ2 = 123.5 (p < 0.05) | ANN 15 AUROC: 0.88, HL H statistic: χ2 = 27.7 (p < 0.05). APACHE II AUROC: 0.77, HL H statistic: χ2 = 123.5 (p < 0.05) |