Skip to main content

Table 8 New model development

From: Performance of critical care prognostic scoring systems in low and middle-income countries: a systematic review

Study Ahluwalia et al. (1999) [29] Riviello et al. (2016) [43] Riviello et al. (2016) [43] Nimgaonkar et al. (2004) [35] Nimgaonkar et al. (2004) [35]
Model New score Rwanda MPM (R-MPM) Simplified R-MPM Artificial Neural Network (ANN 22) Artificial Neural Network (ANN 15)
Source Prospective cohort Prospective cohort Prospective cohort Prospective cohort Prospective cohort
Participants Consecutive admissions (>13 years) to eight-bed medical ICU, India; inclusion period NR;
participant age range 13–80, mean = 46
Consecutive patients (>15 years) admitted to two ICUs in different hospitals; exclusion criteria: not specified; August 2013–October 2014;
participant age range 34 years (IQR 25–47) (median)
Consecutive patients (>15 years) admitted to two ICUs in different hospitals; exclusion criteria: not specified; August 2013–October 2014;
participant age range 34 years (IQR 25–47) (median)
All consecutive patients (>12 years) admitted to 17-bed medical–neurological ICU, tertiary referral hospital, India; January 1996–May 1998 All consecutive patients (>12 years) admitted to 17-bed medical–neurological ICU, tertiary referral hospital, India; January 1996–May 1998
Outcomes Hospital mortality Hospital mortality Hospital mortality Hospital mortality Hospital mortality
Predictors 1. pH (at admission); 2. serum albumin (at admission); 3. heart rate (at 48 hours); 4. GCS (at 48 hours); 5. bilirubin (at 48 hours) Only the following five variables were included: 1. age; 2.confirmed or suspected infection within 24 hours of ICU admission; 3. hypotension or shock as a reason for ICU admission; 4. heart rate at ICU admission; 5. GSC at time of admission Altered mental status on ICU admission (present vs not present) used in place of the GCS score in the R-MPM (see previous model) 22 APACHE II variables 15 APACHE II variables with the highest information gain (measured by calculation of entropy)
Sample size 79 427 427 2962 2962
Missing data Not reported Normal values attributed as in original study;
two patients excluded due to lack of discharge status
Normal values attributed as in original study;
two patients excluded due to lack of discharge status
Not reported Not reported
Model development Based on APACHE II (Knaus et al. 1985 [10]) and 11 other clinical and laboratory parameters. Backward step method used to remove non-significant (p > 0.05) variables (of univariate analysis) Based on the 16 MPM III (initial) and additional variables. Variables for inclusion in model selected from the univariate analyses, based on their predictive power (as determined by p < 0.05) as well as their ease of capture based on experience, the proportion of missing values in the dataset, and their clinical significance Based on the 16 MPM III (initial) and additional variables. Variables for inclusion in model selected from the univariate analyses, based on their predictive power (as determined by p < 0.05) as well as their ease of capture based on experience, the proportion of missing values in the dataset, and their clinical significance Artificial Neural Network trained on an Indian patient dataset using the 22 APACHE II variables Artificial Neural Network trained on an Indian patient dataset using the 15 APACHE II variables with the highest information gain (measured by calculation of entropy)
Model performance Discrimination measured in terms of AUROC, sensitivity and specificity. Multivariate and univariate regression Discrimination measured in terms of AUROC.
Calibration measured as Hosmer–Lemeshow.
Multivariate and univariate regression
Discrimination measured in terms of AUROC.
Calibration measured as Hosmer–Lemeshow.
Multivariate and univariate regression
Discrimination measured as AUROC.
Calibration measured as Hosmer–Lemeshow
Discrimination measured as AUROC.
Calibration measured as Hosmer–Lemeshow
Model evaluation Developmental dataset only, no further evaluation (compared with APACHE II at 48 hours) Internal validation with bootstrapping (compared with MPM III (initial)) Internal validation with bootstrapping (compared with MPM III (initial)) Data from 1962 patients were used to train the neural network using a back-propagation algorithm. Data from the remaining 1000 patients were used for testing this model and comparing it with APACHE II Data from 1962 patients were used to train the neural network using a back-propagation algorithm. Data from the remaining 1000 patients were used for testing this model and comparing it with APACHE II
Results New score ROC: 0.90, sensitivity: 98.2%, specificity: 66.6%.
APACHE II (after 48 hours) ROC: 0.74, sensitivity: 92.8%, specificity: 23.6%
Rwanda MPM (R-MPM) AUROC: 0.81 (0.77–0.86),
HL: χ2 = 11.94 (p = 0.154).
MPM III (initial)
AUROC: 0.72,
HL: χ2 = 17.66 (p = 0.024)
Simplified R-MPM
AUROC: 0.76,
HL: χ2 = 11.46 (p = 0.177).
MPM III (initial)
AUROC: 0.72,
HL: χ2 = 17.66 (p = 0.024)
ANN 22 AUROC: 0.87, HL
H statistic:
χ2 = 22.4 (p < 0.05).
APACHE II
AUROC: 0.77,
HL
H statistic:
χ2 = 123.5 (p < 0.05)
ANN 15
AUROC: 0.88,
HL
H statistic:
χ2 = 27.7 (p < 0.05).
APACHE II
AUROC: 0.77,
HL
H statistic:
χ2 = 123.5 (p < 0.05)
  1. APACHE Acute Physiology and Chronic Health Evaluation, MPM Mortality Probability Models, ICU intensive care unit, GCS Glasgow Coma Score, IQR interquartile range, HL Hosmer–Lemeshow statistic, AUROC area under the receiver operating characteristic