Skip to main content

Table 3 Prediction performance of the reference and machine learning models in the test set

From: Emergency department triage prediction of clinical outcomes using machine learning models

Outcome and model AUC P value* NRI† P value† Sensitivity Specificity PPV NPV
Critical care outcome
 Reference model 0.74 (0.72–0.75) Reference Reference Reference 0.50 (0.47–0.53) 0.86 (0.82–0.87) 0.07 (0.05–0.08) 0.988 (0.988–0.988)
 Lasso regression 0.84 (0.83–0.85) < 0.001 0.39 (0.32–0.46) < 0.001 0.75 (0.72–0.78) 0.77 (0.75–0.80) 0.06 (0.06–0.07) 0.993 (0.993–0.994)
 Random forest 0.85 (0.84–0.87) < 0.001 0.07 (0.003–0.14) 0.04 0.86 (0.83–0.88) 0.68 (0.68–0.71) 0.05 (0.05–0.06) 0.996 (0.996–0.996)
 Gradient boosted decision tree 0.85 (0.83–0.86) < 0.001 0.32 (0.25–0.38) < 0.001 0.75 (0.73–0.79) 0.77 (0.75–0.80) 0.06 (0.06–0.07) 0.993 (0.993–0.994)
 Deep neural network 0.86 (0.85–0.87) < 0.001 0.73 (0.67–0.79) < 0.001 0.80 (0.77–0.83) 0.76 (0.73–0.78) 0.06 (0.06–0.07) 0.995 (0.994–0.995)
Hospitalization outcome
 Reference model 0.69 (0.68–0.69) Reference Reference Reference 0.87 (0.86–0.87) 0.42 (0.39–0.43) 0.23 (0.22–0.23) 0.94 (0.94–0.94)
 Lasso regression 0.81 (0.80–0.81) < 0.001 0.53 (0.50–0.55) < 0.001 0.71 (0.70–0.72) 0.76 (0.75–0.77) 0.36 (0.35–0.37) 0.93 (0.93–0.93)
 Random forest 0.81 (0.81–0.82) < 0.001 0.66 (0.63–0.68) < 0.001 0.77 (0.76–0.78) 0.71 (0.70–0.72) 0.34 (0.33–0.35) 0.94 (0.94–0.94)
 Gradient boosted decision tree 0.82 (0.82–0.83) < 0.001 0.63 (0.61–0.66) < 0.001 0.75 (0.73–0.76) 0.75 (0.74–0.76) 0.37 (0.36–0.38) 0.94 (0.94–0.94)
 Deep neural network 0.82 (0.82–0.83) < 0.001 0.68 (0.65–0.70) < 0.001 0.79 (0.78–0.80) 0.71 (0.69–0.72) 0.35 (0.34–0.36) 0.95 (0.94–0.95)
  1. Abbreviations: AUC area under the curve, NRI net reclassification improvement, PPV positive predictive value, NPV negative predictive value
  2. *P value was calculated to compare the area under the receiver-operating-characteristics curve (AUC) of the reference model with that of each machine learning model
  3. †We used continuous NRI and its P value