Skip to main content

A study on the reliability and validity of two four-level emergency triage systems


We compared the reliability and validity of two four-level emergency triage systems: the ETS1, used in our emergency department (ED); and the ETS2, a triage algorithm derived from the Emergency Severity Index [1].


This is an observational retrospective study of 189 patients admitted to our ED in 2 weeks of October 2006. Triage scenarios were designed with medical records. Ten trained triage nurses were randomly assigned either to training in ETS2, or to refresh the ETS1. They independently assigned triage scores to the scenarios, at time zero and after 6 months. Both triage systems have four urgency categories (UC): UC1 = immediate response; UC2, UC3, UC4 = assessment within 20, 60, 120 minutes. We collected demographic and clinical characteristics, nurse triage category, admission status and site, nurse triage forms with presenting complaint, mode and time of arrival, past diseases, vital signs, and pain score. For each scenario we considered 'true triage' the mode of the UC assigned by the nurses. Weighted kappa (K) was used to calculate inter-rater and intra-rater reliability in each of the two groups of nurses. The relationships between the 'true triage' and admission, and admission site were assessed.


The UCs assigned were similar in two groups: 20% versus 21% with UC4, 50% versus 48% with UC3, 28% versus 28% with UC2, 2% versus 3% with UC1. A complete disagreement in UC was found in 3% and 5% cases of ETS1 and ETS2; a complete agreement in 52% and 56% cases of ETS1 and ETS2. Inter-rater reliability among nurses using ETS1 and ETS2 was K = 0.73 (95% CI: 0.59–0.87) and K = 0.79 (95% CI: 0.65–0.93), respectively; intra-rater reliability was: K = 0.82 (95% CI: 0.67–0.96) and K = 0.78 (95% CI: 0.62–0.93) in ETS1 and ETS2. Hospital admission by ETS1 and ETS2 was similar for UC2 (39% vs 37%), UC3 (5% vs 8%), and UC4 (3% vs 0%); 100% of patients with UC1 in ETS1 and 60% in ETS2 were admitted to the ICU.


The two emergency triage systems showed similar reliability and validity. ETS2 is easier to consult but worse in prediction of ICU admission. To our knowledge this is the first study on the intra-rater reliability of two four-level emergency triage systems.


  1. 1.

    Wuerz RC, Milne LW, Eitel DR, et al.: Reliability and validity of a new five-level triage instrument. Acad Emerg Med 2000, 7: 236-242. 10.1111/j.1553-2712.2000.tb01066.x

    PubMed  CAS  Article  Google Scholar 

Download references

Author information



Rights and permissions

Reprints and Permissions

About this article

Cite this article

Parenti, N., Ferrara, L., Sangiorgi, D. et al. A study on the reliability and validity of two four-level emergency triage systems. Crit Care 12, P352 (2008).

Download citation


  • Weighted Kappa
  • Observational Retrospective Study
  • Complete Disagreement
  • Nurse Triage
  • Admission Status