A retrospective cohort pilot study to evaluate a triage tool for use in a pandemic

Abstract

Introduction

The objective of this pilot study was to assess the usability of the draft Ontario triage protocol, to estimate its potential impact on patient outcomes, and ability to increase resource availability based on a retrospective cohort of critically ill patients cared for during a non-pandemic period.

Methods

Triage officers applied the protocol prospectively to 2 retrospective cohorts of patients admitted to 2 academic medical/surgical ICUs during an 8 week period of peak occupancy. Each patient was assigned a treatment priority (red -- 'highest', yellow -- 'intermediate', green -- 'discharge to ward', or blue/black -- 'expectant') by the triage officers at 3 separate time points (at the time of admission to the ICU, 48, and 120 hours post admission).

Results

Overall, triage officers were either confident or very confident in 68.4% of their scores; arbitration was required in 54.9% of cases. Application of the triage protocol would potentially decrease the number of required ventilator days by 49.3% (568 days) and decrease the total ICU days by 52.6% (895 days). On the triage protocol at ICU admission the survival rate in the red (93.7%) and yellow (62.5%) categories were significantly higher then that of the blue category (24.6%) with associated P values of < 0.0001 and 0.0003 respectively. Further, the survival rate of the red group was significantly higher than the overall survival rate of 70.9% observed in the cohort (P < 0.0001). At 48 and 120 hours, survival rates in the blue group increased but remained lower then the red or yellow groups.

Conclusions

Refinement of the triage protocol and implementation is required prior to future study, including improved training of triage officers, and protocol modification to minimize the exclusion from critical care of patients who may in fact benefit. However, our results suggest that the triage protocol can help to direct resources to patients who are most likely to benefit, and help to decrease the demands on critical care resources, thereby making available more resources to treat other critically ill patients.

Introduction

On 11 June, 2009 the World Health Organization acknowledged that the world was facing a pandemic caused by a novel strain of H1N1 influenza [1]. Although to date the overall prevalence of severe H1N1 illness has been low, experiences in Mexico [2, 3] and Manitoba [4] have increased concern that scarcities of critical care resources, such as mechanical ventilators [5, 6], will occur if a large second wave strikes during the fall in the Northern Hemisphere. Surge response strategies [7–11] will partially mitigate the surge impact, but may be inadequate to fully address health care demands. When faced with scarce resources, the principles of biomedical ethics and international law dictate that triage protocols be used to guide resource allocation [12–14].

In 2006, an expert panel convened by the Ontario Health Plan for an Influenza Pandemic Steering Committee developed a draft critical care triage protocol [15]. The purpose of the protocol is to provide guidance for making triage decisions if critical care services are overwhelmed [5, 6, 16, 17].

The objective of this pilot study was to assess the usability of the draft triage protocol, to estimate its potential impact on patient outcomes, and its potential to increase resource availability based on a cohort of critically ill patients cared for during a non-pandemic period.

Materials and methods

We identified two retrospective cohorts of consecutive patients admitted to two academic medical/surgical intensive care units (ICUs) during an eight-week period of peak occupancy. ICU 'A' is a 16-bed general medical-surgical unit with 800 to 850 annual admissions serving a 472-bed tertiary care hospital and a 220-bed cancer hospital. ICU 'B' is a 23-bed medical-surgical-trauma-neurosurgical ICU with approximately 1200 admissions per year serving a 972-bed tertiary care hospital. Research ethics board approval was obtained at each of the participating institutions. The requirement for informed consent was waived by the research ethics board. Trained research assistants abstracted data from patient charts to create the case summary. An intensivist member of the study team at each site was available to answer clinical questions and review difficult cases. All case report forms were reviewed by the principal investigator, MDC, who requested additional data or site investigator chart review as necessary to clarify missing or unclear information. Data were entered into an Excel spreadsheet (Microsoft Excel 2003, Microsoft Corporation. USA).

An overview of the triage process is presented in Figure 1. Research assistants identified all patients admitted to these two participating ICUs from the ICU admission logs. Patients were enrolled into the study if they met the inclusion criteria for ICU admission defined in the triage protocol [see Additional data file 1] [15]. Patients were then screened to see if they met any of the protocol's initial exclusion criteria or 'minimum qualifications for survival' (MQS) [see Additional data file 1] [15]. Any patients who did not fulfill the triage protocol's initial exclusion criteria were summarized and presented to two of four intensivists who served as 'triage officers' to apply the triage protocol. Three of the four officers were involved in drafting the Ontario triage protocol. All triage officers received one hour of training on how to apply the protocol for this study (three in a single session and the fourth was one-on-one with the principal investigator due to scheduling difficulties). The training focus was triaging sample cases and calibration of the protocol application. Triage officers were instructed to imagine that they were actually conducting triage during a pandemic where demand for critical care services exceeded the available capacity. For study purposes, they were instructed that following the triage protocol was to be considered the 'standard of care' during the pandemic period. Deviations from the protocol would hold the same potential risks and consequences as do deviations from the standard of care during non-pandemic situations.

Figure 1

Triage process flow. ICU = intensive care unit; MQS = minimum qualifications for survival.

Patients at each of the two sites were scored independently by two of the triage officers. The triage officers did not review patients from the institution in which they practice, did not discuss these cases with one another during the triage process, and had no prior knowledge of the patients or their outcomes. Patient profiles were presented electronically in PDF format to the triage officers. The profiles included the patient's demographic data, admission diagnosis and limited past medical history. Daily triage reports were presented to the triage officers with the sequential organ failure assessment (SOFA) score component data and a calculated SOFA score, one day at a time, for each of the first five ICU days. The triage officers were instructed to make and record their determination of the patient's treatment priority before advancing to the next day's triage report, although they were allowed to move back in the PDF to review the patient profile and prior day's SOFA scores. Thus, at the time of their triage assessments, the triage officers were blinded to future status, and patient outcome.

Each patient was assigned a treatment priority (red = 'highest', yellow = intermediate', green = 'discharge to ward', or blue/black = 'expectant') by the triage officers at three separate time points (at the time of admission to the ICU, and 48, and 120 hours after admission). In addition, on day 1, 3 and 4 of their ICU admission, patients were assessed by the triage officer for exclusion criteria. Disagreements were resolved by a third intensivist (the principal investigator) scoring the case, in the same manner as described above, blinded to the decisions of the other triage officers and patient outcomes.

Protocol endpoints due to triage protocol exclusion criteria events or reaching discharge criteria beyond day 5 were assessed by a research assistant in conjunction with the principal investigator. We recorded the date and vital status of the patient at discharge from the ICU and hospital, or at day 90 following enrollment if the patient was still in the hospital.

Usability of the draft triage protocol was assessed based on: intensivist rating of confidence in their assignment of triage priority as reported in a brief questionnaire using five-point Likert-style scales assessing completeness of data, relevance of information provided, ability to develop a clear clinical understanding of the case and overall confidence in decision; requirement for third intensivist assessment; and chance-corrected (kappa) in assignment of triage priority. Based on the usability criteria, the triage protocol was considered successful in its current format if overall confidence was rated as confident or very confident in at least 90% of cases and disagreements requiring a third assessment in less then 5% of cases.

We assessed the impact of the triage protocol on resource availability by calculating the difference in days between when a patient achieved an endpoint in the triage protocol, exclusion criteria or prioritized as either blue 'expectant' or green 'dischargeable', compared with the actual date of extubation and discharge from the ICU. This allowed calculation of the number of ICU and ventilator days made available under the triage protocol.

Patient status was recorded at discharge from ICU, discharge from hospital or day 90 following enrollment if the patient remained admitted to hospital. Patients were categorized as deceased or alive at discharge from ICU and hospital, or if still admitted to hospital patients were categorized as: 'requiring ongoing life support', 'undergoing active medical care', or 'awaiting transfer to chronic care/rehabilitation'. Patients who were discharged from hospital alive had their destination of discharge categorized during chart abstraction as: 'home', 'chronic care', 'rehabilitation' or 'transfer to another acute care facility'.

Statistical analysis

We present descriptive data using mean and standard deviation for continuous variables, or median and interquartile ranges if data were skewed, and proportions and 95% confidence intervals for dichotomous variables. We compared continuous variables using unpaired t-tests (means) or Wilcoxon Rank-Sum (medians), and dichotomous variables using a chi-squared test or a Fisher's exact test if any expected cell frequencies were less than five. All tests were two-sided and P < 0.05 was considered statistically significant. We used Cohen's kappa (κ) with 95% confidence intervals to calculate chance-corrected agreement between triage officers. Based on the usability criteria the triage protocol would be considered successful in its current format if overall confidence is rated as confident or very confident in at least 90% of cases and disagreements requiring third assessment in less then 5% of cases. Missing data for the calculation of the SOFA score were assumed to be normal if the parameter was never collected by the clinicians caring for the patient, or the last reported value was used if the parameter had previously been measured but was missing at the time in question.

Results

A total of 234 patients were included in the cohort (Figure 1 and Table 1), of which 178 (76.1%) met the triage inclusion criteria and would have been admitted to ICU during a pandemic based on the protocol. Of the overall cohort, 39.7% at some point met either the triage exclusion criteria or MQS and thus would have been managed expectantly (triaged blue). The number of patients who met triage inclusion or exclusion criteria was similar between hospitals. The mean age of the cohort was 59.8 years old and was similar between hospitals. A lower percentage of patients in hospital B were women, most likely because it is a trauma center. Overall, 69.2% of patients were mechanically ventilated, while 5.1% were never ventilated but were hypotensive and required vasopressor support. Overall, ICU survival was 76.9% and hospital survival was 70.9%.

Table 1 Cohort description.

The primary outcome measure was the usability of the triage protocol as measured by triage officer confidence in their assignment of priority, requirement for arbitration and agreement with each other. Overall, triage officers were either confident or very confident in 68.4% of their scores and arbitration was required in 54.9% of cases. The agreement between raters (Table 2) was highest on admission (kappa 89.2 (77.9, 100) and 73.7 (55.6, 91.7) for hospitals A and B, respectively. Agreement at 48 hours and 120 hours were substantially lower (48 hours: kappa 60.7 (41.2, 80.2) and 31.5 (16.1, 46.8); 120 hours: kappa 42.4 (28.0, 57.0) and 0 (-17.2, 16.2) for hospitals A and B). The latter analyses were based on fewer patients because only patients who were not triaged blue and were still in the ICU were triaged at these subsequent time points.

Table 2 Usability of protocol

Secondary outcome measures of this pilot study included an assessment of the impact of the triage protocol on the availability of critical care resources and measurement of patient outcomes. Total ventilator and ICU days on and off protocol are reported in Table 3. Application of the triage protocol would potentially decrease the number of ventilator days required by 49.3% (568 days) and a decrease of 52.6% (895 days) in the total ICU days. Table 4 presents outcomes based on triage category at admission, 48 hours and 120 hours. On the triage protocol at ICU admission the survival rate in the red (93.7%) and yellow (62.5%) categories were significantly higher then that of the blue category (24.6%) with associated P value of less than 0.0001 and 0.0003, respectively. Further, the survival rate of the red group was significantly higher than the overall survival rate of 70.9% observed in the cohort (P < 0.0001). At 48 hours the survival rates in the red (90.5%) and yellow (94.1%) categories remain significantly higher than that of the blue category (45.4%) with P values of 0.002 and 0.001; however, beyond 48 hours the number of patients remaining in the ICU had decreased substantially, limiting inferences from these analyses.

Table 3 Cumulative resource utilization

Table 4 Survival rate by triage category

To examine the outcomes of those patients who would have been triaged to the blue category yet survived in the observation cohort, the outcomes and criteria for those ever triaged to blue category (Table 5) are compared with outcomes for those never triaged to blue category (Table 6). Of the patients triaged to blue, 32.3% survived to hospital discharge. Almost half of the survivors triaged to the blue category had been triaged blue for failing to sufficiently improve their SOFA score at either 48 or 120 hours. The most common exclusion criterion triggering a triage to category blue in those patients who survived to hospital discharge was the presence of metastatic cancer.

Table 5 Analysis of patients who were ever triaged blue or met exclusion criteria

Table 6 Analysis of survivors who met inclusion criteria but never triaged blue or excluded

Following the triage exercise, triage officers were polled to assess if they believed they were the 'type' of person who could make triage decisions; if they would volunteer to be a triage officer in a pandemic; and their view regarding the training required to be a triage officer. All except triage officer 3 said they believed they were the type of person who could make triage decisions and would volunteer in a pandemic. All stated that specific comprehensive training for intensivists to be triage officers is required.

Discussion

This study provides insight into the use of triage protocols during pandemic periods, and informs the design for a larger multi-center prospective study to evaluate the Ontario triage protocol. Our results highlight the need to develop a selection process for triage officers and to provide comprehensive training for triage officers. Many cases required arbitration due to disagreement between triage officers 3 and 4. The arbitrator ruled in agreement with triage officer 4 on 95.7% of the decisions.

The overall degree of confidence in triage decisions fell below the a priori target of greater than 90% of decisions being rated as either confident or very confident. Additionally, the rate of arbitration was high, although primarily the result of decisions by one triage officer. A lack of consistency between triage officers threatens the equity of the process. The clinical information provided about patients was much less then intensivists would typically receive in clinical practice, and triage officers reported that the data were insufficient to establish a clear clinical picture in 20.5% of cases (data not shown). Thus, more specific clinical information will have to be provided in a future triaging study, which may be available during prospective application of the protocol, but increasing dependency on information can limit the usability of the protocol. Other possible reasons for lower than anticipated confidence among triage officers and high rate of arbitration is inexperience with the protocol, inadequate training, and the lack of prior studies exploring the impact of triage protocols such as this. Future qualitative research could better illuminate factors that contribute to lack of confidence in triage decision-making using a protocol such as the one studied.

Although not specifically designed or powered to evaluate the impacts of the triage protocol on patient outcome, this study provides some insight into its performance. The primary goal of a tertiary triage protocol is to direct the limited available resources to those who are most likely to benefit from them. On this point, the protocol appears to serve its function with those triaged to the highest priority for ICU care (red) having survival rates significantly higher than the rate in the observation cohort and markedly higher than those triaged blue. The alternative to the use of a protocol would be leaving individual physicians to make allocation decisions on their own. Prior research suggests that such ICU physicians' ability to predict patient outcome without the aid of a decision support tool, such as a triage protocol, is poorer than we have observed in this study [18]. Additionally, failure to use a standard triage protocol to guide decisions regarding the allocation of scarce resources is less efficient and ethically less desirable [19–21].

A second goal of tertiary triage is to make more critical care resources available. On this point, the protocol shows promise. Through the application of the protocol's inclusion criteria, exclusion criteria, MQS and discharge criteria, the demand for ventilators could be decreased by 49.3% and for ICU admission by 52.6% compared with standard practices. As an illustrative example, based upon the 568 days of ventilation made available through the use of the protocol, using rates from the first wave of H1N1 in Canada assuming an average of 10 days of ventilation and an 89% survival rate, 50 lives could potentially be saved by the resources made available.

Ethical frameworks suggest that restrictions placed on the allocation of scarce resources must be proportionate to the expected and observed shortfalls [22]. Thus, it is particularly important to minimize the number of people who are triaged blue but may possibly survive under normal circumstances. Evaluating this outcome is somewhat difficult in this study given that patients were receiving what is essentially optimum care in tertiary ICUs, whereas during a pandemic, triage would be instituted only when emergency mass critical care is being used which requires significantly modified standards of care. Thus, one would expect the mortality rate in the sickest of patients (those excluded or triaged blue) would be higher than seen in this observation cohort. Further, survival in this study was defined as status at hospital discharge, whereas the triage protocol was designed to direct resources to patients with greater than 50% two-year survival. Many of the patients who survived to hospital discharge but would have been triaged blue or excluded, such as those with metastatic malignancies, are unlikely to meet the higher bar of greater than 50% two-year survival. Patients are often repatriated to the facilities from which they were transferred once they no longer require services only available at tertiary centres. As we were unable to collect information from the referring facilities in this study, some of these patients may not have survived to discharge from the referring facility.

This study is limited in that it is a pilot study of a relatively small cohort of critically ill patients from two Ontario hospitals. Although clinical data were presented to triage officers prospectively, data were less than would typically be available in practice, and some data were missing. Our results highlight the need for a central triage committee, comprehensive training, and data management infrastructure so the committee can monitor triage outcomes during a pandemic, adjusting the protocol to correct for either over-triage or under-triage [15, 23–25]. Further, the exclusion criteria and prioritization criteria should be incorporated into triage decisions on a graduated basis depending upon the anticipated shortfall in resource supply. Additionally, the protocol exclusion, MQS and prioritization criteria require further study and modification to minimize the potential for denying resources to those who may benefit from them.

Conclusions

We evaluated the Ontario critical care triage protocol in this pilot study, which generated insights about future triage practices. Although further research is needed, our results suggest that the triage protocol can help to direct resources to patients who are most likely to benefit from them, and help to decrease the demands on critical care resources, thereby making available more resources to treat other critically ill patients. We also documented the need for more comprehensive patient summaries when such decisions are being made, careful triage officer selection, improved triage officer training, and infrastructure to allow timely tracking and analysis of the consequences of triage protocols.

Key messages

• Attention must be paid to the appropriate training and selection of triage officers in to improve confidence in their decisions and agreement between triage officers.

• Application of the triage protocol can potentially free up significant critical care resources, which may be re-directed towards managing critically ill patients during a pandemic or disaster.

• The current protocol is able to identify patients who are most likely to survive and allow resources to be targeted to this group. However, the protocol does require further investigation and modification to minimize the number of patients who would potentially be excluded critical care in a pandemic but whom may survive if they were to receive critical care.