Protocol-directed care in the ICU: making a future generation of intensivists less knowledgeable?

Expanded abstract

Citation

Prasad M, Holmboe ES, Lipner RS, Hess BJ, Christie JD, Bellamy SL, Rubenfeld GD, Kahn JM. Clinical Protocols and Trainee Knowledge About Mechanical Ventilation. JAMA. 2011; 306(9):935-941. PubMed PMID: 21900133 This is available on http://www.pubmed.gov

Background

Clinical protocols are associated with improved patient outcomes; however, they may negatively affect medical education by removing trainees from clinical decision making.

Methods

Objective: To study the relationship between critical care training with mechanical ventilation protocols and subsequent knowledge about ventilator management.

Design: A retrospective cohort equivalence study linking a national survey of mechanical ventilation protocol availability with knowledge about mechanical ventilation. Exposure to protocols was defined as high intensity if an intensive care unit had 2 or more protocols for at least 3 years and as low intensity if 0 or 1 protocol.

Setting: Accredited US pulmonary and critical care fellowship programs.

Subjects: First-time examinees of the American Board of Internal Medicine (ABIM) Critical Care Medicine Certification Examination in 2008 and 2009.

Intervention: N/A

Outcomes: Knowledge, measured by performance on examination questions specific to mechanical ventilation management, calculated as a mechanical ventilation score using item response theory. The score is standardized to a mean (SD) of 500 (100), and a clinically important difference is defined as 25. Variables included in adjusted analyses were birth country, residency training country, and overall first-attempt score on the ABIM Internal Medicine Certification Examination.

Results

The 90 of 129 programs (70%) responded to the survey. Seventy seven programs (86%) had protocols for ventilation liberation, 66 (73%) for sedation management, and 54 (60%) for lung-protective ventilation at the time of the survey. Eighty eight (98%) of these programs had trainees who completed the ABIM Critical Care Medicine Certification Examination, totaling 553 examinees. Of these 88 programs, 27 (31%) had 0 protocols, 19 (22%) had 1 protocol, 24 (27%) had 2 protocols, and 18 (20%) had 3 protocols for at least 3 years. 42 programs (48%) were classified as high intensity and 46 (52%) as low intensity, with 304 trainees (55%) and 249 trainees (45%), respectively. In bi-variable analysis, no difference in mean scores was observed in high-intensity (497; 95% CI, 486-507) vs low-intensity programs (497; 95% CI, 485-509). Mean difference was 0 (95% CI, -16 to 16), with a positive value indicating a higher score in the high-intensity group. In multivariable analyses, no association of training was observed in a high-intensity program with mechanical ventilation score (adjusted mean difference, -5.36; 95% CI, -20.7 to 10.0).

Conclusions

Among first-time ABIM Critical Care Medicine Certification Examination examinees, training in a high-intensity ventilator protocol environment compared with a low-intensity environment was not associated with worse performance on examination questions about mechanical ventilation management.

Commentary

Evidence-based protocol-directed care has permeated the practice of mechanical ventilation, especially with respect to ventilation liberation [1], lung-protective strategies for acute lung injury (ALI) [2] and sedation management [3]. These protocols have been endorsed by multiple organizations and societies including the American College of Chest Physicians, American College of Critical Care Medicine and the Surviving Sepsis Campaign. However, there remains a necessary tension between patient care and experiential learning [4]. The institution of protocol-directed care in the intensive care unit (ICU) may be detrimental in that it potentially distances physicians from direct patient care and may disallow trainees from thinking beyond an algorithm. Conversely, protocols provide a summary of best practices and minimize practice variation. The above study by Prasad et al. aimed to elucidate the effects of high vs. low-intensity protocol-directed care on trainee knowledge about mechanical ventilation.

In this study, 88 accredited U.S. pulmonary and critical care medicine training programs were analyzed. Protocols studied included those for sedation management, mechanical ventilation liberation and lung-protective strategies in ALI. Of these programs, 42 (42%) were considered to be high-intensity (i.e., 2-3 protocols for at least 3 years) and 46 (48%) were low-intensity (i.e., 0-1 protocol for at least 3 years). Program characteristics were similar between the 2 groups, with 89% being in the university setting. Trainee characteristics were also similar with 64% of trainees receiving training in the U.S. or Canada. In a bivariable analysis comparing mechanical ventilation exam scores to protocol intensity, results were nearly identical (High-intensity - mean 497; 95% CI, 486-507. Low-intensity - mean 497; 95% CI, 485-509. Mean difference, 0; 95% CI, -16 to 16). In a multivariable analysis adjusting for potential confounders determined a priori (birth country, training country), there was no statistical difference between the two groups (adjusted mean difference, -5.36; 95% CI, -20.7 to 10.0).

Strengths of the study include the fact that mechanical ventilation is fairly ubiquitous in all ICUs and training programs [5] and quality of training is likely to have a significant effect on outcomes. One weakness is that the authors were unable to determine by their survey whether or not protocol availability was equal to protocol utilization. Additionally, there may be a large amount of protocol variation. For example, programs that were considered to have a sedation titration protocol did not necessarily have to include a protocol for daily sedation interruption [6]. The participating institutions were also largely in the university setting and it is unknown whether or not these results can be generalized to the community setting. Furthermore, the study assessed performance on a standardized exam rather than overall clinical competency, although the authors showed no difference between examinees in overall residency program director rating.

The results of this study contribute to the ongoing debate regarding the use of protocols in the ICU setting. Despite the results of this study, there have been ongoing concerns that protocol-directed care may promote disinterest, stifle learning, and remove the physician from direct patient care [7]. A potential approach to overcome this issue is that educators should periodically question trainees on different aspects of the protocol. For example, during bedside rounds, educators could ensure that trainees understand the protocol, determine reasons for failure in an individual patient, and whether protocol was performed under optimal conditions. Rather than simply saying, "This patient has failed his spontaneous breathing trial" and moving on, understanding why the patient failed the weaning trial and whether interventions can be instituted to mitigate some of these factors then becomes important. Thus, while mechanical ventilation protocols have been shown to improve outcomes [1–3], they cannot replace clinical judgment. They do not, however, necessarily stifle clinical judgment.

Recommendation

While protocol-directed care remains a source of debate, this study shows that training in a high-intensity mechanical ventilation protocol environment was not associated with worse performance on certification exam mechanical ventilation questions. Further studies are needed to examine whether protocol-directed care or other interventions targeted for patient safety affect competency of trainees.