Clinical review: Checklists - translating evidence into practice

Checklists are common cognitive tools that can help complete a task as simple as shopping or as complex as flying a Boeing 737. The complexity and the time required for completing tasks and the context in which the work occurs vary widely. Yet all tasks are prone to human error given the natural limitations of our memory and attention span and our ability to cope with stress, fatigue, illness, interruptions, new situations, and production pressures. Intensive care units (ICUs) are complex and time-sensitive. On any given day in the ICU, the typical patient will require 178 interactions in their care [1]. A checklist standardizes the process to ensure that all elements or actions are addressed. The structure and predictability of checklists facilitate the careful and systematic delivery of care, which reduces variability and improves performance. The science of developing checklists in health care is new. In an informal review of the literature, we did not find any standardized methodology to develop and design checklists in medicine [2]. Based on a review of checklist development in aviation and our own research and experience, we propose a process for creating checklists. If medical checklist elements are wise and based on best practices, they can help translate evidence into practice.

One great advantage of a checklist is the ability to democratize knowledge. Medicine, like many fields, is infused with jargon; physicians, nurses, and patients have distinct ways of describing the same thing. As such, translation errors are imminent when one member of the care team talks to another. Given that each caregiver has a unique perspective and professional culture and that clinical disciplines train separately, it is understandable why miscommunication is common and a major contributor of medical errors [3–6]. Checklists reduce these risks. They standardize and improve the reliable translation of information so the same knowledge is available to doctors, nurses, and patients.

Although checklists have tremendous potential to improve safety and quality and reduce the costs of health care, they are underused. Because we lack a clear definition for a medical checklist, it is difficult to know how widely they are used in health care. To be certain, other efforts to standardize work and create independent checks, such as protocols, critical pathways, and order sets, exist in health care. Yet they differ in significant ways from the quality/safety checklists we describe. Order sets are pre-printed instructions that attempt to include everything that a clinician may want to consider ordering for a particular group of patients (for example, those on anticoagulation therapy). Many things on this order set will not be checked. Conversely, quality/safety checklists include only the things that should be addressed. Clinical/critical pathways are time-continuum tools that support clinical decision-making and function primarily to improve work flow for a specific diagnosis. Pathways have nodes with drop-down checklists of tasks along their continuum. However, many pathways do not include explicit details regarding patient-level interventions. Rather, they generally present expected milestones along a patient care trajectory. Nevertheless, both order sets and pathways can be viewed as a type of checklist.

The underuse of quality/safety checklists is partly due to the paucity of scholarly research to identify where to use checklists, how to build and implement them and assess their effectiveness at improving patient outcomes, and whether or how checklist use is sustained over time [2, 7]. In this review, we explore some of the principles and possible strategies to develop and encourage the implementation of checklists in medical practice.

There are four principal types of checklists: static parallel, static sequential with verification, static sequential with verification and confirmation, and dynamic [8]. The differences among these checklists are the number of operators involved and the extent to which the correct action is verified. Static parallel checklists are completed by one operator (person) and executed as a series of read-and-do tasks. The anesthesia machine checklist (Figure 1) is an example of this format [9].

The static sequential with verification checklist involves a challenge and response. An operator reads a series of items and a second person verifies that each item has been completed or is within parameters. A checklist to ensure appropriate insertion of central lines is an example of this format [10, 11]. As the physician performs the insertion, a nurse challenges the completion of each behavior/task and the physician responds to confirm whether the behavior/task has been accomplished.

Static sequential with verification and confirmation checklists are used more often in team-based settings where a set of items/tasks are done by varying team members. A designated person reads the items (challenge) and each responsible party verifies the completion of their specific task. Time-outs and briefings in the operating room (OR) use this format [12]. For example, the surgeon will state the patient's name, procedure, and surgical site and ask about availability of equipment (to which the nurse would confirm the information) and then ask about the patient's medical condition and availability of blood (to which the anesthesiologist would respond).

Finally, a dynamic checklist uses a flowchart to guide complex decision-making. Typically, there are multiple options to choose from and the care team must decide the optimal course (an algorithm). An example of a dynamic checklist is the American Society of Anesthesiologists difficult airway algorithm [13], which provides guidance on the intubation of a difficult airway. The team leader would use this algorithm checklist to develop a plan and then discuss it with other members of the team.

Checklists may be further categorized by their application to high-risk or normal operations. High-risk checklists are useful back-up plans to fix or mitigate harm when single or multiple failures of redundant systems occur [14]. High-risk checklists can be used during a crisis to prevent further mistakes and to ensure reliable communication and operations. For example, their use during disaster relief efforts would standardize tasks and processes, thereby organizing different groups. Pre-operative preparation of patients is an example of a normal operations checklist.

The structure of a checklist may or may not be important, depending on the context in which they are used. For some checklists, the sequential execution of each item is critical. For others, the focus is solely on executing or considering the items. When sequential action is required, an independent check is crucial to ensure that each task is completed prior to executing the next task. A static checklist would be risky here, particularly in high-risk situations given our natural cognitive and stress-induced limitations [15].

Failure to properly follow checklists (sequential or not) can have disastrous consequences [16]. For example, if the proper checklist procedure for identifying and verifying a patient and their blood type is not followed, the unit of blood transfused may be incompatible and lead to a potentially lethal hemolytic transfusion reaction.

In medicine, physicians have largely resisted using checklists. Some feel that relying on a checklist insults their intelligence, whereas others doubt that a document with check boxes will prevent a medical mistake [1]. Physicians believe they know their job and do not need a prompter to guide or remind them [17]. However, modern medicine has become exceedingly complex, specialized, and interdisciplinary, offering hope for fantastic cures, but also inadvertently introducing potentially devastating risks.

Well-designed checklists standardize what, when, how, and by whom interventions are done and can reduce errors in routine and emergency situations. In addition, they provide a public framework to ensure adherence to clinical or procedural requirements. The shared knowledge of checklist content also allows caregivers to mutually support each other by cross-checking what is being done and in what order. These assurances are important when time is short, the pressure is on, and competing priorities distract our attention.

Checklists have been implemented in isolated clinical settings to improve processes of care [11, 18–25] and in the ICU to facilitate bedside teaching and assessment of resident performance [26]. Checklists can be implemented as a stand-alone intervention [22], but in most cases they are part of an intervention bundle with several other components to improve quality and safety of care [11, 27]. Studies to improve care using checklists have reduced uncertainty over the correct surgical site [20] and problems with laparoscopic equipment in the OR [21], clarified [22] or changed patient care plans [23] to mitigate or prevent medical errors, dramatically reduced central line-associated bloodstream infections (CLABSIs) [11], and helped diagnose communication deficiencies and depression in adults with intellectual disabilities [24, 25]. Despite the corpus of evidence regarding the benefits of checklists, medicine remains slow in broadly adopting them into practice.

Seminal work in the psychology of memory found that we can retrieve seven (plus or minus two) pieces of information from our memory with relative accuracy [28]. When complex procedures, stress, and fatigue are introduced, our memory becomes increasingly unreliable [29]. And as the number of tasks/problems we simultaneously manage exceeds three, we show significant decline in the accuracy and speed of handling these tasks/problems [30]. A checklist can compensate for these fallibilities. The mandatory use of checklists can empower nurses and junior-level physicians to insist that those higher in the traditional hierarchy adhere to approved and safe procedures.

Although there is no published evidence indicating a negative impact by using checklists, they could pose risks. Any time that we change the system to improve safety, we may defend against some risks but invariably will introduce new risks. Checklists are not immune to this tendency. Poorly designed checklists or excessive use of checklists could overburden clinicians, complicate tasks, and reduce efficiency. If emerging evidence is not incorporated into checklists, they could hinder patients from getting state-of-the-art care. Another potentially negative effect could occur if clinicians strictly adhere to a checklist rather than exercise critical thinking when evidence is incomplete, when an individual patient's risk-benefit analysis favors not using the checklist, or when an unforeseen event that requires different interventions occurs. A recent example of this last point is the pilot who landed the Airbus A320 plane on the Hudson River without fatalities. Hence, it is imperative to be cautious when deciding when to use checklists and to be mindful of potential negative effects.

The literature review of medical checklists by Hales and colleagues [2] found few strategies to develop checklists and no standardized methodologies. Ideally, the process would incorporate diverse input (for example, empiric evidence, tacit experience, local input, regulatory input, and community input). Unfortunately, the process is often delegated to committees with homogeneous members, who may lack the experience and knowledge to explore appropriate items for checklists and the potential risks of these items. Checklists with elements that pose risks or those that exclude important elements may be neither effective nor efficient at improving patient care. While we need investment in the 'basic science' of checklist development, the human factors engineering literature describes how to design information tools, which can be applied in quality and safety. Briefly, the recommended steps to develop checklists include the following: review the existing literature, understand the needs and work-place of the users, include a multidisciplinary group in the design, and use an iterative approach for rigorous pilot testing and validation of your tool [31–33]. We recently published an approach to develop checklists which adds simulation to the above steps [34]. Below, we provide more detailed guidance to develop checklists.

First, we must pick a patient population, procedure, and/or outcome. For example, we sought to reduce infections from central lines in ICU patients [10]. This could include checklists for diagnosing, treating, or monitoring patients. Then, we need to assemble an interdisciplinary team to lead the development. This team should review and identify empiric evidence from published literature and tacit evidence from experience to decide on the content of the checklist (Figure 2).

In identifying evidence, the team could start by looking for evidence summaries (for example, systematic reviews or practice guidelines). For most items, however, empiric evidence will be either absent or incomplete. As such, teams should tap into the 'wisdom of crowds' by getting broad input regarding checklist items from diverse sources [35]. In addition to literature database sources such as PubMed, Embase, and Cochrane reviews, teams should consider knowledge banks, expert and social networking, and focus groups with frontline practitioners. Once a list of potential interventions is compiled, the team should consider those with the strongest impact and the lowest barriers to use in clinical practice. Because role and/or task ambiguity is associated with failure to use the checklist as intended [36], each intervention should be translated into an explicit, concise, and unambiguous behavior. This culling process is critical to make the checklist cognitively and logistically functional. In the case of health care-associated pneumonia (HCAP), there are over a hundred potential steps to prevent this complication, with multiple tasks for each step. Without discipline and data reduction, developing an HCAP prevention or similar checklist could become unwieldy.

In our experience, long unorganized checklists are often not useful or used. As previously mentioned, our ability to manage information deteriorates as the number of items increases [28, 30, 37]. Memory checklists should especially adhere to this principle. If a checklist requires a large number of items, it is better to separate the process into substeps and create a checklist for each substep. In the OR, for example, there are critical steps that could be broken down into individual checklists, such as preventing wrong-sided surgery and surgical site infection. Each of these is part of the pre-induction/incision process but involves distinct tasks and behaviors. The creation process should be iterative and undergo as many revisions as necessary to achieve consensus on the checklist before attempting any pilot testing.

The checklist should provide unambiguous guidance on what, when, how, and who should do a particular intervention and should be logistically efficient and easily performed. The aviation industry offers guidance. Figure 3 shows a standard aviation procedure checklist for landing a Boeing 737-800 aircraft. This checklist uses the challenge/response format with simple, direct, and unambiguous language. Health care checklists should seek to emulate this model. The anesthesia machine static 'read-and-do' checklist (Figure 1) [9] is a close example of the aviation model with concise questions.

Once a beta-version of a checklist is completed, it should be pilot-tested by potential users and revised according to their findings. This testing could be done on the clinical units where it will be used and/or in a simulation setting. Usability and potential risks of a checklist can be assessed by (a) testing different real-life scenarios in a simulated environment, (b) heuristic evaluation by a human factors/usability testing expert, and (c) subjective evaluation by potential users through interviews, focus groups, and surveys [38]. Validity, reliability, and potential harms of a checklist should be assessed before it is broadly implemented. Face validity can be accomplished by asking potential users to review the tool for relevance and completeness and to suggest how it can be improved. Theoretical saturation (collecting data from additional people until no new information emergences) [39] can be used to determine the number of people needed to pilot-test a checklist. Reliability of a checklist can be assessed using inter-rater and intra-rater reliability methods.

Finally, checklists must remain wise. To accomplish this, checklists must be dynamic and evolve, and empiric and tacit evidence must be continually evaluated to support each checklist item and to explore unintended consequences that are the norm rather than the exception. If this is accomplished, checklists can be efficient and effective.

First, we will consider several widely used checklists in health care. The anesthesia machine checklist is universally accepted by anesthesiologists [40, 41] and, over the course of several decades of use, has undergone modifications to keep pace with new technology. This checklist helps to ensure that every practitioner's equipment has undergone the same standardized assessment (although machines of individual manufacturers may vary somewhat) before transporting the patient to the OR. Like many aviation checklists, this one focuses on a particular piece of equipment. One practitioner completes the checklist and the effectiveness depends on strict adherence to its completion [16].

We developed the catheter insertion checklist to address the problem of CLABSIs in the ICU [10]. It was informed by a detailed practice guideline developed by the Centers for Disease Control and Prevention (Atlanta, GA, USA) [42]. From this guideline, the team selected five practices that had the strongest evidence and lowest barriers to use. Each of the interventions was worded as a behavior, and we pilot-tested the checklist in our ICUs at the Johns Hopkins Hospital. This checklist uses the static sequential with verification format. The 'challenger' (nurse) is empowered to prevent the procedure from proceeding until the physician inserting the catheter confirms that the checklist item was completed. This empowerment of the nurse to halt the process makes the checklist particularly robust. Nevertheless, it requires a culture in which nurses are comfortable questioning physicians and in which physicians are willing to accept and modify their behavior. The successful use of interdisciplinary checklists such as this one often requires efforts to improve the culture of safety [43].

Another checklist we developed was a daily goals sheet (Figure 4) [22]. This checklist was built after residents and nurses on the ICU health care team stated that communication errors posed substantial risks to patients. It clarifies the patient's daily care plan, helps ensure delivery of evidence-based interventions, and plans for potential safety risks. It follows a static sequential with verification format. At the end of rounds on a patient, a member of the team reads off the items and team members (including the nurse) respond with the status of those items and discuss the patient's goals. Items that need to be changed, deleted, or added become the tasks for the day. The checklist is completed for every patient to ensure uniformity of care delivery and to ensure that all care team members understand the patient's status and care plan prior to moving to the next patient.

The daily goals checklist is fairly complex and has many items. To reduce the burden on clinicians, items are grouped into categories consistent with how ICU clinicians think. Each category contains a limited number of items. It was decided to keep this list intact since the checklist is kept by the bedside and revisited throughout the day.

In addition to addressing specific problems and goals, checklists may be applied to the whole spectrum of the care process. While acknowledging the complexity of this undertaking, the process of medicine can be divided into three major phases: diagnosing, treating, and monitoring. Each phase can be broken down into three elements: decision (whether and what to do), execution (carrying the process out), and interpretation (what is the result and what does it mean). Medical errors may occur anywhere along this hypothetical spectrum of patient care. As such, checklists can help improve care anywhere along this spectrum. For example, we are developing checklists (from the patient's perspective) for diagnosing, treating, and monitoring lung cancer. The use of these checklists should help to reduce variability and errors in patient care.

Ventilator-associated pneumonia (VAP) is a useful model to examine how checklists can improve a spectrum of care. The diagnostic criteria and the recommended therapies for preventing and treating VAP are well established. A section on our daily goals form includes a checklist of evidence-based prevention interventions. The weaning item on the checklist prompts clinicians to monitor and wean the patient as soon as safely possible. This is followed by a separate checklist to evaluate the patient's readiness for extubation. This approach has shortened the average duration of mechanical ventilation and reduced ventilator days, failed extubations, and VAP rates [44].

The diagnosis phase checklist would cognitively prompt us to recognize the possibility of a VAP and then execute the entire process to interpret whether the patient meets the criteria for VAP. It would also assist in deciding whether to treat the patient. A treatment phase checklist would include therapies based on evidence-based medicine guidelines, local knowledge of probable pathogens, and anti-microbial sensitivity patterns. In addition, there may be a policy or structural-level checklist for preventing VAP (such as not routinely changing ventilator circuits). A monitoring checklist can help to determine whether the patient requires continued ICU care, monitoring of oxygen saturation, or additional respiratory care.

This seems like a lot of checklists. Yet each step in the diagnosis, treatment, and monitoring process poses risks for error that we need to defend against. We do not know how many checklists are too many, when they are most useful, and when we have overloaded the checklist users. However, we do know that checklists should be short and simple, and the benefits demonstrated rather than assumed. In aviation, processes are broken down into manageable segments, each with an assigned checklist. This model has been successful in preventing errors. Although there may be some redundancy from this design, redundancy is a crucial element of safe and reliable systems [45].

One final example of a medical checklist that underscores this redundancy is decision support tools built into computerized provider order entry (CPOE) systems. While implementation of CPOE may have difficulties and unintended consequences [46, 47], smooth and effective implementation of these systems can be enhanced by building a large number of order sets (checklists) for therapies and management. The more order sets created, the more efficient and effective the CPOE system [48, 49]. Information technology will likely be the best mechanism to manage a large number of checklists. As such, we look forward to the day when our checklists are all automated and the required checklist can be selected electronically.

While the science of checklists, much like the science of safety and quality, is immature, many believe that medical checklists can help prevent errors, mitigate harm, and reduce the costs associated with them. Yet using the process we describe to develop checklists for the multitude of diagnoses and procedures is a daunting and likely slow process. When we reflect on the underlying problem in translating evidence into practice, like many health care quality and safety shortcomings, it is clearly impaired by an inefficient knowledge market, much like the subprime credit crisis. Lack of diverse input and timely access to such information contributes to this inefficiency.

The delay between knowledge creation, dissemination, and translation into practice is significant - sometimes taking years [50, 51]. Clearly, there are many elements to this process which must be addressed. Nevertheless, more efficient knowledge markets that quickly synthesize diverse input into usable tools and make these tools widely accessible could substantially improve some of the delays in knowledge translation.

A strategy to improve the efficiency and effectiveness of the health care knowledge market would be to create broad learning communities to continually create and update checklists. Clinicians organized around a particular diagnosis or procedure can create checklists for a particular disease or process. These checklists would be informed by empiric and tacit evidence and revisited often to incorporate new evidence. If we believe in the 'wisdom of crowds', in which the broader community has tremendous knowledge and collectively will get it right, then this learning community can effectively codify this wisdom into checklists that can improve patient care.

Checklists are powerful tools to standardize work processes and create independent checks for key processes. Although they can have wide application in medicine, they are relatively underused. Checklists could create a more efficient and effective knowledge market by summarizing evidence into explicit behaviors, incorporating empiric and tacit evidence, and being continually updated by the health care community.

Further research is needed to advance the science for developing, implementing, and evaluating checklists. Although their format and content may vary, simple steps to identify, check, and verify what you have done or are about to do can determine whether you succeed or fail. They should be succinct, unambiguous, focused, and ultimately effective and efficient. When faced with a crisis, we can react quickly and decisively, knowing that the items we act out from the checklist are well thought out, tested, and will provide us with the results we want. We hope the science advances to help us build, refine, and use checklists wisely.