What do we know about medication errors made via a CPOE system versus those made via handwritten orders?

Abstract

This commentary on the article by Shulman et al. examines what we understand by 'medication errors', what we mean by 'computerized physician order entry (CPOE) systems', how we measure errors, and what types of errors we are 'reducing' with CPOE systems. As the research of Shulman and colleagues highlights, much of the existing research on CPOE systems does not differentiate among: types of medication errors; consequential versus inconsequential medication errors; CPOE systems that include/exclude formal decision support packages; and the extent to which decision support information is implicitly presented to physicians via the CPOE system, for example, pull down menus with dosages. I discuss these issues and their implications for the evaluation of CPOE systems and of other emerging healthcare technologies.

Shulman and colleagues [1] have contributed a thoughtful study on medication orders at an intensive care unit that shifted from handwritten orders to a computerized physician order entry (CPOE) system. They examine whether errors were intercepted or not, and the frequency, severity, and types of those errors. They explore the role of the CPOE system in preventing and perhaps facilitating errors.

Their findings are complex. When they combined intercepted and non-intercepted medication errors (potential and actual errors), the CPOE system was associated with fewer errors, a finding they repeatedly stress. When they examined major medication errors, however, or even moderate errors that were not intercepted by the pharmacists, their data show that all of these more serious errors occurred only via the CPOE system.

They stress the need to consider differences in types of errors made with CPOE systems compared to handwritten orders. I find in Shulman et al.'s article essential questions that are too often glossed over or assumed to have obvious answers. Their work obliges us to re-examine our understanding of 'medication errors', 'CPOE', how we measure errors, and what types of errors we are 'reducing' with CPOE systems. I find five lessons in their work.

The complexity of medication prescribing error

Shulman and colleagues assign medication errors into a 12 category schema that illuminates the many types of medication prescribing errors and, key here, how these errors vary according to the type of ordering system used. For example, according to their study, errors of 'dose/units/frequency omitted on prescription' and putting orders on 'incorrect drug chart section' are far more prevalent with handwritten orders than they are with CPOE orders. But 'wrong drug prescribed', 'dose errors' and 'required drug not prescribed' are more likely to occur with the CPOE system. Dose errors, in fact, were almost twice as likely to be made with the CPOE system; and all of the errors involving 'required drug not prescribed' occurred under the CPOE system.

Beyond the comparison of paper versus computer, Shulman et al.'s taxonomy of errors shows us that the more careful we are in examining the types of errors occurring, the more clear it is we are often lumping together different problems in ways that are neither intellectually nor clinically satisfying.

The definitions of medication prescribing errors are critical when we measure the role of CPOE systems in preventing errors

The statement that the definitions of medication prescribing errors are critical when we measure the role of CPOE systems in preventing errors remains valid even if we don't categorize the types of errors and even though we benefit from well-accepted error severity scales [2]. If we use pharmacist interventions in determining errors, we are measuring possible/potential errors. If we examine patients' charts, we may see both prevented and administered errors. (There are undoubtedly other, undetected errors.) Berger and Kichak [3] make the critical point that studies of prescribing errors overwhelmingly count errors that do not affect patients. We almost always count potential errors, not actual adverse drug events; and even then, we usually find the inconsequential errors.

When Berger and Kichak [3] analyzed studies by Bates et al. [4, 5] and focused on consequential errors, they found "the reality is that no significant decrease in patient morbidity and mortality occurred as a result of the institution of CPOE" [3]. The oft-noted 84% to 55% decrease in errors when using CPOE [4, 5] drops to a statistically insignificant 17% when examining consequential errors. As Bates and colleagues [2] write with intentional irony, "...it seems easiest to prevent those [errors] that rarely cause injury."

Thus, we must consider that among CPOE systems' many virtues is their ability to reduce errors that seldom reach patients (which neither negates their many valuable contributions nor precludes their extraordinary promise).

Delineating the purview of a CPOE system is seldom clear

Every time Shulman et al. [1] describe their CPOE system, they add that it is "without a decision support system" (DSS). And while that is true, it is perhaps also too facile. As they note, their system does not offer DSS-type warnings about drug-drug interactions, allergies, or toxic doses; however, it does have pull down menus indicating dosing and route, a feature that influences physicians' decisions. That is, CPOE systems have implicit decision support even though it may not be understood as such by CPOE designers or by physicians. Also, the CPOE Shulman et al. [1] examined included an available (but not interactive) on-line information system with drug interactions, contraindications, side effects, formulary, and IV administration guide.

Thus, the demarcation between CPOE systems and forms of decision support, which might reduce or influence errors, is seldom the bright line we imagine. When added to the reality that many studies claiming to be of CPOE systems are actually studies of CPOE and DSS, the waters get even more muddied.

Shulman et al. posit a direct link between the most serious medication errors and the use of their CPOE system

Shulman et al. [1] detail, for example, how their CPOE system's pull down menu for dosages led to prescribing an injection of 7 mg/kg instead of 7 mg of diamorphine. They speculate that their CPOE's connection to serious errors is a "result of physicians choosing the wrong drug template, selecting from multiple options, or as a consequence of constructing their own drug prescriptions using pull down menus."

They offer more severe warnings than Koppel et al. [6]. Shulman et al. [1] write, "As clinicians embrace CPOE, they should not make the assumption that CPOE removes errors; in fact different types of errors emerge."

Evaluation of CPOE systems, and of all healthcare information technology, is mostly terra incognita

This research reminds us that while CPOEs undoubtedly reduce several forms of medication error, measuring such reductions requires us to address the multifaceted reality of error cause, error type, error certainty, error severity and, indeed, the ability to determine that an error occurred. Moreover, because error reduction is far from the only benefit we anticipate from CPOEs (e.g., they also confer speedy links to pharmacies) we presumably will seek to measure all of these benefits and costs with some precision. But comprehensive data or even a consensus methodology are still forthcoming.

In summary, Shulman et al. [1] provide insights about the consequences of CPOE systems. Their analysis offers an uncommon balance; addressing both the benefits and dangers of CPOEs, and highlighting differences in the types of errors prevented and perhaps enhanced through their use. They provide a nuanced understanding of how CPOE systems affect medication errors and we gain useful insights into how we might evaluate all emerging healthcare technologies.