Open Access

Quality in quality improvement research - a new benchmark

Critical Care201115:316

Published: 9 December 2011

Expanded abstract


Scales DC, Dainty K, Hales B, Pinto R, Fowler RA, Adhikari NK, Zwarenstein M: A multifaceted intervention for quality improvement in a network of intensive care units: a cluster randomized trial. JAMA 2011, 305:363-72.


Evidence-based practices improve intensive care unit (ICU) outcomes, but eligible patients may not receive them. Community hospitals treat most critically ill patients but may have few resources to devote to quality improvement.


To determine the effectiveness of a multicenter quality improvement program to increase delivery of 6 evidence-based ICU practices.

Design, setting, and participants

Pragmatic, cluster-randomized trial among 15 community hospital ICUs in Ontario, Canada. A total of 9269 admissions occurred during the trial (November 2005 to October 2006) and 7141 admissions during a decay-monitoring period (December 2006 to August 2007).


The authors implemented a videoconference-based forum including audit and feedback, expert-led educational sessions, and dissemination of algorithms to sequentially improve delivery of 6 practices. The ICUs were randomized into 2 groups. Each group received this intervention, targeting a new practice every 4 months, while acting as control for the other group, in which a different practice was targeted in the same period.

Main outcomes

The primary outcome was the summary ratio of odds ratios (ORs) for improvement in adoption (determined by daily data collection) of all 6 practices during the trial in intervention vs control ICUs.


Overall, adoption of the targeted practices was greater in intervention ICUs than in controls (summary ratio of ORs, 2.79; 95% confidence interval [CI], 1.00-7.74). Improved delivery in intervention ICUs was greatest for semi recumbent positioning to prevent ventilator-associated pneumonia (90.0% of patient-days in last month vs. 50.0% in first month; OR, 6.35; 95% CI, 1.85-21.79) and precautions to prevent catheter-related bloodstream infection (70.0% of patients receiving central lines vs. 10.6%; OR, 30.06; 95% CI, 11.00-82.17). Adoption of other practices, many with high baseline adherence, changed little.


In a collaborative network of community ICUs, a multi-faceted quality improvement intervention improved adoption of care practices.


Certain therapies have clearly established benefits for critically ill patients, such as semi-recumbent positioning to decrease the incidence of ventilator associated pneumonia, and anticoagulant prophylaxis to prevent deep venous thrombosis [1, 2]. However, there are significant gaps in the implementation of 'best-care' practices in non-academic hospitals, where a majority of critically ill patients are managed [35].

Scales and colleagues conducted a cluster randomized trial to examine the effectiveness of a multi-faceted quality intervention approach, targeting community intensive care units (ICUs) to improve adoption of six evidence-based care practices over time. Fifteen ICUs were randomized to two groups, which were studied over a twelve month period and divided into three study phases. During each phase, one group of ICUs actively received quality improvement (QI) interventions targeting one particular best care practice, and simultaneously served as the control for the paired group of ICUs receiving interventions targeting another evidence-based practice. The odds ratio of improvement over time was calculated for each practice, in both the active and control group, and reported as ratio of the odds ratio over time. The summary ratio of ORs for all practices indicated greater adoption of evidence-based practices in intervention ICUs than controls.

The greatest strength of the study lies in its sophisticated design and analysis. The trial had a pragmatic approach [6], using easy to monitor process measures rather than clinical outcomes [7], and targeting community ICUs, which face immense challenges in implementation of evidence-based practices. Furthermore, the design facilitated the delivery of quality measures to every ICU by having an 'active control' model [8], thereby enhancing individual participation and motivation. A broad range of quality improvement measures were instituted, targeting diverse evidence-based practices.

The main limitation of the study is exposed by the lack of substantial improvement in rates of adoption over time for most practices, despite delivery of a comprehensive quality improvement package. This may be due to high rates of baseline adherence for most process-of-care measures studied. Furthermore, the study showed an improved delivery of interventions to prevent ventilator-associated pneumonia and catheter-related blood stream infections in the intervention group compared to the control group ICUs. However, the control ICUs had high baseline adherence rates for both practices. Thus a ceiling effect in the control group may have partially explained the comparatively high rate of change over time in the intervention group. Additionally, the basis for a positive trend in adoption rates in control ICUs also remains in question. One wonders whether increased embracement of evidence-based practices, particularly in the control group, was simply related to being observed in the study [9], or whether interventions targeting one practice had beneficial 'cross-over effects' on adoption of other diverse practices. Finally, the strategies chosen to accomplish behavior change are known to generally exert only modest effects on clinicians' behavior.


Moving forward, the study offers new insight to accomplish meaningful quality improvement in the community setting. Although the telecommunication network proved to be a crucial tool in delivery of QI interventions, many frontline clinicians identified relatively easy-to-implement approaches, such as regular audit and feedback, and 'friendly inter-ICU competition', to improve outcomes over time. An important area of future research may be to explore whether more potent behavioral interventions, such as changing default plans of care [10] and including forced reminders, result in larger effects that are sustainable over time. This methodically sound study has created a benchmark for future quality improvement research.


Authors’ Affiliations

Department of Critical Care Medicine, University of Pittsburgh
Program of Ethics and Decision Making in Critical Illness The Clinical Research, Investigation and Systems Modeling of Acute Illness (CRISMA), Center University of Pittsburgh


  1. Samama MM, Cohen AT, Darmon JY, Desjardins L, Eldor A, Janbon C, Leizorovicz A, Nguyen H, Olsson CG, Turpie AG, Weisslinger N: A comparison of enoxaparin with placebo for the prevention of venous thromboembolism in acutely ill medical patients. Prophylaxis in Medical Patients with Enoxaparin Study Group. N Engl J Med 1999, 341: 793-800. 10.1056/NEJM199909093411103View ArticlePubMedGoogle Scholar
  2. Drakulovic MB, Torres A, Bauer TT, Nicolas JM, Nogue S, Ferrer M: Supine body position as a risk factor for nosocomial pneumonia in mechanically ventilated patients: a randomised trial. Lancet 1999, 354: 1851-1858. 10.1016/S0140-6736(98)12251-1View ArticlePubMedGoogle Scholar
  3. Bach PB, Carson SS, Leff A: Outcomes and resource utilization for patients with prolonged critical illness managed by university-based or community-based subspecialists. Am J Respir Crit Care Med 1998, 158: 1410-1415.View ArticlePubMedGoogle Scholar
  4. Ilan R, Fowler RA, Geerts R, Pinto R, Sibbald WJ, Martin CM: Knowledge translation in critical care: factors associated with prescription of commonly recommended best practices for critically ill patients. Crit Care Med 2007, 35: 1696-1702. 10.1097/01.CCM.0000269041.05527.80View ArticlePubMedGoogle Scholar
  5. McGlynn EA, Asch SM, Adams J, Keesey J, Hicks J, DeCristofaro A, Kerr EA: The quality of health care delivered to adults in the United States. N Engl J Med 2003, 348: 2635-2645. 10.1056/NEJMsa022615View ArticlePubMedGoogle Scholar
  6. Zwarenstein M, Treweek S, Gagnier JJ, Altman DG, Tunis S, Haynes B, Oxman AD, Moher D: Improving the reporting of pragmatic trials: an extension of the CONSORT statement. BMJ 2008, 337: a2390. 10.1136/bmj.a2390View ArticlePubMedGoogle Scholar
  7. Center for Medicare & Medicaid Services. Physician Quality Reporting Initiative[]
  8. Hauschke D, Pigeot I: Establishing efficacy of a new experimental treatment in the 'gold standard' design. Biom J 2005, 47: 782-786. 10.1002/bimj.200510169View ArticlePubMedGoogle Scholar
  9. Verstappen WH, van der Weijden T, ter RG, Grimshaw J, Winkens R, Grol RP: Block design allowed for control of the Hawthorne effect in a randomized controlled trial of test ordering. J Clin Epidemiol 2004, 57: 1119-1123. 10.1016/j.jclinepi.2004.03.009View ArticlePubMedGoogle Scholar
  10. Halpern SD, Ubel PA, Asch DA: Harnessing the power of default options to improve health care. N Engl J Med 2007, 357: 1340-1344. 10.1056/NEJMsb071595View ArticlePubMedGoogle Scholar


© BioMed Central Ltd 2011