Preventable mortality evaluation in the ICU
© BioMed Central Ltd 2012
Published: 30 April 2012
Mortality is the most widely measured outcome parameter. Improvement of this outcome parameter in critical care is nowadays expected not to come from new technologies or treatment, but from delivering the right care at the right moment in a safe way. The measurement of mortality as an outcome parameter confronts us with a problem in providing follow-up to the results. Especially when proven structure and process interventions are applied already, the cause of a suboptimal performance cannot be deduced easily. One possibility is to evaluate the causes of death and to judge preventability. In this article we explore the opportunities and difficulties of a tool to evaluate preventable mortality in the ICU.
Critical illness is still associated with high mortality. The development of new technologies to improve outcome in the ICU has brought advances, but also at least as many disappointments. In 1999 the first appearance of the report To Err is Human confronted us with impressive numbers of patients dying each year in the USA as the result of preventable harm . Since then it has been recognized that delivering the right care at the right moment in a safe way might have a greater impact on outcome than implementing new technology or treatments. As a consequence, many quality improvement efforts have been initiated in critical care. Most of these initiatives are based on the paradigm of Donabedian  and stress the importance of implementing proven structures and processes of care as the most effective step to improve outcome [3–6]. Structure and process interventions associated with improved outcome have been performed; for example, an improved outcome in ICUs with high-intensity staffing has been associated with better performance, and this has lead to more critical care patients being cared for continuously by critical care specialists . As process measures, bundles to recognize and treat sepsis and ventilator-associated pneumonia have been widely implemented [8, 9].
Because the intention of quality improvement efforts is objective improvement, measuring safety and performance or outcome has received a tremendous boost and indicators have been defined . Survival is the most important outcome of critical care and several measures have been developed to quantitate and compare mortality rates. However, not only rates are important. If we could analyze the causes of death and judge preventability, we might identify possibilities for improvement . There is an analogy with the airline industry. We have better planes, safer airports, established procedures, and trained pilots; despite this there is still a plane crash once in a while. To improve safety further, every crash is extensively investigated. A similar approach is used in trauma care, evaluating deaths for preventable causes . In critical care, there is a paucity of data about preventable mortality. In the present viewpoint we will explore the opportunities and difficulties of a tool to evaluate preventable mortality in the ICU.
How can preventable mortality in the ICU be defined?
Probably the most difficult issue in evaluating preventable mortality in the ICU is the definition of preventability . In most studies, preventability is defined as an event (death) that would not have occurred if the patient had received ordinary standards of care appropriate for the time of study. The problem is that standards of care are not always clear for the individual, complex ICU patient. Moreover, established standards of care may show variations in time (for example, glucose regulation or activated protein C administration).
Another definition can be that death is preventable (retrospectively) if it would have been possible to prevent dying by using (or not using) some treatment. In the ICU setting we often have to deal with patients with an extensive medical history with a complex course upon which risky procedures are performed. Despite receiving the best possible care, some patients will develop complications or die. The effect of errors cannot always be distinguished from progression of the disease. Patients can die from a preventable event occurring before ICU admission. Patients can also die after ICU admission in the ward due to an event that happened during the ICU admission. Even more important is the fact that mortality in the ICU is mostly multifactorial and is often due to withdrawal or withholding . Sometimes patients choose to die. These patients may have had a preventable event leading to that decision. Evaluation of treatment decisions should be regarded relating to life expectancy and health-related quality of life.
How do we evaluate mortality in the ICU now?
Mortality is the most used outcome measure in the ICU, and has been estimated in several ways.
The mortality rate is the easiest method to evaluate mortality. The definition seems clear and data are already automatically collected in almost all ICUs. Mortality rates give no information about preventability and cannot be compared between units or in time because of differences in severity of illness of the patient population. Local admission and discharge policies also influence ICU mortality.
The standardized mortality ratio or case-mix-adjusted mortality is a more relevant measure to evaluate mortality because risk adjustment is made. Data are collected in many ICUs, and in several countries national databases are in use that make benchmarking possible. Data collection takes more time and education is needed for correct scoring of illness severity . The question remains whether risk adjustment using severity scores is always adequate . Several patient categories are excluded from severity scoring systems. The standardized mortality ratio gives little information about preventability, because statistically unexpected death is not the same as preventable death . The area in which the preventability could exist is not shown. Furthermore, a low standardized mortality ratio does not mean that preventable deaths do not occur . The standardized mortality ratio can therefore be used as a screening tool for excess mortality rather than as a diagnostic tool for causes of mortality that can be prevented . ICU mortality evaluation is also influenced by local admission and discharge policies .
Morbidity and mortality conferences can be used to evaluate deceased patients in a unit or hospital. Causes of death can be evaluated and preventability can be discussed. If preventable causes of death are identified, measures can be taken to prevent reoccurrence in the future. These conferences are often based on autopsy reports . Autopsy is only performed in a selected portion (nonrandom sample) of deceased patients . Usually, therefore, only a nonrandom sample of deceased patients is discussed. Morbidity and mortality conferences are time consuming and require a nonpunitive environment to be effective, but are potentially a good method to evaluate quality of care .
Regulatory authorities in most western countries require immediate acknowledgement of all lethal incidents. By definition these lethalities are preventable and require analysis and preventive measures. Root-cause analysis is often performed in these cases, which means all factors (roots) that resulted in harm are identified in order to evaluate what has to be improved or changed to prevent reoccurrence. Sometimes there are even legal consequences. The number of never events reported, however, is far less than the actual number of preventable incidents. For incidence estimation of preventable death and identification of improvement options, this type of registration is insufficient.
Can we use hazards and/or harm registration in the ICU for mortality evaluation? (Table 1)
Terminology and definitions used to describe hazards and/or harm in patient care
Patient safety incident: event or circumstance that could have resulted in, or did result in, unnecessary harm to a patient
Adverse event (harmful incident):
• Injury or harm related to (or from) the delivery of care (Institute of Healthcare Improvement)
• A patient safety incident; undesirable health event that may or may not be related to the treatment
• Any injury due to medical management, rather than to the underlying disease 
• In several studies: an incident that resulted in death, life-threatening illness, disability at time of discharge, admission to hospital or prolongation of hospital stay
• An adverse event that is preventable, inaccurate or incomplete diagnosis and/or treatment
• Failure of a planned action to be completed as intended or the use of a wrong plan to achieve an aim 
Sentinel event (Joint Commission on Accreditation of Healthcare Organizations): a serious medical error; any unanticipated event in a healthcare setting resulting in death or serious physical or psychological injury
Never event: a serious reportable event; occurrence that should never happen in a hospital and can be prevented
Near miss: a patient safety incident; event or situation that could have resulted in an accident, injury or illness but did not, either by chance or through timely intervention
Critical incident: adverse event with the potential to harm patients, staff or visitors 
Complication: unfavorable evolution of a disease, health condition or medical treatment
Adverse events can cause morbidity and mortality. Adverse event examination as a trigger (clue) might therefore identify patients with a preventable cause of death . Implementation of system changes following analysis of event reports has the potential to improve outcome by preventing the event .
Event reporting systems are a valuable tool to identify hazards and to learn from them by performing (root-cause) analysis. Routine reporting systems, however, have a poor performance in identifying patients with harm . The most important problem of these systems is under-reporting, a possible cause of which is fear of blame or punishment and lack of clear definitions. The reporting rate can be increased by anonymity, regular feedback, and the existence of a safety climate . In the ICU, defining events and deciding which events have to be registered is a challenge because of the multidisciplinarity and complexity of the patients and the environment. The correlation between an event and death is not always clear and there might be preventable deaths for which no adverse event has been registered because it was not recognized.
The Institute of Healthcare Improvement has developed an intensive care trigger tool that can be used as an adjunct to voluntary reporting of adverse events . With this method a random selection of patient records is retrospectively screened for the presence of pre-determined criteria by trained reviewers to identify harmful events. Unexpected factors leading to adverse events will be missed.
Systematic evaluation of the incidence of selected clearly defined events can be used as a screening tool as part of a quality system. The main advantage of this method is that more accurate information is obtained because of focusing on a well-defined subject and rates of events can be measured. Benchmarking is possible and evaluation over time of improvement measures can be studied. Disadvantages are that this method is usually time-consuming and costly. Only a sample of the patients is evaluated and all hazards not defined as a trigger point are left out of the evaluation.
Prospective evaluation of adverse events by direct observation is the best available method to identify adverse events. A high number of adverse events has been identified  but usually no rates of events are obtained because not the whole population at risk is identified. Correlation between event and death cannot always be shown. Far more events than deaths will be found. Direct observation was found especially valuable in detecting near misses , as they are less frequently reported or documented in patient charts than adverse events but near misses have by definition no influence on mortality. Direct observation of performance therefore probably presents the least under-reporting but is very labor intensive, and the effect on preventable mortality is to be established.
In summary, hazards and/or harm registration in the ICU is valuable to identify hazards and learn from them by performing analysis, but is not sufficient for evaluation of preventable mortality.
Lessons from experiences outside the ICU
Trauma care has a longstanding tradition for evaluation of care . Preventable deaths have been used for many years as a performance indicator and as a tool for improving the delivery of optimal care. Most experience has been gained at hospital level and national level [12, 30]. Several study designs have been used, with preference for cohort studies and case series [12, 30, 31].
Review of hospital charts is a frequently used method. Early studies were questioned about reliability and validity because of low reproducibility of implicit judgments of single reviewers. Later studies used panels of reviewers and more explicit judgment criteria. Data extraction by a professional followed by review of a summary of the patient record to reduce workload is an accepted method, but has not been widely tested for reliability and accuracy. An internal review panel might be more rigorous in identifying preventable harm than an external review panel . A recent review of quality indicators to evaluate adult trauma care demonstrated evidence of reliability and validity and improved outcomes after implementation only for peer review of preventable death .
Another method used in trauma care for identifying possible preventable deaths is determining statistically unexpected death using the modified Trauma and Injury Severity Score. This method was found to have high sensitivity but moderate specificity for identifying deaths judged to be preventable by peer review .
Important differences between trauma care and critical care are the lower variation of patient categories in trauma care and the much smaller numbers of major trauma patients and deaths.
Related experience in other healthcare settings
Retrospective chart review is the most widely applied and thoroughly studied method for measurement of patient safety. This method is used on a national level in large studies evaluating preventable deaths and adverse events in the hospital, such as those leading to the To Err is Human report [1, 33–36]. In these studies, selections of patient charts were reviewed and results were extrapolated. Although several comments are published about the reliability of this method of investigation [37–39], it remains probably the best benchmark for estimating the extent of adverse events and preventable deaths at this level.
Quality and selection of patient charts is of utmost importance. Effective chart review depends on the quality of the patient chart. Only what is documented can be evaluated. Adverse events may therefore be missed, such as medication errors or team performance problems. In the large studies, charts were first reviewed on trigger points. These trigger points are predetermined screening criteria known to be sensitive to the occurrence of an adverse event, such as unplanned readmission [33–36]. Only the charts of patients with trigger points present were reviewed.
Review can be performed in several ways, but the choice has an impact on the results. A nondirected (holistic or implicit) review process will give broader information but lower inter-rater reliability. A criterion-based (or explicit) review process will be less sensitive, but has better reproducibility . Reviewers can be external or internal, involved or not involved in patient care, doctors, nurses or others. How many reviewers preferably should be used is not known. Training of reviewers increases inter-rater reliability, as does discussion between reviewers, but whether this is due to congruence or precision is not clear [41, 42].
The goal of the review process has been defined differently in respective studies. Is a high inter-rater agreement required for valid incidence estimation? Or are as many preventable deaths reported as possible to identify as many improvement options as possible? If more reviewers per chart are used, a majority opinion, a panel consensus, or a unanimous decision can be used. These different methods will give large variations in the incidence of preventable mortality.
These studies show that the incidence of preventable mortality in the hospital, depending on the methodology, lies around 4% of total hospital mortality .
Practices outside healthcare
High-reliability organizations - such as aviation, aircraft carriers, the nuclear industry, and the oil industry - have extensive quality and safety programs despite the fact that they have very low numbers of incidents. The use of protocols, checklists, teamwork, and a focus on safety are strongly imbedded in these organizations. If an incident occurs despite all of the safety measures, retrospective analysis is performed immediately to evaluate the causes and reasons for failure and to find possibilities for improvement and prevention. Failure is not hidden, but rather is used as a way to gain insight into the performances and weaknesses of the system . Although there are many similarities between high-reliability organizations and critical care, there are also some important differences - such as more variation and uncertainty of outcome in patients, harm at an individual patient level versus many casualties, including one's own personnel, and different team structures . Death cannot always be considered a failure in critical care. We can, however, learn from high-reliability organizing and use evaluation of our deceased patients as a way to further improvement.
Consequences for the evaluation of preventable mortality in the ICU
From high-reliability organizations we learn about safety culture, standard operating procedures, and incident evaluation. Many of these items can be put in place in the ICU. In high-reliability organizations, however, all catastrophes with low incidence can be analyzed thoroughly, which we cannot copy to the ICU environment because of the considerable (expected) mortality. Possible ways to analyze preventable mortality in the ICU can be learnt from trauma care and the studies that have been performed to develop benchmarks and nationwide incidence of preventable in-hospital mortality. Retrospective chart review is the method most widely applied and studied. Preselecting based on trigger points such as statistically unlikely death or occurrence of adverse events will probably miss too many preventable deaths in the ICU setting. From trauma care we learn that charts can be extracted by an experienced healthcare worker to reduce the workload for the reviewers. In this way all reviewers can still evaluate all deceased patients. Experiences with preventable mortality evaluation teach us that the definition of preventability is cumbersome in healthcare. This will be even more the case in critical care. The inter-rater variability will be large, but larger judging committees will identify more possibilities for improvement. An implicit review method is subjective and will increase inter-rater variability but will probably identify more preventability. The incidence of preventable mortality is probably at least 4% of all deaths because more preventable death is expected in the ICU environment with the high risk for errors.
Definition difficulties, low incidence, and large inter-rater variability will prohibit the use of a preventable mortality score as a benchmark.
Proposal for a preventable death evaluation system in the ICU
We consider it worthwhile to start a project for the development of an evaluation system for preventable mortality in the ICU. Based on the experience from other fields we have the following suggestions.
All deceased patients in the unit without preselection should be evaluated. In our unit, with approximately 3,000 admissions per year and 10% mortality, this would mean that we probably have to analyze 300 cases to find 12 preventable deaths.
An experienced healthcare worker can extract the patient data.
A multidisciplinary team consisting of about five nurses and doctors (internal and external) should be able to review all extracted records and identify most preventable events leading to death.
Definition of preventable death
A broad and intuitive definition should be used to identify as many probabilities for improvement as possible. Death should occur in the ICU, otherwise facts cannot be retrieved reliably. The illness must be survivable and life expectancy should be taken into account.
The aim should be quality improvement. We therefore need to use a method that gives the maximum amount of information for improvement.
Discussion is important but consensus between all reviewers to conclude preventability is not required as too many opportunities for improvement may be missed that way.
The conclusions are for internal use only because statistical reliability is insufficient. Possible improvements based on the results should be implemented.
Possible limitations and pitfalls
First, we have to present definitions and criteria for which deaths in the ICU are preventable and which deaths are not . As previously discussed, a nondirected review process, based on the reviewer's own professional judgment, will provide broader information but lower inter-rater reliability; while a criterion-based review process, based on explicit standards, will be less sensitive but will have better reproducibility. In our institution we chose to start with the use of a nondirected review process because we think standards of care are often not clear in the complex ICU environment and many unexpected factors influencing outcome might be present. In the future we intend to evaluate the reproducibility and usefulness of this nondirected review process.
Second, if we succeed in defining a preventable death, we have to define what actions to take next. This important subject is beyond the scope of the present viewpoint and is discussed elsewhere . One of the big challenges for healthcare organizations is not to identify harm in individual cases, but to create a learning organization. The preventable death evaluation system should be used to maximize improving patient care and to minimize adding to the long list of reports detailing the failures of modern healthcare.
Third, an important pitfall, is how to prepare the patient data and how to present these data to the reviewers. We started this process by letting the first and second authors of this article screen the charts and summarize the cases into anonymized (for patient and caregiver information) case vignettes. All available information considered necessary for identification of preventable death was extracted from the records. However, this is also a subjective process. In the future our research will focus on how and whether this case abstraction process can be both sensitive and specific enough to be worthwhile.
The ICU should be the safest place possible for our patients. Structure and process should be shaped according to widely accepted standards. Outcome should be measured and compared with existing benchmarks with all limitations in mind. Adverse events should be registered, evaluated and, whenever possible, prevented. After performing and evaluating care according to all standards, retrospective evaluation of mortality might reveal unnoticed preventable causes of death. These causes should be known to the ICU team so analysis can be performed and improvement actions can be taken.
Gaining insight into preventable mortality in the ICU is difficult. In this article we have evaluated possible methods to identify modifiable causes of death. Retrospective case review seems a feasible method for preventable mortality evaluation in the ICU. Ample evaluation of our own daily practice might reveal opportunities to prevent reoccurrence of events leading to death and thus improve the outcome of the critically ill patient, thereby creating the safest place possible.
- Kohn L, Corrigan JM, Donaldson MS, Committee on Quality of Health Care in America: To Err is Human: Building a Safer Health System. Washington, DC: National Academy Press; 2000.Google Scholar
- Donabedian A: Evaluating the quality of medical care. 1966. Milbank Q 2005, 83: 691-729. 10.1111/j.1468-0009.2005.00397.xPubMed CentralView ArticlePubMedGoogle Scholar
- Pronovost PJ, Goeschel CA, Marsteller JA, Sexton JB, Pham JC, Berenholtz SM: Framework for patient safety research and improvement. Circulation 2009, 119: 330-337. 10.1161/CIRCULATIONAHA.107.729848View ArticlePubMedGoogle Scholar
- Brennan TA, Gawande A, Thomas E, Studdert D: Accidental deaths, saved lives, and improved quality. N Engl J Med 2005, 353: 1405-1409. 10.1056/NEJMsb051157View ArticlePubMedGoogle Scholar
- Curtis JR, Cook DJ, Wall RJ, Angus DC, Bion J, Kacmarek R, Kane-Gill SL, Kirchhoff KT, Levy M, Mitchell PH, Moreno R, Pronovost P, Puntillo K: Intensive care unit quality improvement: a 'how-to' guide for the interdisciplinary team. Crit Care Med 2006, 34: 211-218. 10.1097/01.CCM.0000190617.76104.ACView ArticlePubMedGoogle Scholar
- Moreno RP, Rhodes A, Donchin Y: Patient safety in intensive care medicine: the Declaration of Vienna. Intensive Care Med 2009, 35: 1667-1672. 10.1007/s00134-009-1621-2PubMed CentralView ArticlePubMedGoogle Scholar
- Gajic O, Afessa B: Physician staffing models and patient safety in the ICU. Chest 2009, 135: 1038-1044. 10.1378/chest.08-1544View ArticlePubMedGoogle Scholar
- Bouadma L, Deslandes E, Lolom I, Le Corre B, Mourvillier B, Regnier B, Porcher R, Wolff M, Lucet J: Long-term impact of a multifaceted prevention program on ventilator-associated pneumonia in a medical intensive care unit. Clin Infect Dis 2010, 51: 1115-1122. 10.1086/656737View ArticlePubMedGoogle Scholar
- Dellinger RP, Levy MM, Carlet JM, Bion J, Parker MM, Jaeschke R, Reinhart K, Angus DC, Brun-Buisson C, Beale R, Calandra T, Dhainaut JF, Gerlach H, Harvey M, Marini JJ, Marshall J, Ranieri M, Ramsay G, Sevransky J, Thompson BT, Townsend S, Vender JS, Zimmerman JL, Vincent JL: Surviving Sepsis Campaign: international guidelines for management of severe sepsis and septic shock: 2008. Intensive Care Med 2008, 34: 17-60. 10.1007/s00134-007-0934-2PubMed CentralView ArticlePubMedGoogle Scholar
- de Vos M, Graafmans W, Keesman E, Westert G, van der Voort PH: Quality measurement at intensive care units: which indicators should we use? J Crit Care 2007, 22: 267-274. 10.1016/j.jcrc.2007.01.002View ArticlePubMedGoogle Scholar
- Dubois RW, Brook RH: Preventable deaths: who, how often, and why? Ann Intern Med 1988, 109: 582-589.View ArticlePubMedGoogle Scholar
- Chiara O, Cimbanassi S, Pitidis A, Vesconi S: Preventable trauma deaths: from panel review to population based-studies. World J Emerg Surg 2006, 1: 12. 10.1186/1749-7922-1-12PubMed CentralView ArticlePubMedGoogle Scholar
- Pronovost PJ, Colantuoni E: Measuring preventable harm: helping science keep pace with policy. JAMA 2009, 301: 1273-1275. 10.1001/jama.2009.388View ArticlePubMedGoogle Scholar
- Vincent JL, Parquier JN, Preiser JC, Brimioulle S, Kahn RJ: Terminal events in the intensive care unit: review of 258 fatal cases in one year. Crit Care Med 1989, 17: 530-533. 10.1097/00003246-198906000-00009View ArticlePubMedGoogle Scholar
- Polderman KH, Thijs LG, Girbes AR: Interobserver variability in the use of APACHE II scores [letter]. Lancet 1999, 353: 380.View ArticlePubMedGoogle Scholar
- Vincent JL, Moreno R: Clinical review: scoring systems in the critically ill. Crit Care 2010, 14: 207.PubMed CentralView ArticlePubMedGoogle Scholar
- Dijkema LM, Dieperink W, Heesink A, Zijlstra JG: Evaluating (potential) preventable mortality in an adult ICU [abstract]. Intensive Care Med 2011, 37: S137.Google Scholar
- Guru V, Tu JV, Etchells E, Anderson GM, Naylor CD, Novick RJ, Feindel CM, Rubens FD, Teoh K, Mathur A, Hamilton A, Bonneau D, Cutrara C, Austin PC, Fremes SE: Relationship between preventability of death after coronary artery bypass graft surgery and all-cause risk-adjusted mortality rates. Circulation 2008, 117: 2969-2976. 10.1161/CIRCULATIONAHA.107.722249View ArticlePubMedGoogle Scholar
- Ben Tovim DI: Hospital mortality ratios. Death is final: getting the balance right. BMJ 2010, 340: c2741. 10.1136/bmj.c2741View ArticlePubMedGoogle Scholar
- Rosenberg AL, Hofer TP, Strachan C, Watts CM, Hayward RA: Accepting critically ill transfer patients: adverse effect on a referral center's outcome and benchmark measures. Ann Intern Med 2003, 138: 882-890.View ArticlePubMedGoogle Scholar
- De Vlieger GY, Mahieu EM, Meersseman W: Clinical review: What is the role for autopsy in the ICU? Crit Care 2010, 14: 221. 10.1186/cc8925PubMed CentralView ArticlePubMedGoogle Scholar
- Girbes AR, Zijlstra JG: Is routine autopsy in the intensive care unit viable? Crit Care 2010, 14: 425. 10.1186/cc9069PubMed CentralView ArticlePubMedGoogle Scholar
- Ksouri H, Balanant PY, Tadie JM, Heraud G, Abboud I, Lerolle N, Novara A, Fagon JY, Faisy C: Impact of morbidity and mortality conferences on analysis of mortality and critical events in intensive care practice. Am J Crit Care 2010, 19: 135-145. 10.4037/ajcc2010590View ArticlePubMedGoogle Scholar
- Org G, Timsit JF, Soufir L, Tafflet M, Adrie C, Philippart F, Zahar JR, Clec'h C, Garrouste-Orgeas M, Jamali S, Dumenil AS, Azoulay E, Carlet J: Impact of adverse events on outcomes in intensive care unit patients. Crit Care Med 2008, 36: 2041-2047. 10.1097/CCM.0b013e31817b879cView ArticleGoogle Scholar
- Frey B, Schwappach D: Critical incident monitoring in paediatric and adult critical care: from reporting to improved patient outcomes? Curr Opin Crit Care 2010, 16: 649-653. 10.1097/MCC.0b013e32834044d8View ArticlePubMedGoogle Scholar
- Sari AB, Sheldon TA, Cracknell A, Turnbull A: Sensitivity of routine system for reporting patient safety incidents in an NHS hospital: retrospective patient case note review. BMJ 2007, 334: 79. 10.1136/bmj.39031.507153.AEPubMed CentralView ArticlePubMedGoogle Scholar
- Resar RK, Rozich JD, Simmonds T, Haraden CR: A trigger tool to identify adverse events in the intensive care unit. Jt Comm J Qual Patient Saf 2006, 32: 585-590.PubMedGoogle Scholar
- Rothschild JM, Landrigan CP, Cronin JW, Kaushal R, Lockley SW, Burdick E, Stone PH, Lilly CM, Katz JT, Czeisler CA, Bates DW: The Critical Care Safety Study: the incidence and nature of adverse events and serious medical errors in intensive care. Crit Care Med 2005, 33: 1694-1700. 10.1097/01.CCM.0000171609.91035.BDView ArticlePubMedGoogle Scholar
- Cales RH, Trunkey DD: Preventable trauma deaths. A review of trauma care systems development. JAMA 1985, 254: 1059-1063. 10.1001/jama.1985.03360080071032View ArticlePubMedGoogle Scholar
- MacKenzie EJ: Review of evidence regarding trauma system effectiveness resulting from panel studies. J Trauma 1999, 47: S34-S41. 10.1097/00005373-199909001-00008View ArticlePubMedGoogle Scholar
- Stelfox HT, Straus SE, Nathens A, Bobranska-Artiuch B: Evidence for quality indicators to evaluate adult trauma care: a systematic review 4. Crit Care Med 2011, 39: 846-859. 10.1097/CCM.0b013e31820a859aView ArticlePubMedGoogle Scholar
- Saltzherr TP, Wendt KW, Nieboer P, Nijsten MW, Valk JP, Luitse JS, Ponsen KJ, Goslings JC: Preventability of trauma deaths in a Dutch Level-1 trauma centre. Injury 2011, 42: 870-873. 10.1016/j.injury.2010.04.007View ArticlePubMedGoogle Scholar
- Brennan TA, Leape LL, Laird NM, Hebert L, Localio AR, Lawthers AG, Newhouse JP, Weiler PC, Hiatt HH: Incidence of adverse events and negligence in hospitalized patients. Results of the Harvard Medical Practice Study I. N Engl J Med 1991, 324: 370-376. 10.1056/NEJM199102073240604View ArticlePubMedGoogle Scholar
- Baker GR, Norton PG, Flintoft V, Blais R, Brown A, Cox J, Etchells E, Ghali WA, Hebert P, Majumdar SR, O'Beirne M, Palacios-Derflingher L, Reid RJ, Sheps S, Tamblyn R: The Canadian Adverse Events Study: the incidence of adverse events among hospital patients in Canada. CMAJ 2004, 170: 1678-1686. 10.1503/cmaj.1040498PubMed CentralView ArticlePubMedGoogle Scholar
- Zegers M, de Bruijne MC, Wagner C, Hoonhout LH, Waaijman R, Smits M, Hout FA, Zwaan L, Christiaans-Dingelhoff I, Timmermans DR, Groenewegen PP, van der Wal G: Adverse events and potentially preventable deaths in Dutch hospitals: results of a retrospective patient record review study. Qual Saf Health Care 2009, 18: 297-302. 10.1136/qshc.2007.025924View ArticlePubMedGoogle Scholar
- Wilson RM, Runciman WB, Gibberd RW, Harrison BT, Newby L, Hamilton JD: The Quality in Australian Health Care Study. Med J Aust 1995, 163: 458-471.PubMedGoogle Scholar
- Hayward RA, Hofer TP: Estimating hospital deaths due to medical errors: preventability is in the eye of the reviewer. JAMA 2001, 286: 415-420. 10.1001/jama.286.4.415View ArticlePubMedGoogle Scholar
- Thomas EJ, Lipsitz SR, Studdert DM, Brennan TA: The reliability of medical record review for estimating adverse event rates. Ann Intern Med 2002, 136: 812-816.View ArticlePubMedGoogle Scholar
- Hofer TP, Hayward RA: Are bad outcomes from questionable clinical decisions preventable medical errors? A case of cascade iatrogenesis. Ann Intern Med 2002, 137: 327-333.View ArticlePubMedGoogle Scholar
- Hutchinson A, Coster JE, Cooper KL, McIntosh A, Walters SJ, Bath PA, Pearson M, Rantell K, Campbell MJ, Nicholl J, Irwin P: Assessing quality of care from hospital case notes: comparison of reliability of two methods. Qual Saf Health Care 2010, 19: e2. 10.1136/qshc.2010.040352View ArticlePubMedGoogle Scholar
- Zegers M, de Bruijne MC, Wagner C, Groenewegen PP, van der Wal G, de Vet HC: The inter-rater agreement of retrospective assessments of adverse events does not improve with two reviewers per patient record. J Clin Epidemiol 2010, 63: 94-102. 10.1016/j.jclinepi.2009.03.004View ArticlePubMedGoogle Scholar
- Hofer TP, Bernstein SJ, DeMonner S, Hayward RA: Discussion between reviewers does not improve reliability of peer review of hospital quality. Med Care 2000, 38: 152-161. 10.1097/00005650-200002000-00005View ArticlePubMedGoogle Scholar
- Christianson MK, Sutcliffe KM, Miller MA, Iwashyna TJ: Becoming a high reliability organization. Crit Care 2011, 15: 314. 10.1186/cc10360PubMed CentralView ArticlePubMedGoogle Scholar
- Reader TW, Cuthbertson BH: Teamwork and team training in the ICU: where do the similarities with aviation end? Crit Care 2011, 15: 313. 10.1186/cc10353PubMed CentralView ArticlePubMedGoogle Scholar
- Welters ID, Gibson J, Mogk M, Wenstone R: Major sources of critical incidents in intensive care. Crit Care 2011, 15: R232. 10.1186/cc10474PubMed CentralView ArticlePubMedGoogle Scholar