Skip to main content

Unexpected versus all-cause mortality as the endpoint for investigating the effects of a Rapid Response System in hospitalized patients



The purpose of this study was to assess the effect of replacing all-cause mortality by death without limitation of medical treatments (LOMT) as the endpoint in a study of rapid response teams (RRTs) in hospitalized patients. We also described the time course of LOMT orders in patients dying on a general ward and the influence of RRTs on such orders.


This study is a secondary analysis of the COMET trial, a pragmatic prospective Dutch multicenter before-after study. We repeated the original analysis of the influence of RRTs on death before hospital discharge by replacing all-cause mortality by death without an LOMT order. In a subgroup of all patients dying before hospital discharge, we documented patient demographics, admission characteristics and LOMT orders of each patient. Patients age 18 years or above were included.


In total, 166,569 patients were included in the study. The unadjusted ORs were 0.865 (95 % CI 0.77-0.98) in the original analysis using all-cause mortality and 0.557 (95 % CI 0.40-0.78) when choosing death without LOMT as the endpoint. In total, 3408 patients died before discharge. At time of death, 2910 (85 %) had an LOMT order. Median time from last change in LOMT status and death was 2 days (IQR 1–5) in the before-phase and median time after introduction of the RRT was 1 day (IQR 1–4) (p value not significant).


The improvement in survival of hospitalized patients after introduction of a rapid response team in the COMET study was more pronounced when choosing death without limitation of medical treatment, rather than all deaths as the endpoint. Most patients who died during hospitalization had limitation of medical treatments ordered, often shortly before death. Rapid response teams did not influence the institution of limitation of medical treatments.


Patients who are admitted to general wards in hospitals may deteriorate, which may result in unplanned ICU admission, cardiac arrest, or even death [1]. Rapid response systems have been developed for timely identification and treatment of patients on general wards, who are at risk of clinical deterioration [2]. These systems have different names in the literature, including rapid response team, outreach team, or medical emergency team. In this paper we will use the term rapid response team (RRT) for both the actual outreach team and the rapid response system as a whole.

Three large controlled studies investigated the effects of the introduction of an RRT on clinical outcomes [35]. The endpoints of these studies were mortality, unplanned ICU admission and cardiac arrest rates. While studies in the UK and the Netherlands reported improved survival [4, 5] and decreased cardiac arrest rates [4], in an Australian study there was no improvement in a composite endpoint including mortality, unplanned ICU admission, and cardiac arrests [3].

Crude mortality may not be the optimal endpoint for studying the effects of an RRT on survival. Patients with untreatable diseases may be admitted to a hospital for palliative end-of-life care. Clearly, RRTs are not set up to prevent death in those patients. For this reason, unexpected death has been proposed as a more suitable endpoint for studying the effects of RRTs on survival [3]. Death was considered as expected if a patient was subject to a limitation of medical treatment (LOMT) order at the time of death. This, however, may not be the correct definition of expected death. First, some patients may prefer not to undergo life-sustaining treatments in the event of cardiac arrest, but this does not mean that death is imminent or that these patients do not want optimal treatment. Furthermore, treatment limitation orders are sometimes instituted shortly before death when the clinical condition has deteriorated progressively to a point that survival is no longer considered possible. Clearly, RRTs could have been beneficial in these patients if called in an earlier phase when the clinical condition was not yet hopeless.

The aim of our study was to explore the association between treatment-limitation orders and hospital death in a multicenter study of RRTs in the Netherlands. First, what is the effect of an RRT on mortality if all-cause hospital mortality is replaced by the endpoint of death without an LOMT order? Second, what proportion of patients dying on a general hospital ward are given an LOMT order, how do these LOMT orders change over time during hospitalization, and are LOMT policies influenced by the introduction of an RRT.


Design, setting, and participants

This study is a part of the Cost and Outcomes analysis of Medical Emergency Teams (COMET) multi-center study. The COMET study was designed as a prospective pragmatic before-after trial enabling the analysis of clinical outcomes after sequential introduction of the rapid response system components. Twelve Dutch hospitals participated in this study. Four study wards, comprising two surgical and two medical wards were included in each hospital, the so-called COMET wards. Patients included were 18 years of age or above. The full design of this study has been described previously [4, 6] and is shown in Fig. 1.

Fig. 1
figure 1

Design of the Cost and Outcomes analysis of Medical Emergency Teams (COMET) study. Following the baseline period of 5 months, the modified early warning score (MEWS)/situation background assessment recommendation (SBAR) was implemented for 7 months and subsequently followed up for 17 months during which the rapid response team (RRT) was available. Effects of the RRT on outcomes were measured during the last 5 months and compared with the 5 months baseline period. During the entire length of the study, data were collected on all the endpoints. For further clarification, hospitals were able to start with the study in a 3-month time period. The total study took 30 months, in which each hospital participated for 27 months

The study consisted of a before-period followed by two study phases. The before-period comprised 5 months during which baseline characteristics were collected. After that a two-step implementation of the RRT was performed. The first phase lasted 7 months during which the modified early warning score (MEWS) and the situation background assessment recommendation (SBAR) communication tools were implemented. The RRT was introduced during the second phase, which lasted 17 months. This phase was divided into the RRT implementation phase and the final RRT phase. The before-period and the final RRT phase were used to compare the effects on the outcome of the patients. To exclude seasonal effects on the outcome, the before-period and the final RRT phase in each hospital covered the same calendar months.


Unexpected death was defined as all deaths in patients without a pre-existing LOMT order [3, 7]. Definitions of the LOMT in this study were: Code A for “full active care”, Code C for “do not perform cardiopulmonary resuscitation” and/or “do not admit to ICU”, and Code D for “only palliative care”. Code B had been used in the past, but was no longer used in any of the participating hospitals. In this study, if no LOMT was recorded in the charts, this was considered equivalent to code A “for full active care”.

Ethical consideration

The medical ethics committee of the Academic Medical Center in Amsterdam waived the need for formal evaluation of the study due to the observational nature of the study. Consequently, the need for informed consent was not applicable.


All deaths were recorded during the study period using a clinical report form. All deaths included those in patients who had been admitted to the COMET ward and transferred at a certain point to a non-COMET ward. Clinical information systems in the hospitals were used to identify death during this study. We collected the following data: basic patient demographic data (age and gender), admission characteristics (date of admission, date of transfer to the COMET ward, COMET ward specialty, length of hospital stay, and date and time of death), and limitation of medical treatment (date of recorded LOMT).

After implementation of the RRT, members of the RRT collected the following data during consultation: the personnel who activated the RRT, the indication for summoning the RRT, the direct outcome after RRT, and the treatment code before and after consultation.

Statistical analysis

Data analysis was performed using SPSS version 20.0 (Armonk, New York, USA). Generalized linear mixed modeling (GLMM) was applied to assess differences in outcomes per 1000 admissions between the before-period and final RRT period, while correcting for potential confounding due to the before-after study design. Death was assumed to have a binominal distribution in the GLMM. Potential confounders were included as fixed or random variables. Hospital was modeled as a random variable. The age of patients was modeled as a random component, whereas patient’s sex and admission type (planned vs unplanned/emergency) were modeled as fixed variables. The uncorrected odds ratios (ORs) and ORs after correction for confounding are reported with their CIs and corresponding p values. Descriptive analyses are presented as raw numbers and percentages. Continuous data are presented as medians with interquartile range (IQR) due to the non-normally distributed data. The non-parametric Mann–Whitney U test was used to compare non-normally distributed continuous variables between groups. Categorical variables were compared between groups using the chi squared (χ2) test. The level of significance was set at p < 0.05.


In total 166,569 patients were included in the COMET study, of whom 2345 patients died on a medical ward and 1063 patients on a surgical ward. Of the patients who died, surgical patients were older median 81.4 years (IQR 73.6–87.0) in comparison to medical patients, median 78.4 years (68.3–85.6). The median hospital length of stay (LOS) was 7 days (IQR 3–16 days) for surgical patients compared to 6 days (3 to 13 days) for medical patients. In 13 % of patients who died and for whom an RRT was called, an LOMT was instituted or changed after consultation of the RRT. Baseline characteristics of patients are presented in Table 1.

Table 1 Demographic data

The ORs for death before hospital discharge for patients admitted during the last 5 months of the RRT phase (n = 27,820) were compared with the baseline period before implementing the RRT (n = 26,659). The originally reported unadjusted OR for all-cause mortality in the final RRT period compared to the before period was 0.865 (95 % CI 0.77-0.97) [4]. In the same cohort of patients, the unadjusted OR for death without LOMT (unexpected death) was 0.557 (95 % CI 0.40-0.78). Likewise, the ORs after adjustment for age, gender, individual hospital, and urgent vs planned admission were 0.802 (95 % CI 0.64-1.0) in the original analysis using all-cause mortality and 0.549 (95 % CI 0.38-0.78) when choosing death without LOMT as the endpoint (Table 2).

Table 2 Comparison of effect of RRT on all-cause in-hospital mortality vs death without LOMT in hospitalized patients

Table 3 shows the treatment limitations at different time points in patients who died during hospital admission. In both medical and surgical patients, most of the patients who subsequently died already had an LOMT at hospital admission. The median time between last LOMT order and death was 3 days in patients who were assigned Code C and 1 day in patients assigned code D. There was also a short time between the issue of the LOMT order and death in patients who had a prolonged hospital length of stay. Unexpected death was defined as death without a pre-existing LOMT order. There was no LOMT order at the time of death in 12 % of medical patients and 20 % of surgical patients.

Table 3 Limitation of medical treatment (LOMT) order status at different time points in patients who died during hospital admission

In Table 4 the effect of RRT implementation on treatment limitations in patients who died during their hospital stay is presented. No differences were found in institution of LOMT after introduction of the rapid response system. The delta time between the last code change and death was 2 days (median 1–5) in the before-phase and 1 day (median 1–4) in the final RRT phase; this was not significant.

Table 4 Effects of implementation of rapid response system on limitation of medical treatment (LOMT) order status


In this study we demonstrate that the effects of introducing an RRT on death in hospital is more pronounced if death without LOMT is used as the endpoint compared to all-cause mortality as was used in the original COMET analysis [4]. The underlying hypothesis as to why death without LOMT might be a better endpoint than all deaths is that patients with LOMT are expected to die and for these patients an RRT call will not be initiated. Thus, it has been argued that the true effects of an RRT are underestimated if all patients are analyzed as was done in the original analyses in the COMET study [6]. In one earlier controlled trial on the effects of an RRT in Australian hospitals, unexpected death, i.e., death in patients with no LOMT, was included in the composite endpoint consisting of unplanned ICU admission, or cardiac arrest, or unexpected death. However, the negative findings in this study may be related to factors such as insufficient statistical power and contamination of the control group [3, 8, 9].

In this cohort of patients who all died before hospital discharge, 85 % had some LOMT at the end of life. At hospital admission 65 % of patients who died in hospital had LOMT. We are not the first to show that most hospitalized patients who eventually die have LOMT. In a study from Canada and the USA, in a cohort of patients with community-acquired pneumonia who required admission to a hospital, 51 of 65 patients (78 %) who died had do-not-resuscitate orders instituted before death [10]. In 1995 in the USA, among a representative sample of Medicare patients hospitalized with congestive heart failure, acute myocardial infarction, pneumonia, cerebrovascular accident, or hip fracture, 49 % of patients who died had LOMT orders [11]. In a study in Saudi Arabia, after implementing an RRT, 2793 out of 3191 patients (88 %) dying in hospital, died on the general ward with LOMT orders instituted [12].

Patients with an LOMT are believed not to benefit from the RRT because death is expected. This, however, is not necessarily true. First, there may be many reasons for limiting medical treatments. Patients may prefer not to undergo some invasive procedures, such as mechanical ventilation, or physicians may consider treatments inappropriate due to a patient’s poor prognosis. In both circumstances, patients may still be successfully treated and discharged from the hospital. Moreover, in our study, we found that 84 % of patients who died had some limitation of medical treatment at the time of death. However, in most of these patients the LOMT order was instituted in the last days before death, sometimes even less than one day earlier. Thus, having treatment limitations at the time of death cannot be interpreted as death being expected during the entire hospital stay. It appears that LOMT instituted shortly before death is more a reflection of the deteriorating condition of the patient during their hospital stay, eventually leading to the clinical conclusion that death is inevitable and that some treatments would be better withheld. It does not imply that RRT could not have improved the outcome in the earlier period in these patients.

RRTs have been installed in hospitals with the aim of timely identification and treatment of patients deteriorating on general wards, and preventing morbid outcomes. An additional role of the RRT is to be involved in decisions and discussions with the physicians on the ward about palliative care and LOMT if patients have no real prospect of surviving with a reasonable quality of life [13]. In an earlier study, an RRT was associated with improved documentation of comfort care orders, pain scores, patient distress, and chaplain visits [14]. In a recent review, Jones and coworkers mentioned several reasons why RRTs may need to be involved in end-of-life decisions. First, the usual care team may not have recognized or may not accept that the patient is dying. Second, the usual team may not be comfortable or skilled in having end-of-life care discussions with patients or families. Last, the usual team may have difficulty in accepting LOMT despite the presence of advanced comorbidities and an irreversible new illness, due to personal or religious reasons [15]. Also, RRTs may confront situations in which LOMT orders are postponed awaiting discussion with team or family members [16].

In our study 13 % of RRT calls were followed by the institution of LOMT orders. This is less than found by others. Smith and coworkers reported that 28 % of RRT activations were associated with new LOMT orders [17]. Casamento and coworkers reported LOMT orders after 32 % of RRT calls [18]. In a study by Jones et al., 31 % of RRT activations were associated with LOMT [19]. A possible explanation for the low rate of LOMT orders after RRT calls in our study is the already high prevalence of LOMT orders at hospital admission. It appears that most patients at the end of life already had LOMT before the RRT was called. Accordingly, in our study, we found no differences in the institution of LOMT before and after implementation of the RRT, although the relatively small number of patients cannot exclude a small effect in favor of the RRT period.

In this study there are some limitations. First, during the review of the medical charts of the patients who died, we assumed that medical treatments were not limited if there was no LOMT order recorded in the patient charts. However, it is possible that implicit limitations of medical treatment were present in some of these cases. Therefore, we cannot exclude some underestimation of the LOMT during this study and consequently an overestimation of the number of patients dying unexpectedly. Second, to estimate the effect of replacing all-cause hospital mortality by death without LOMT when studying the effects of the RRT, patients dying with LOMT were considered as not having reached the endpoint, but as patients surviving up to hospital discharge. Preferentially, patients with LOMT orders should be excluded from the study population. However, as information about LOMT was only present for patients who died, this was not possible. When excluding only patients who died with LOMT, we found ORs that were almost identical to those presented here. As relatively few patients surviving up to hospital discharge have LOMT orders, we believe that it is unlikely that these patients have major influence on our findings. Our study was performed in 12 hospitals, thereby increasing the generalizability of our findings. However, all centers were in the Netherlands and it important to realize that common policies about treatment limitations differ between countries [20]. Last, we had a relatively low percentage of RRT calls recorded during this study. This may be due to administrative concerns. It was not always clear to the physician of the ward when to call the RRT or to call the ICU for rapid consultation. Thus, the real number of RRT calls may have been higher than documented.


We observed improved survival up to hospital discharge when choosing the endpoint of death without LOMT rather than all deaths, in a study on the effect of implementation of RRTs in Dutch hospitals. Implementation of rapid response systems was not associated with significant change in LOMT. Most patients who died during hospitalization had LOMT orders instituted, often shortly before death. LOMT does not necessarily mean that death is expected and that these patients could not benefit from treatment by the rapid response team.

Key messages

  • Replacing all-cause mortality by death without LOMT (unexpected death) results in improved survival up to hospital discharge

  • Most patients who died during hospitalization had LOMT orders instituted, often shortly before death

  • The presence of LOMT does not necessarily mean that death is expected and that these patients could not benefit from treatment by the rapid response team



confidence interval


Cost and outcomes analysis of medical emergency teams


generalized linear mixed modeling


Intensive Care Unit


interquartile range


limitation of medical treatments


length of stay


modified early warning score


odds ratio


rapid response teams


situation background assessment recommendation


  1. Maharaj R, Raffaele I, Wendon J. Rapid response systems: a systematic review and meta-analysis. Crit Care. 2015;19:254.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Jones DA, DeVita MA, Bellomo R. Rapid-response teams. N Engl J Med. 2011;365:139–46.

    Article  CAS  PubMed  Google Scholar 

  3. Hillman K, Chen J, Cretikos M, Bellomo R, Brown D, Doig G, et al. Introduction of the medical emergency team (MET) system: a cluster-randomised controlled trial. Lancet. 2005;365:2091–7.

    Article  PubMed  Google Scholar 

  4. Ludikhuize J, Brunsveld-Reinders AH, Dijkgraaf MG, Smorenburg SM, de Rooij SE, Adams R, et al. Outcomes associated with the nationwide introduction of rapid response systems in The Netherlands. Crit Care Med. 2015;43:2544–51.

    Article  PubMed  Google Scholar 

  5. Priestley G, Watson W, Rashidian A, Mozley C, Russell D, Wilson J, et al. Introducing critical care outreach: a ward-randomised trial of phased introduction in a general hospital. Intensive Care Med. 2004;30:1398–404.

    Article  PubMed  Google Scholar 

  6. Ludikhuize J, Dijkgraaf MGW, Smorenburg SM, de Rooij SEJA, Brunsveld-Reinders AH, Tangkau P, et al. Cost and Outcome of Medical Emergency Teams (COMET) study. Design and rationale of a Dutch multi-center study. Br J Med Med Res. 2013;3:13–28.

    Article  Google Scholar 

  7. De MK, Verspuy M, Monsieurs KG, Van BP. SBAR improves nurse-physician communication and reduces unexpected death: a pre and post intervention study. Resuscitation. 2013;84:1192–6.

    Article  Google Scholar 

  8. Chen J, Flabouris A, Bellomo R, Hillman K, Finfer S. The medical emergency team system and not-for-resuscitation orders: results from the MERIT study. Resuscitation. 2008;79(3):391–7.

  9. Hillman K, Chen J, May E. Complex intensive care unit interventions. Crit Care Med. 2009;37:S102–6.

    Article  PubMed  Google Scholar 

  10. Marrie TJ, Fine MJ, Kapoor WN, Coley CM, Singer DE, Obrosky DS. Community-acquired pneumonia and do not resuscitate orders. J Am Geriatr Soc. 2002;50:290–9.

    Article  PubMed  Google Scholar 

  11. Wenger NS, Pearson ML, Desmond KA, Brook RH, Kahn KL. Outcomes of patients with do-not-resuscitate orders. Toward an understanding of what do-not-resuscitate orders mean and how they affect patients. Arch Intern Med. 1995;155:2063–8.

    Article  CAS  PubMed  Google Scholar 

  12. Al-Qahtani S, Al-Dorzi HM, Tamim HM, Hussain S, Fong L, Taher S, et al. Impact of an intensivist-led multidisciplinary extended rapid response team on hospital-wide cardiopulmonary arrests and mortality. Crit Care Med. 2013;41:506–17.

    Article  PubMed  Google Scholar 

  13. Jones DA, McIntyre T, Baldwin I, Mercer I, Kattula A, Bellomo R. The medical emergency team and end-of-life care: a pilot study. Crit Care Resusc. 2007;9:151–6.

    PubMed  Google Scholar 

  14. Vazquez R, Gheorghe C, Grigoriyan A, Palvinskaya T, Amoateng-Adjepong Y, Manthous CA. Enhanced end-of-life care associated with deploying a rapid response team: a pilot study. J Hosp Med. 2009;4:449–52.

    Article  PubMed  Google Scholar 

  15. Jones D, Moran J, Winters B, Welch J. The rapid response system and end-of-life care. Curr Opin Crit Care. 2013;19:616–23.

    Article  PubMed  Google Scholar 

  16. Micallef S, Skrifvars MB, Parr MJ. Level of agreement on resuscitation decisions among hospital specialists and barriers to documenting do not attempt resuscitation (DNAR) orders in ward patients. Resuscitation. 2011;82:815–8.

    Article  PubMed  Google Scholar 

  17. Smith RL, Hayashi VN, Lee YI, Navarro-Mariazeta L, Felner K. The medical emergency team call: a sentinel event that triggers goals of care discussion. Crit Care Med. 2014;42:322–7.

    Article  PubMed  Google Scholar 

  18. Casamento AJ, Dunlop C, Jones DA, Duke G. Improving the documentation of medical emergency team reviews. Crit Care Resusc. 2008;10:29.

    PubMed  Google Scholar 

  19. Jones DA, Bagshaw SM, Barrett J, Bellomo R, Bhatia G, Bucknall TK, et al. The role of the medical emergency team in end-of-life care: a multicenter, prospective, observational study. Crit Care Med. 2012;40:98–103.

    Article  PubMed  Google Scholar 

  20. Sprung CL, Cohen SL, Sjokvist P, Baras M, Bulow HH, Hovilehto S, et al. End-of-life practices in European intensive care units: the Ethicus Study. JAMA. 2003;290:790–7.

    Article  PubMed  Google Scholar 

Download references


All co-authors, including the COMET study group, read and approved this manuscript. Members of the COMET study group: M.A. van Putten, R. Adams (MSc), P.F. de Maaijer (MMI) - Academic Medical Center; S.E.J.A. de Rooij (MD, PhD), UMCG Groningen,C. Kerkhoven - Catharina hospital; A. Braber, (MD) - Gelre hospital; F.J. Schoonderbeek, (MD, PhD) - Ikazia hospital; B.M. Kors, (MD) - Kennemer Gasthuis; D.P. Sep, (MD) – Noord West Ziekenhuis; J.W. Vermeijden, (MD, PhD) - Medical Spectrum Twente; B.G. Fikkers, (MD, PhD) - Radboud University Medical Center; P. Tangkau, (MD) - Reinier de Graaf Hospital; P.K.C. van der Weijden, (MD) - Rijnland Hospital; S. Koenders, (MD) - Rivas Beatrix Hospital; M. Meertens, (MD) -Sint Lucas Andreas Hospital; A.H. Brunsveld-Reinders, (MSc) - University Medical Center Leiden; M. Hoeksema, (MD) - Zaans Medical Center; S.M. Smorenburg, (MD, PhD) - Ben Sajetcentrum.

Authors’ contributions

ABR contributed to the development of the manuscript concept and design, carried out the study, performed the data analysis, and performed primary writing and editing of all drafts of the manuscript. JL contributed to the development of the manuscript concept and design, carried out the study, and contributed to the revision of the manuscript. MSA helped interpret the data and contributed to all drafts of the manuscript. MW performed the data analysis and contributed to the revision of the manuscript. EdJ contributed to the development of the manuscript concept and design and performed editing of all drafts of the manuscript. All authors read and approved the final version of the manuscript.

Competing interests

The authors declare that they have no competing interests.

Author information

Authors and Affiliations



Corresponding author

Correspondence to Anja H. Brunsveld-Reinders.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (, which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Brunsveld-Reinders, A.H., Ludikhuize, J., Dijkgraaf, M.G.W. et al. Unexpected versus all-cause mortality as the endpoint for investigating the effects of a Rapid Response System in hospitalized patients. Crit Care 20, 168 (2016).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: