Model for predicting short-term mortality of severe sepsis

Introduction To establish a prognostic model for predicting 14-day mortality in ICU patients with severe sepsis overall and according to place of infection acquisition and to sepsis episode number. Methods In this prospective multicentre observational study on a multicentre database (OUTCOMEREA) including data from 12 ICUs, 2268 patients with 2737 episodes of severe sepsis were randomly divided into a training cohort (n = 1458) and a validation cohort (n = 810). Up to four consecutive severe sepsis episodes per patient occurring within the first 28 ICU days were included. We developed a prognostic model for predicting death within 14 days after each episode, based on patient data available at sepsis onset. Results Independent predictors of death were logistic organ dysfunction (odds ratio (OR), 1.22 per point, P < 10-4), septic shock (OR, 1.40; P = 0.01), rank of severe sepsis episode (1 reference, 2: OR, 1.26; P = 0.10 ≥ 3: OR, 2.64; P < 10-3), multiple sources of infection (OR; 1.45, P = 0.03), simplified acute physiology score II (OR, 1.02 per point; P < 10-4), McCabe score ([greater than or equal to]2) (OR, 1.96; P < 10-4), and number of chronic co-morbidities (1: OR, 1.75; P < 10-3, ≥ 2: OR, 2.24, P < 10-3). Validity of the model was good in whole cohorts (AUC-ROC, 0.76; 95%CI, 0.74 to 0.79; and HL Chi-square: 15.3 (P = 0.06) for all episodes pooled). Conclusions In ICU patients, a prognostic model based on a few easily obtained variables is effective in predicting death within 14 days after the first to fourth episode of severe sepsis complicating community-, hospital-, or ICU-acquired infection.


Introduction
Severe sepsis remains a leading cause of death in industrialised countries, and the number of deaths caused by sepsis is increasing despite improved survival rates [1,2]. Apart from measures directed to the infectious cause (antibiotics and surgery), the treatment remains chiefly supportive despite many randomised controlled trials [3,4]. Sepsis is a syndrome, not a disease; and many factors explain the variability of outcomes, APACHE II: Acute Physiologic and Chronic Health Evaluation II; AUC: area under the curve; CI: Confidence Intervals; DNR: do not resuscitate; FiO 2 : fraction of inspired oxygen; HL: Hosmer-Lemeshow chi-squared test; ICU: intensive care unit; LOD: Logistic Organ Dysfunction; MPM II 0 : Mortality Probability models II 0 ; OR: odds ratio; PaO 2 : partial pressure of arterial oxygen; PCO 2 : partial pressure of carbon dioxide; ROC: receiver-operating characteristics; SAPS II: Simplified Acute Physiology Score II; SIRS: systemic inflammatory response syndrome. such as differences in infection sites, causative pathogens, and time and location of infection onset (community, hospital or intensive care unit (ICU)) [1]. This heterogeneity explains that no reliable measures of disease activity have been identified. Attempts to select uniform populations often used illdefined non-inclusion criteria such as moribund status.
Despite the current tendency to focus on mortality rates after one year or longer, which are highly relevant to cost-effectiveness issues, short-term mortality may be a more appropriate outcome for determining whether new treatments correct the acute effects of severe sepsis. This is because many patients who recover from severe sepsis die later from pre-existing chronic illnesses. Moreover, outcomes and risk factors of patients with severe sepsis vary considerably with the number of episodes and with the time and place (community, hospital or ICU) of acquisition.
The objective of this study was to design a prognostic model for predicting death within 14 days of severe sepsis onset at any time during the first 28 days of the ICU stay. The model was to be based on variables collected at admission and on the day the sepsis episode was diagnosed. Up to four sepsis episodes per patient were included. We evaluated the performance of our model separately in subgroups defined based on the place of infection acquisition. We compared our model with other, widely used scores. Our model may prove useful for designing future studies.

Data source
We conducted a prospective observational study using data entered into a multicentre database (OUTCOMEREA ® ) from November 1996 to April 2007. The database, with input from 12 French ICUs, contains data on admission features and diagnosis, daily disease severity, iatrogenic events, nosocomial infections and vital status. Data for a random sample of at least 50 patients older than 16 years and having ICU stays longer than 24 hours were consecutively entered into the database each year. Each participating ICU chose to perform random sampling by taking either consecutive admissions to selected ICU beds throughout the year or consecutive admissions to all ICU beds over a single month. The contact physicians for the database in the participating ICUs, who are listed in the appendix, are accredited according to French law [5].

Ethical issues
According to French law, this study did not require patient consent, because it involved research on a database. The study was approved by the institutional review board of the Centres d'Investigation Rhône-Alpes-Auvergne.

Data collection
Data were collected daily by senior physicians in the participating ICUs. For each patient, the data were entered into an electronic case-report form using VIGIREA ® and RHEA ® datacapture software (OUTCOMEREA™, Rosny-sous-Bois, France), and all case-report forms were then entered into the OUTCOMEREA ® data warehouse. All codes and definitions were established prior to study initiation. The following information was recorded for each patient: age, sex, admission category (medical, scheduled surgery or unscheduled surgery), origin (home, ward or emergency room) and McCabe score [6]. Based on previously reported reproducibility data, the McCabe score was transformed into a dummy variable, that is, 'death expected within five years, yes or no' [7]. Severity of illness was evaluated on the first ICU day using the Simplified Acute Physiology Score (SAPS II) [8], Logistic Organ Dysfunction (LOD) score [9], Sequential Organ Failure Assessment (SOFA) score [10], Mortality Probability models II0 score (MPM0 II score) [11,12], and Acute Physiologic and Chronic Health Evaluation (APACHE) II score [13]. Knaus scale definitions were used to record pre-existing chronic organ failures including respiratory, cardiac, hepatic, renal and immune system failures [13]. Patients were followed until the end of the hospital stay in order to record the vital status 14 days after sepsis onset. For the model, we computed SAPS II and LOD scores based on the data immediately available on admission or on the day (up to 24 hours) before the diagnosis of each episode of sepsis.

Quality of the database
The data-capture software automatically conducted multiple checks for internal consistency of most of the variables at entry in the database. Queries generated by these checks were resolved with the source ICU before incorporation of the new data into the database. At each participating ICU, data quality was controlled by having a senior physician from another participating ICU check a 2% random sample of the study data.

Study population
Because diagnostic coding has been found to be unreliable [14], we used parameters collected by our data-capture software to select patients with severe sepsis, defined as systemic inflammatory response syndrome (SIRS) combined with an infectious episode and dysfunction of at least one organ, occurring at or within 28 days after admission to the ICU. We excluded patients with treatment-limitation decisions taken before or on the day of the diagnosis of severe sepsis. At least two of the following criteria were required for the diagnosis of SIRS: core temperature of 38°C or above or 36°C or less, heart rate of 90 beats/min or above, respiratory rate of 20 breaths/min or above, partial pressure of carbon dioxide (PCO 2 ) of 32 mmHg or less or use of mechanical ventilation, and peripheral leukocyte count of 12,000/mm 3 or above or 4000/mm 3 or less. Organ dysfunction was defined as follows: cardiovascular system failure was a need for vasoactive and/ or inotropic drugs, and/or systolic blood pressure less than 90 mmHg, and/or a drop in systolic blood pressure by more than 40 mmHg from baseline; renal dysfunction was urinary output of 700 ml/day or less in a patient not previously undergoing haemodialysis for chronic renal failure; respiratory dysfunction was a partial pressure of arterial oxygen (PaO 2 ) of less 70 mmHg or mechanical ventilation or a PaO 2 /fraction of inspired oxygen (FiO 2 ) ratio of 250 or less (or 200 or less in patients with pneumonia); thrombocytopenia was a platelet count of less than 80,000/mm 3 , and elevated plasma lactate was a lactate level of 3 mmol/L or above. Severe sepsis was defined as sepsis associated with at least one organ dysfunction as described above, and septic shock was defined as sepsisinduced hypotension persisting despite adequate fluid resuscitation together with organ dysfunction. Thus, patients receiving inotropic or vasoactive agents who had organ dysfunction but who were no longer hypotensive were classified as having septic shock [15]. Lengths of ICU and hospital stays were computed starting at ICU admission.
The presence or absence of infection was documented according to the standard definitions developed by the Centers for Disease Control [16]. In addition, quantitative cultures of specimens obtained by bronchoalveolar lavage, protected specimen brush, protected plugged catheter or tracheal aspiration were required to diagnose ventilator-associated pneumonia [17]. Community-acquired infection was defined as infection manifesting before or within 48 hours after hospital admission. Hospital-acquired infection was infection manifesting at least 48 hours after hospital admission but before ICU admission. ICU-acquired infection was diagnosed at least 48 hours after ICU admission. Infection sites were categorised as follows: pneumonia, peritonitis, urinary tract infection, exacerbation of chronic obstructive pulmonary disease, primary bacteraemia (excluding untreated Staphylococcus epidermidis bacteraemia), miscellaneous sites (mediastinitis, prostatitis, osteomyelitis and others), and multiple sites. Early effective antibiotic therapy was defined as effectiveness on the causative agent of at least one of the empirically selected antibiotics on the day of the diagnosis of an episode of severe sepsis. Relapse/recurrence was defined as a new episode of severe sepsis with the same microorganism and the same infected organ. New episodes of severe sepsis involving different microorganisms or different organs from the previous episode were classified as separate episodes [18].

Outcome variable of interest
The outcome variable of interest was death within 14 days after the diagnosis of an episode of severe sepsis (up to four) acquired in the community, hospital or ICU.
We then compared the accuracy of these models with the main ones usual used (SAPS II and APACHE II scores and MPM II 0 ).

Statistical analysis
Our main objective was to develop a patient-based prognostic model that predicted death within 14 days after the diagnosis of the first, second, third or fourth episode of severe sepsis present within 28 days after ICU admission. We randomly allocated two-thirds of the study patients to the training cohort and the remaining one-third to the validation cohort. Up to four episodes of severe sepsis per patient were included, so we conducted a cluster analysis, in which each cluster was composed of one patient with one to four sepsis episodes.
Results were expressed as numbers (percentages) for categorical variables and as medians (quartiles) for continuous variables. Qualitative variables were compared using the chisquares or Fisher's exact test and continuous variables using the Wilcoxon or Kruskal-Wallis test. A correlation exists between the 14-day outcomes of two consecutive episodes of severe sepsis occurring in the same patient. Consequently, the relation between early death and the study variables was evaluated using generalised estimating equations [19], which are well suited to the analysis of correlated data. We used a logit link function, because the distribution of the outcome variable (14-day mortality) was binary. Correlations between multiple episodes of severe sepsis occurring in the same patient were estimated using Pearson residuals and parameters, according to the maximum likelihood method. We assumed an exchangeable-structure correlation matrix for the data within each cluster. The number of the sepsis episode and the time from admission to the severe sepsis episode were introduced successively into the global model, and the final model that minimised the Akaike information criterion was retained.
Variables associated with early death at the 0.2 level by univariate analysis were introduced in the multivariate model and subsequently selected in order to improve model deviance.
The assumption that quantitative variables were linear in the logit was checked using cubic polynomials and graphical methods. In the absence of log-linearity, continuous variables were transformed into qualitative variables according to the slope of the cubic polynomial functions and to the distribution of the variables. A pooled test of clinically relevant two-way interactions was performed on the final model and correlations between selected variables were verified. We checked for potential co-linearity of the variables included in the final model. R values of less than 0.2 were considered acceptable.
Our primary assessment of model performance was goodness-of-fit as evaluated by the Hosmer-Lemeshow statistic and by calibration curves. Lower Hosmer-Lemeshow values and higher P values (> 0.05) indicate better fit. We also assessed discrimination (i.e., the ability of the model to separate survivors and non-survivors) using the area under the curve (AUC) of the receiver-operating characteristics (ROC) curve. AUC values greater than 0.80 indicate good discrimination.
The quality of our model was tested separately in communityacquired, hospital-acquired and ICU-acquired sepsis. Then, the final model was evaluated in the validation cohort and compared with other models (SAPS II scores, APACHE II scores and MPM II 0 score) using the method of Hanley and McNeil to compare AUC-ROC values [19]. Analyses were computed using the SAS 9.1.3 package (SAS Institute, Cary, NC, USA), R and Medcalc 5.00 (Medcalc, Ghent, Belgium).

Results
Among the 7719 patients in the OUTCOMEREA ® base, 2268 experienced 2737 episodes of severe sepsis, including 674 patients who had 793 episodes of septic shock. Of the 2268 patients, 1458 patients with 1716 episodes of severe sepsis were included in the training cohort and 810 patients with 1021 episodes of severe sepsis were included in the validation cohort (Figure 1), using a 2:1 randomisation procedure. Characteristics at ICU admission and on the day of severe sepsis onset in 14-day survivors and non-survivors are shown in Tables 1 and 2, respectively. Factors that were significantly associated with early death included worse SAPS II and LOD scores at ICU admission, septic shock (e.g. requiring either inotropic therapy or vasoactive agent support), multiple organ failure (which showed the strongest association) and co-morbidities (immunodeficiency, chronic heart failure, chronic hepatic failure, acute respiratory failure and acute heart failure). On the day of the diagnosis of severe sepsis (Table 2), factors significantly associated with early death included the use of invasive procedures and a need for vasoactive agents and/or inotropic support. Escherichia coli, Pseudomonas species, methicillin-resistant Staphylococcus aureus, Candida species, bacteraemia and multiple sources of infection were also associated with early death in the univariate analysis.
We determined the best generalised linear model, that is, the model comprising variables that were both readily available and independently associated with early death (Table 3). Among variables collected on the day of diagnosis of severe sepsis, four were associated with an increased risk of early death: worse LOD score, vasoactive and/or inotropic therapy (e.g., septic shock), second episode of severe sepsis compared with the first, and third or fourth episode of sepsis compared with the first. Among infection characteristics entered into the model, only multiple sources of infection significantly   increased the risk of early death. Interestingly, the nature of the causative microorganism was not an independent predictor of death. Among variables collected at ICU admission, the following significantly predicted death within 14 days of a sepsis episode: worse SAPS II score, presence of a fatal underlying disease yielding a McCabe score of two or three, presence of one chronic illness, and presence of two or more chronic illnesses. Corticosteroid therapy did not predict early death, even when interactions with septic shock were tested (odds ratio (OR) = 0.99, 95% CI 0.66 to 1.49, P = 0.96), and therefore was not included in our model. Absence of early effective antibiotic therapy was associated with death (OR = 0.69, 95% CI 0.53 to 0.91, P = 0.01) but was not introduced in the model because this information was not available on the day of severe sepsis.
Despite the risk of co-linearity, we considered that LOD on the first day of sepsis and SAPS II at admission could be used in the same model. First, when sepsis was acquired in the ICU, the variables shared by these two scores were not recorded at the same time. Second, using two scores in the same model decreases the loss of information caused by differences in cutoffs. There was no significant co-linearity between our variables (All R values < 0.2).
We tested our model in the training cohort in each of the three categories of patients defined by the site of infection acquisition (community, hospital or ICU; Figure 2 [20]. Moreover, calibration was satisfactory for both groups (HL chi squares P > 0.10).
Interestingly, model accuracy was similar for severe sepsis at ICU admission (n = 586, AUC = 0.85) and later in the ICU stay (days 2 to 4: n = 670, AUC = 0.82; days 5 to 7: n = 133, AUC = 0.80; days 8 to 14: n = 200, AUC = 0.80; and days 15 to 28: n = 127, AUC = 0.80). Furthermore, multiple-site infection was not associated with the rank of severe sepsis episode and therefore did not correlate with the number of episodes (P = 0.87 by Fisher's exact test).
Performance was slightly lower in the validation cohort ( Figure  3). The final model used on all episodes of severe sepsis showed good calibration (HL chi-squared, 15.3, P = 0.06) and good discrimination (AUC-ROC curve, 0.76). Results for community-and hospital-acquired infections were satisfactory, with AUC-ROC curve values of 0.80 and 0.79, respectively, and with HL chi-squares P values greater than 0.05 in both groups (0.35 and 0.06, respectively). Prediction of early death after ICU-acquired severe sepsis was less accurate, with an AUC-ROC curve of 0.70 but an HL chi-squared P value of 0.02. These data are similar to those obtained from calibration curves [See Additional Data File 1, Figure 1].   Moreover, we compared our model with daily severity scores. APACHE II, MPM II 0 and SAPS II scores were significantly less accurate than our model, with AUCs of 0.73, 0.66 and 0.72, respectively (P value < 10 -4 in all cases), and poor calibration (HL chi-squared P values of 0.03, < 10 -4 and 0.02, respectively; Figure 4).

Discussion
We found that predicting death within 14 days after the onset of severe sepsis during the first 28 days in the ICU was feasible in patients with no to three previous episodes of severe sepsis. By adjusting for confounders, we were able to build a predictive model in a training cohort that performed well in the validation cohort. If used in randomised trials, this prognostic model might help to include patients with similar disease severity and to improve adjustment for confounders.
We chose to study short-term mortality, despite the current trend among researchers to focus on long-term mortality [21][22][23]. Most studies of sepsis used 28-day all-cause mortality as the primary end-point. However, life-limiting disease is a com-mon risk factor for sepsis and may cause death shortly after successful treatment of the septic episode. Early morbidity associated with sepsis is dominated by the side effects of lifesupporting interventions (e.g., mechanical ventilation, dialysis and vasoactive agents), whereas delayed morbidity (e.g., neuromuscular weakness, cognitive dysfunction and neuropsychiatric sequelae) is chiefly related to prolonged ICU management. Sepsis is an acute event and its main manifestation, acute organ dysfunction, does not seem to be associated with long-term mortality in patients who survive the original insults [23]. Furthermore, many studies failed to adjust appropriately for treatment-limitation decisions such as do not resuscitate (DNR) given early (less than two days) or later during the ICU stay. Underlying illness is the main reason for DNR orders, which are taken in up to half the patients who die in the ICU [24]. Moreover, treatment-limitation decisions were found to be independently associated with ICU death [25].
Severe infections per se are associated with a decrease in life expectancy. In a study that included controls from the general population, sepsis not only caused acute mortality, but also increased the risk of death for up to five years after the septic episode, even after adjustment for pre-existing co-morbidities [26]. The risk of delayed death during the first year was associated with the severity of the septic episode [26]. Several other studies showed that mortality and morbidity rates remained increased for several years among hospital survivors of infection and sepsis [27][28][29][30][31]. However, there is a two-way relation between acute and chronic illnesses. Chronic disease   increases the risk of infection and severe sepsis, and survivors of severe sepsis may experience an increase in their burden of chronic disease, which in turn may further elevate the risk for subsequent acute illnesses, thereby initiating a spiral of events that eventually causes death [23]. Therefore, a reasonable hypothesis is that early mortality (e.g., within 14 days) can be ascribed to the severity of acute severe sepsis [32,33] and to the effectiveness of treatment, rather than to underlying chronic illnesses, provided patients with treatment-limitation decisions are excluded, as in our study. Short-term survival may need to be viewed as a surrogate measure, because it is desirable only when followed by long-term survival with an acceptable quality of life. On the other hand, focusing on very long-term mortality, which is extremely relevant to healthcarecost issues, may mask beneficial effects of drugs used to treat sepsis if the patient dies later on as a result of an underlying chronic illness associated with a risk of sepsis [23]. High death rates due to underlying diseases may explain why many therapeutic trials in patients with severe sepsis failed to detect benefits related to the experimental treatments. Although emphasis is often put on the α risk of false-positive results, the β risk of missing true effects as a result of inadequate statisti-cal power is just as important for the overall population, because false-negative results deprive patients of effective treatments. Therefore, when designing large trials of treatments for severe sepsis, it may be appropriate to select candidate treatments in preliminary trials that use short-term mortality as the primary endpoint.

Site of infection
We found that mortality from severe sepsis could be predicted based on variables associated with the PIRO concept [34] (P: co-morbidities, McCabe; I: multiple-site infection, number of severe sepsis episodes; and R and O: organ dysfunction and vasoactive drug use). These findings are in accordance with a recent report of a PIRO-based score designed to predict 28day mortality from sepsis, thus focusing on a nearer time horizon than many recent studies evaluating longer term outcome (e.g., longer than three months) [21]. Studies of pneumonia already used 14-day mortality as the primary outcome of interest, to separate the impact of pneumonia from that of co-morbidities or other factors [32,33].
Our study has several limitations. First, the location of the patient before hospital admission was not recorded in the early  years of our database. Patients who are chronically in contact with healthcare systems on an outpatient or day-hospital basis (e.g., for chronic dialysis or other chronic treatments) are at risk for severe infection with resistant strains, although they are not admitted [35]. We have been recording this variable since April 2000 and have found that fewer than 5% of patients directly admitted to the ICUs included in the database are recipients of chronic hospital-based outpatient care. Moreover, hospital-acquired infection (in patients transferred to the ICU from other wards) was diagnosed in more than half our patients and was not associated with 14-day mortality ( Table  1, P = 0.85). Second, calibration as assessed using the HL goodness-of-fit was unsatisfactory (HL chi-squared P < 0.05), although discrimination remained good (AUC-ROC = 0.7, 95% CI 0.65 to 0.75) for ICU-acquired severe sepsis in the validation dataset ( Figure 3). Third, our model was developed in a single type of healthcare system. External validation studies are needed before the model can be used in countries that have different healthcare systems from the one in France. Finally, it should be borne in mind that the relevance of 14-day mortality to long-term treatment benefits remains to be evaluated. However, our model was clearly superior to widely used models ( Figure 4) and may prove helpful for designing and analysing future trials.

Conclusions
We developed a model for predicting death within 14 days after the diagnosis of the first, second, third or fourth episode of severe sepsis occurring within 28 days after ICU admission.
The model is based on a few readily available variables. It may help to evaluate the effectiveness of new drugs or treatment strategies in reversing severe sepsis. In contrast, long-term mortality may be a better marker for the efficacy of treatments directed against sepsis, because recovery from sepsis may be followed by death due to underlying illnesses.

Figure 4
Comparison of our prediction model with other, widely used models Comparison of our prediction model with other, widely used models. The final study model (blue line) used on all episodes of severe sepsis showed good calibration (Hosmer-Lemeshow (HL) chi-squared 15.3, P = 0.06) and good discrimination (area under the curve (AUC)-receiveroperating characteristics (ROC) curve, 0.76). Acute Physiologic and Chronic Health Evaluation (APACHE) II, Mortality Probability models II 0 (MPM0 II) and Simplified Acute Physiology Score (SAPS) II scores were significantly less accurate than our model, with AUCs of 0.73, 0.66 and 0.72, respectively (P value < 10 -4 in all cases), and poor calibration (HL chi-squared P values of 0.03, < 10 -4 and 0.02, respectively).