Open Access

Results from the national sepsis practice survey: predictions about mortality and morbidity and recommendations for limitation of care orders

  • James M O'BrienJr1Email author,
  • Scott K Aberegg1,
  • Naeem A Ali1,
  • Gregory B Diette2 and
  • Stanley Lemeshow3
Critical Care200913:R96

https://doi.org/10.1186/cc7926

Received: 24 March 2009

Accepted: 23 June 2009

Published: 23 June 2009

Abstract

Introduction

Critically ill patients and families rely upon physicians to provide estimates of prognosis and recommendations for care. Little is known about patient and clinician factors which influence these predictions. The association between these predictions and recommendations for continued aggressive care is also understudied.

Methods

We administered a mail-based survey with simulated clinical vignettes to a random sample of the Critical Care Assembly of the American Thoracic Society. Vignettes represented a patient with septic shock with multi-organ failure with identical APACHE II scores and sepsis-associated organ failures. Vignettes varied by age (50 or 70 years old), body mass index (BMI) (normal or obese) and co-morbidities (none or recently diagnosed stage IIA lung cancer). All subjects received the vignettes with the highest and lowest mortality predictions from pilot testing and two additional, randomly selected vignettes. Respondents estimated outcomes and selected care for each hypothetical patient.

Results

Despite identical severity of illness, the range of estimates for hospital mortality (5th to 95th percentile range, 17% to 78%) and for problems with self-care (5th to 95th percentile range, 2% to 74%) was wide. Similar variation was observed when clinical factors (age, BMI, and co-morbidities) were identical. Estimates of hospital mortality and problems with self-care among survivors were significantly higher in vignettes with obese BMIs (4.3% and 5.3% higher, respectively), older age (8.2% and 11.6% higher, respectively), and cancer diagnosis (5.9% and 6.9% higher, respectively). Higher estimates of mortality (adjusted odds ratio 1.29 per 10% increase in predicted mortality), perceived problems with self-care (adjusted odds ratio 1.26 per 10% increase in predicted problems with self-care), and early-stage lung cancer (adjusted odds ratio 5.82) were independently associated with recommendations to limit care.

Conclusions

The studied clinical factors were consistently associated with poorer outcome predictions but did not explain the variation in prognoses offered by experienced physicians. These observations raise concern that provided information and the resulting decisions about continued aggressive care may be influenced by individual physician perception. To provide more reliable and accurate estimates of outcomes, tools are needed which incorporate patient characteristics and preferences with physician predictions and practices.

Introduction

Sepsis affects at least 750,000 patients annually in the USA with incidence increasing at a rate of approximately 1.5% per year [1, 2]. Critically ill patients, including those with sepsis, and their families desire prognostic information early in the hospital course to help inform decisions about continued supportive care, even when such information is uncertain [3]. Such early provision of prognostic information and shared decision-making, including clinician recommendations about appropriate treatments and goals of care, are evidence-based endorsements of the American College of Critical Care [4] and the Surviving Sepsis Campaign [5]. However, the patient and provider factors that influence physician prognostication in the intensive care unit (ICU) are largely unknown.

A series of reports from the Level of Care Study suggest that such physician predictions are influential on subsequent care and outcome. Based on their observations, physician predictions about ICU mortality and recovery are strongly predictive of subsequent withdrawal of mechanical ventilation [6], do-not-resuscitate (DNR) orders [7], and ICU mortality [8]. Therefore, better understanding of the factors that influence physician prognostication may allow for an appreciation for the mechanisms underlying factors associated with poorer outcomes among septic patients and improved risk-adjusting methodology, which could incorporate physician intuition with clinical data.

In a national survey of physicians with experience treating sepsis, we used simulated clinical vignettes to measure physician predictions about outcomes from septic shock, to test the influence of selected patient factors on these predictions and to determine how these factors and predictions affect recommendations for limitation of care. We hypothesized that physician estimates of outcomes would vary widely. We also believed that patient factors obvious to a treating clinician (older age, body mass index (BMI) for obesity, and cancer diagnosis) would be associated with higher estimates of mortality, despite identical measures of acute illness severity. Finally, we hypothesized that increasing estimates of mortality and morbidity and clinical factors would be associated with suggestions for limitations of care when no patient preference was provided.

Materials and methods

Study sample and administration

We randomly selected potential subjects from members of the Critical Care Assembly of the American Thoracic Society with a US mailing address. The study was reviewed by the Planning Committee of the Assembly and approved by the Ohio State University Biomedical Institutional Review Board. From 18 June to 24 September, 2007, we mailed self-administered surveys including a letter explaining the study purpose and a stamped return envelope. The initial mailing included $10 cash incentive. Non-respondents received a duplicate survey 30 days after the initial mailing with no additional incentive. Surveys returned for inaccurate addresses and by those who do not care for septic adults were replaced by random selection.

Questionnaire

We developed study vignettes through focus groups and a pilot administration to intensivists at The Ohio State University Medical Center. Vignettes involved a male patient with community-acquired pneumonia who received initial care, including mechanical ventilation, volume resuscitation, and antibiotics. All had an acute physiology and chronic health evaluation (APACHE) II score of 25 with sepsis-associated shock, respiratory failure, and lactic acidosis. The patient was admitted to the ICU for further care. No patient preferences regarding goals of care were provided.

Each vignette had either a normal BMI (22 kg/m2) or an obese BMI (40 kg/m2), was either younger (50 years) or older (70 years), and had either no co-morbidities or recently diagnosed stage IIA non-small cell lung cancer. Obesity was of interest because of our prior work [9, 10] and because it is consistently associated with negative physician attitudes [11, 12] but is not consistently associated with outcomes [13, 14]. We studied age to extend observations about aggressiveness of care in elderly patients with serious illnesses [15] and to determine the effect age has on physician decision-making beyond its contribution to APACHE II score. We included a recent diagnosis of a potentially curable cancer [16] to evaluate the effect of a chronic condition on predictions about acute illness. All respondents received the vignettes with the lowest [see Additional data file 1] (50 years old, no co-morbidities, normal BMI) and highest (70 years old, stage IIA non-small cell lung cancer, obese BMI) mortality rates in pilot testing. Two additional vignettes were randomly selected for each survey with weighting designed to provide adequate sample sizes for comparisons of interest. The order of the vignettes within each survey was random.

For each vignette, the respondent was asked if he or she would choose additional therapies, and, if so, which ones. Respondents were asked to predict outcomes, including the probability of hospital survival without additional interventions chosen after ICU admission (referred to as 'baseline mortality') and the probability of the patient being able to wash and dress himself six months after hospital discharge (assuming survival). Respondents indicated their prediction by placing an 'X' on visual-analog scale, represented by a 10 cm horizontal line. All outcome predictions were determined by measuring the location of the X placed on the visual-analog scale in mm. We also collected demographic information about respondents.

Sample size and statistical plan

Our primary hypothesis was that the studied patient factors would be associated with the predicted probability of hospital survival without additional interventions chosen after ICU admission (or baseline mortality). Our secondary hypotheses were that between-respondent estimates would have a wide range despite identical patient factors and that mortality and morbidity predictions and vignette factors were associated with recommendations to limit care. We classified choices of a DNR order, restriction of further escalation of care, and/or termination of supportive care as recommendations to limit care.

We used data from pilot testing for sample size calculations. We planned to demonstrate at least a 10% difference in baseline mortality between pairs of vignettes of interest with a two-sided alpha of 0.05 and power of 0.8 and expected a 50% response rate. This required an estimated sample of 355 completed surveys. We used the 5th to the 95th percentile of the estimated mortality predictions (inclusive of 90% of respondents) for each vignette as a measure of the variability in these predictions.

The unit of analysis for all results was the individual study vignette. Each respondent completed multiple vignettes (up to four), so we used analyses which accounted for this non-independence. We considered responses to the same vignette by different respondents to be independent. All tables display the association in such analyses including either a single independent variable ('univariable') or multiple independent variables ('multivariable') in linear or logistic regression models, as appropriate.

For the final risk-adjusting analyses with physician predictions as the outcome variable, we included the clinical factors from the vignettes (regardless of statistical significance) and studied respondent factors, which were significantly associated with the prediction (P < 0.05) and/or altered the parameter estimate or odds ratio of any of the patient factors by at least 15%. For the risk-adjusting analyses for recommendation to limit care with curative intent, we included the clinical factors from the vignettes (regardless of statistical significance), the prediction about baseline mortality, and problems with washing and dressing oneself in six months, assuming survival (regardless of statistical significance). We also included respondent factors which were significantly associated with the recommendation to limit care (P < 0.05) and/or altered the parameter estimate or odds ratio of any of the patient factors or predictions by at least 15%. We analyzed continuous variables with fractional polynomials to determine if transformation or categorization was appropriate and in no instance was this suggested. We used SAS (v9.1, SAS Institute, Inc., Cary, NC, USA) or STATA (SE10.0, StatCorp LP, College Station, TX, USA) for all analyses. These data were previously presented in abstract form at the 2008 American Thoracic Society International Conference.

Results

Respondents

After both mailings, we received a response rate of 40.8%, representing 81.4% of the projected sample size (Figure 1). Nearly all respondents (99%) reported caring for at least one septic patient per week and most had moderate or extensive self-rated experience in treating sepsis (Table 1). Among the completed vignettes with normal or obese BMIs, with younger and older ages, and with no co-morbidities and early-stage lung cancer, there were no statistically significant differences in respondent characteristics (data not shown).
Figure 1

Responses to National Sepsis Practice Survey.

Table 1

Respondent demographic and practice characteristics

Age, years, mean (SD)

45.7 (9.5)

Years since medical school graduation, mean (SD)

19.9 (9.7)

Decade of medical school graduation, number (%)

 

   1960s

9 (3.1%)

   1970s

65 (22.5%)

   1980s

87 (30.1%)

   1990s

104 (36.0%)

   2000s

24 (8.3%)

Primary employer, number (%)

 

   Private practice, community hospital, and/or managed care

160 (55.6%)

   Academic medical center and/or University medical school

108 (37.5%)

   Other

20 (6.9%)

Weight, pounds, mean (SD)

173.6 (30.6)

Height, inches, mean (SD)

68.7 (4.3)

BMI, kg/m2, mean (SD)

26.0 (4.3)

   Underweight BMI (<18.5 kg/m2), number (%)

2 (0.7%)

   Normal BMI (18.5 to 24.9 kg/m2), number (%)

126 (43.9%)

   Overweight BMI (25 to 29.9 kg/m2), number (%)

124 (43.2%)

   Obese BMI (=30 kg/m2), number (%)

35 (12.2%)

Estimates of BMI of respondent's ICU patients, mean (SD)

 

   Underweight BMI (<18.5 kg/m2)

10.4% (6.5)

   Normal BMI (18.5 to 24.9 kg/m2)

26.8% (13.5)

   Overweight BMI (25 to 29.9 kg/m2)

30.6% (11.7)

   Obese BMI (30 to 39.9 kg/m2)

22.8% (11.7)

   Severely obese BMI (=40 kg/m2)

9.4% (6.5)

Self-reported chronic health problem, number (%)

45 (16.2%)

Percentage of job spent in direct care of ICU patients, mean (SD)

38.8% (22.2)

Number of septic patients cared for per week, number (%)

 

   0

3 (1.0%)

   1 to 2

62 (21.7%)

   3 to 5

115 (40.2%)

   6 to 10

71 (24.8%)

   11 to 15

24 (8.4%)

   15+

11 (3.8%)

Self-rated experience treating sepsis, number (%)

 

   Limited

1 (0.4%)

   Moderate

109 (38.0%)

   Extensive

177 (61.7%)

Specialty, number (%)

 

   Internal medicine

154 (54.0%)

   Family medicine

1 (0.4%)

   Surgery

3 (1.0%)

   Anesthesia

3 (1.0%)

   Pulmonary

267 (93.0%)

   Critical care

262 (91.3%)

   Sleep medicine

66 (23.0%)

   Other

13 (4.5%)

BMI = body mass index; ICU = intensive care unit; SD = standard deviation.

Predicted probability of baseline hospital mortality

For all patients described in the vignettes, the median baseline mortality (the predicted hospital mortality if no additional therapies were added after ICU admission) was 47% (range from 5th to 95th percentile 17% to 78%). When grouped by vignette, the ranges of mortality estimates remained wide (Table 2). For each respondent, the average difference between the highest and lowest baseline mortality prediction was 24.9 percentage points (95% confidence interval (CI) 23.2 to 26.7 percentage points). Despite identical APACHE II scores and organ failure, older age, early-stage lung cancer, and an obese BMI were all associated with higher predictions of baseline mortality (Table 3). No measured respondent factors were associated with the baseline mortality prediction.
Table 2

Predicted hospital mortality, based on clinical factors in study vignettes

Vignette characteristics

Predicted 'baseline' hospital mortality

BMI

Age (years)

Co-morbidities

APACHE II Score

Number of vignettes

Median

5th percentile to 95th percentile range

Normal

50

None

25

285

37%

10% to 69%

Obese

50

None

25

133

42%

16% to 74%

Normal

50

Early-stage lung cancer

25

43

44%

21% to 72%

Normal

70

None

25

123

46%

15% to 74%

Obese

50

Early-stage lung cancer

25

42

49%

24% to 80%

Obese

70

None

25

130

50%

28% to 80%

Normal

70

Early-stage lung cancer

25

98

53.5%

20% to 84%

Obese

70

Early-stage lung cancer

25

287

55%

27% to 81%

All vignettes

1141

47%

17% to 78%

Respondents were asked to provide estimates of hospital mortality (see Methods for details). The median and central 90% of responses are shown based on vignette characteristics.

APACHE II = acute physiology, age and chronic health evaluation II; BMI = body mass index.

Table 3

Patient factors in vignettes and predicted 'baseline' mortality

 

Univariable analyses

Multivariable analyses

 

Percentage point increase in predicted mortality (95% confidence interval)

P value

Percentage point increase in predicted mortality (95% confidence interval)

P value

70 years old

(versus 50 years old)

12.1

(10.0 to 14.2)

<0.0001

8.2

(6.1 to 10.4)

<0.0001

Stage IIA NSCLC

(versus no cancer)

10.8

(8.7 to 13.0)

<0.0001

5.9

(3.6 to 8.1)

<0.0001

BMI 40 kg/m2

(versus 22 kg/m2)

8.6

(6.4 to 10.7)

<0.0001

4.3

(2.5 to 6.2)

<0.0001

Baseline mortality was considered the predicted mortality if no additional care, other than the care instituted prior to admission to the intensive care unit (ICU), was added. The univariable estimates include only the variable indicated in the model while multivariable estimates included all variables with displayed estimates in that column. All analyses accounted for non-independence of responses due to respondents completing multiple vignettes.

BMI = body mass index; NSCLC = non-small cell lung carcinoma.

Predicted probability of problems with self-care among survivors

For all patients described in the vignettes, the median predicted rate of problems among survivors with washing and dressing oneself was 25% (range from 5th to 95th percentile 2% to 74%). As with the baseline mortality predictions, among vignettes with identical patient factors, these ranges of predictions were wide (Table 4). Older age, early-stage lung cancer, and an obese BMI were all associated with higher probabilities of problems with self-care at six months among survivors (Table 5). After adjustment for the clinical factors in the vignettes, respondents who were older and reported chronic health problems predicted fewer problems with self-care for surviving patients than respondents who were younger and who had no health problems (Table 6). After adjustment for these respondent factors, higher BMI, older age, and a cancer diagnosis continued to be associated with higher predicted difficulties with self-care among survivors.
Table 4

Predicted problems with self-care, based on clinical factors in study vignettes

Vignette characteristics

Predicted problems washing and dressing self at six months (assuming survival)

BMI

Age (years)

Co-morbidities

APACHE II Score

Number of vignettes

Median

5th percentile to 95th percentile range

Normal

50

None

25

285

12%

1% to 58%

Obese

50

None

25

133

20%

3% to 66%

Normal

50

Early-stage lung cancer

25

43

25%

3% to 55%

Normal

70

None

25

124

25%

4% to 69%

Obese

50

Early-stage lung cancer

25

42

28.5%

2% to 70%

Obese

70

None

25

130

32%

4% to 74%

Normal

70

Early-stage lung cancer

25

98

31.5%

4% to 77%

Obese

70

Early-stage lung cancer

25

287

39%

8% to 81%

All vignettes

   

1142

25%

2% to 74%

Respondents were asked to provide estimates of problems washing and dressing himself at six months, assuming the patient survived (see Methods for details). The median and central 90% of responses are shown based on vignette characteristics. APACHE II = acute physiology, age and chronic health evaluation II; BMI = body mass index.

Table 5

Patient factors in vignettes and predicted problems with self-care, univariable analyses

 

Univariable analyses

 

Percentage point increase in predicted problems with self-care in six months (95% confidence interval)

P value

70 years old

(versus 50 years old)

16.1

(14.0 to 18.1)

<0.0001

Stage IIA NSCLC

(versus no cancer)

13.5

(11.3 to 15.7)

<0.0001

BMI 40 kg/m2

(versus 22 kg/m2)

10.9

(9.2 to 12.6)

<0.0001

Respondent age (per decade of age)

-4.5

(-6.5 to -2.5)

<0.0001

Respondent self-reported chronic health condition

-7.3

(-11.8 to -2.8)

0.0016

Respondents were asked to predict the probability of each patient having difficulties with washing and dressing himself in six months, assuming the patient survived. Univariable estimates include only the variable indicated in the model. Analyses accounted for non-independence of responses due to respondents completing multiple vignettes.

BMI = body mass index; NSCLC = non-small cell lung carcinoma.

Table 6

Patient factors in vignettes and predicted problems with self-care, multivariable analyses

 

Multivariable analysis

Multivariable analysis, including respondent factors

 

Percentage point increase in predicted problems with self-care in six months (95% confidence interval)

P value

Percentage point change in predicted problems with self-care in six months (95% confidence interval)

P value

70 years old

(versus 50 years old)

11.5

(9.0 to 13.9)

<0.0001

11.6

(9.2 to 13.9)

<0.0001

Stage IIA NSCLC

(versus no cancer)

6.8

(4.3 to 9.3)

<0.0001

6.9

(4.3 to 9.4)

<0.0001

BMI 40 kg/m2

(versus 22 kg/m2)

5.4

(3.3 to 7.5)

<0.0001

5.3

(3.2 to 7.4)

<0.0001

Respondent age (per decade of age)

  

-4.0

(-6.0 to -2.0)

0.0001

Respondent self-reported chronic health condition

  

-5.8

(-10.1 to -1.4)

0.0103

Respondents were asked to predict the probability of each patient having difficulties with washing and dressing himself in six months, assuming the patient survived. Multivariable estimates included all variables with displayed estimates in that column. All analyses accounted for non-independence of responses due to respondents completing multiple vignettes.

BMI = body mass index; NSCLC = non-small cell lung carcinoma.

Recommendations to limit care with curative intent

Limitation of care with curative intent was suggested in 9.1% of vignettes. Most commonly, a DNR order alone (78.4% of those with limitation recommendation) was suggested. In univariable analyses, early-stage lung cancer, older age, an obese BMI and predictions of increased baseline mortality and problems with self-care were associated with limitations of care suggestions (Table 7). In multivariable analyses accounting for other vignette factors, an obese BMI was not associated with limitation of care (Table 8). Once adjusted for predictions about mortality and problems with self-care, older age was also not associated with suggestions to limit care. In the final multivariable model, every 10% increase in predicted baseline mortality and in predicted problems with self-care was independently associated with 29% and 26% increased odds of limitation of care, respectively. A cancer diagnosis was associated with nearly six-fold increased odds of limitation of care in the final multivariable model. In other words, respondents were significantly more likely to recommend limitations in aggressive care for a patient with early-stage lung cancer compared with one without cancer, even when the vignettes had identical mortality and morbidity predictions. Respondents with BMIs suggesting overweight or obesity were significantly less likely to suggest a limitation of care order.
Table 7

Factors associated with suggested limitation of care orders, univariable analysis

 

Univariable analyses

 

Odds ratio (95% CI)

P value

70 years old

(versus 50 years old)

6.76

(3.96 to 11.55)

<0.0001

Stage IIA NSCLC (versus no cancer)

10.95

(6.30 to 19.03)

<0.0001

BMI 40 kg/m2

(versus 22 kg/m2)

2.54

(1.78 to 3.62)

<0.0001

Baseline predicted hospital mortality (per 10% increase)

1.63

(1.41 to 1.89)

<0.0001

Predicted problems with self-care at six months (per 10% increase)

1.46

(1.32 to 1.62)

<0.0001

Overweight or obese respondent BMI

0.55

(0.33 to 0.93)

0.0258

Limitations of care included suggesting a 'do not resuscitate' order, that there be no further escalation of care (e.g., no addition of vasopressors), and/or termination of supportive care with appropriate 'comfort care' measures. Mortality predictions were estimated prior to any additional chosen care, including limitation of care orders. Respondents were asked to predict the ability to perform self-care (wash and dress oneself) in six months, assuming survival. Univariable estimates include only the variable indicated in the model. Analyses accounted for non-independence of responses due to respondents completing multiple vignettes.

BMI = body mass index; CI = confidence interval; NSCLC = non-small cell lung carcinoma.

Table 8

Factors associated with suggested limitation of care orders, multivariable analyses

 

Multivariable analyses

 

Adjusted odds ratio (95% CI)

P value

Adjusted odds ratio (95% CI)

P value

Adjusted odds ratio (95% CI)

P value

70 years old

(versus 50 years old)

2.90

(1.55 – 5.44)

0.0009

1.90

(0.98 to 3.68)

0.0589

1.87

(0.95 to 3.66)

0.0685

Stage IIA NSCLC

(versus no cancer)

7.12

(3.75 – 13.52)

<0.0001

5.70

(2.97 to 10.93)

<0.0001

5.84

(3.05 to 11.20)

<0.0001

BMI 40 kg/m2

(versus 22 kg/m2)

1.18

(0.78 – 1.79)

0.4274

1.02

(0.66 to 1.57)

0.9375

1.01

(0.65 to 1.56)

0.9694

Baseline predicted hospital mortality (per 10% increase)

  

1.30

(1.10 to 1.54)

0.0019

1.29

(1.09 to 1.53)

0.0027

Predicted problems with self-care at six months (per 10% increase)

  

1.26

(1.12 to 1.41)

<0.0001

1.26

(1.12 to 1.42)

0.0002

Overweight or obese respondent BMI

    

0.53

(0.29 to 0.96)

0.0345

Limitations of care included suggesting a 'do not resuscitate' order, that there be no further escalation of care (e.g., no addition of vasopressors), and/or termination of supportive care with appropriate 'comfort care' measures. Mortality predictions were estimated prior to any additional chosen care, including limitation of care orders. Respondents were asked to predict the ability to perform self-care (wash and dress oneself) in six months, assuming survival. Multivariable estimates included all variables with displayed estimates in that column. All analyses accounted for non-independence of responses due to respondents completing multiple vignettes.

BMI = body mass index; CI = confidence interval; NSCLC = non-small cell lung carcinoma.

Because of the generally poor outcome for septic patients requiring cardiopulmonary resuscitation (21), some respondents might not consider a DNR order as a change in the goals of care. We recalculated our analyses considering only limitations of supportive care that included a non-escalation order and/or a change to comfort care (n = 22, 1.96% of vignettes). The results of these analyses were very similar in magnitude and direction to those including DNR as a limitation of care with curative intent (data not shown), although respondent BMI was no longer associated with the limitation of care.

Discussion

In this mail-based survey of physicians with experience caring for septic patients, physician predictions about hospital mortality in septic shock varied widely, even when clinical information was identical. Beyond this variability, older age, an obese BMI, and cancer diagnosis were associated with predictions for greater mortality and morbidity. These findings suggest that physicians incorporate clinical factors into their estimates, which are independent of validated severity of illness scores. These prognostic estimates and the hypothetical patient's diagnosis of early-stage lung cancer were also associated with recommendations to limit care.

Severity of illness scoring systems were developed in an attempt to objectively quantify the risk of hospital mortality to 'evaluate the outcomes of care' [17]. These systems, however, were not designed for prognostication of individual patients [18]. They also may have less ability to discriminate between survivors and nonsurvivors than ICU physicians, although discriminatory capacity is only moderate among ICU physicians [19]. Despite these limitations, physicians are advised to provide prognostic information and recommendations about appropriate treatments and goals of care by the American College of Critical Care [4] and the Surviving Sepsis Campaign [5]. These predictions are then influential on subsequent care and outcome [68]. The patient and provider factors that color the information provided by ICU physicians are largely unknown. By better understanding these factors, it may allow for the development of interventions that should be directed at the patient's illness and ones which should be directed at providing more accurate tools for discriminating outcomes for individual patients. Differences in provider tendencies in prognostication and communication with patients and families could affect the results of observational studies as well. Although adjusting for differences in the clinical status of patients is common, most studies do not incorporate physician predictions or even patient preferences about continued life support in studies of risk factors for outcomes from critical illness.

When presented with identical clinical data, individual physicians experienced in treating sepsis made dramatically different estimates of mortality. The narrowest range (5th to 95th percentile of values) of predictions across respondents was 51 percentage points. In other words, one would not be surprised if two physicians, presented with the same information, would provide estimates of mortality that differed by more than 50 percentage points. Such prognostic variation and disagreement have been reported previously [20] and could influence the expectations of recovery each physician communicates to patients and families. Although we collected limited information about respondents, no measured factor appeared to consistently explain why a respondent might be more optimistic or pessimistic about hospital survival. Older respondents and those with a chronic health condition had more optimistic predictions about the ability of survivors to be independent at six months. This observation raises the possibility that a physician's expectations of recovery are influenced by his or her own health status. Further study should evaluate respondent factors that drive physician predictions and that affect subsequent decisions about continued aggressive care.

Despite identical acute severity of illness measures, respondents predicted poorer short-term outcomes for patients with high BMIs, older age, or limited-stage lung cancer. These findings suggest that physicians use information beyond that contained in severity of illness systems to generate estimates of proximate outcomes for septic shock patients. As physician prognostication may be equivalent or superior to that supplied by severity of illness systems [19], inclusion of these clinical factors may be appropriate. However, their potential prognostic relevance does not provide rationale for the observed variability in predictions.

Beyond provided prognostic information, recommendations regarding the value of continued aggressive care may influence ultimate outcome and not merely hasten the time to certain death. Those with limitation of care orders have higher risk-adjusted mortality for at least one year after ICU admission [21]. We found that poorer expected prognoses were associated with greater odds of recommending a limitation of care with curative intent. Older age and early-stage lung cancer were also associated with higher odds of a suggestion to limit care with curative intent. In the case of the older vignettes, this was mediated by expectations of poorer outcomes. However, even after considering its higher associated estimates of mortality and morbidity, early-stage lung cancer was associated with nearly six-fold increased odds of limitation of care suggestions. Although this may be partly explained by other outcome predictions unmeasured in this study (e.g., increased longer-term mortality among those surviving sepsis), the magnitude of this association is consistent with a higher perceived mortality for lung cancer patients than is supported by existing data [2224]. We do not imply that the observation of higher rates of suggestions to limit care necessarily represents an inappropriate recommendation. Some studies suggest that general severity of illness systems (such as APACHE II) perform poorly for cancer patients in the ICU and may be overly optimistic, compared with systems developed specifically for ICU patients with cancer [25]. However, we suspect that if respondents were influenced by such inaccuracies for cancer patients, the association between recommendations to limit care and cancer diagnosis would be mediated by higher estimates of mortality, rather than being independent of these predictions.

There are important limitations to our study which limit its applicability to actual clinical practice and communications with families. Case-based vignettes are a simulated clinical situation and may not reflect predictions made about real patients. However, vignette-based studies have been found to be a valid measure of delivered care [26, 27]. We forced respondents to provide prognostic information early in the clinical course. Although it is possible that early predictions lose relevance, one study suggests that events 48 hours after ICU admission have little effect on mortality predictions, compared with those made at ICU admission [28]. Also, the majority of surrogate decision-makers seek prognostic information early in a patient's illness, even in the face of uncertainty [3], making these early predictions more relevant. We also used a visual-analogue scale to measure respondent predictions. Although this method has been used for many studies and is an element of validated tools, such as the EuroQol-5D, it has not been specifically validated for physician predictions about septic patient vignettes.

We did not allow respondents to comment on the confidence each had in his or her predictions. Such questions would have allowed us to determine if a respondent felt confident enough to make a prediction about ultimate outcome and if he felt the estimates by another respondent were likely or not. A prior vignette-based study found that confidence in recommendations about care (ranging from 'comfort only' to 'full aggressive care') was higher among intensivists that nurses or residents and among respondents choosing care at one of the two extremes [26]. However, considerable disagreement between respondents remained even when respondents were highly confident. We also did not measure estimates of longer-term mortality, which some might argue is more relevant to decisions about continued ICU care. However, proximate measures of survival, including hospital mortality, have been accepted measures of efficacy of therapies in critically ill patients [29, 30]. Our results also suggest that even such short-term prognostic estimates are associated with recommendations to limit care with curative intent.

Generalizability of our findings beyond those forming the study cohort is unknown, especially for clinicians who do not practice in the USA, those who do not regularly care for ICU patients, non-medical intensivists, or non-physician providers. We cannot comment on the potential influences of patient factors other than those controlled for in the vignettes on physician predictions and decision-making. A BMI of 40 kg/m2 may be less compelling when written as part of a case than when it is observed in an ICU and, thus, we may have underestimated the influence that patient obesity has on physician predictions. We also cannot comment on the influence of unstudied respondent factors, such as ethnicity and religious affiliation, which might affect recommendations to limit aggressive care [31]. Our response rate was below our projections, but it exceeded the reported rates of many mail-based survey studies involving physicians [32, 33]. By incorporating randomization, a non-responder was as likely to receive a vignette as a responder, reducing the likelihood of biased results.

Conclusions

Given the wide range in predictions about mortality and morbidity and their association with recommendations for limitation of care, future research should focus on the patient and provider factors that produce such disparate predictions about outcomes. This is of particular importance in situations in which variation in predictions is associated with subsequent differences in provided care. For example, better tools to aid physician prognostication could reduce variation in such estimates and result in more uniform recommendations about continued aggressive care. Although severity of illness systems are attempts to provide such consistency, they ignore the additional information incorporated by a bedside clinician. Additional study is needed to better understand these subtleties that consistently (and inconsistently) influence physician predictions and practices. Without attending to the role of the provider in patient outcomes, we ignore aspects of the therapeutic relationship which may be more easily modified than patient characteristics and the severity of his/her acute illness.

Key messages

  • Among physicians experienced in caring for patients with septic shock, predictions about mortality and morbidity vary widely.

  • Older age, high BMI, and early-stage lung cancer are associated with poorer predictions of mortality and morbidity, independent of acute severity of illness.

  • Poorer outcome predictions are associated with an increased likelihood of a clinician suggesting a limitation of care order.

  • Early-stage lung cancer is associated with higher odds of a suggestion of a limitation of care order, independent of predictions about mortality and morbidity.

Abbreviations

APACHE II: 

acute physiology, age and chronic health evaluation II

BMI: 

body mass index

CI: 

confidence interval

DNR: 

do not resuscitate

ICU: 

intensive care unit.

Declarations

Acknowledgements

The authors wish to thank Jordi Mancebo, MD, Sheryl Vega, and Monica Simeonova from the American Thoracic Society for their assistance and Mark Kearns, MD, Melissa Slivanya, Roxann Damron, and Shawn Long for preparing and mailing the survey. JMO is supported by the Davis/Bremer Medical Research Grant and NIH K23 HL075076.

Authors’ Affiliations

(1)
Division of Pulmonary, Allergy,Department of Internal Medicine, Critical Care and Sleep Medicine, Center for Critical Care,The Ohio State University Medical Center
(2)
Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Johns Hopkins School of Medicine
(3)
College of Public Health, The Ohio State University

References

  1. Martin GS, Mannino DM, Eaton S, Moss M: The epidemiology of sepsis in the United States from 1979 through 2000. N Engl J Med 2003, 348: 1546-1554. 10.1056/NEJMoa022139View ArticlePubMedGoogle Scholar
  2. Angus DC, Linde-Zwirble WT, Lidicker J, Clermont G, Carcillo J, Pinsky MR: Epidemiology of severe sepsis in the United States: analysis of incidence, outcome, and associated costs of care. Crit Care Med 2001, 29: 1303-1310. 10.1097/00003246-200107000-00002View ArticlePubMedGoogle Scholar
  3. Evans LR, Boyd EA, Malvar G, Apatira L, Luce JM, Lo B, White DB: Surrogate decision-makers' perspectives on discussing prognosis in the face of uncertainty. Am J Respir Crit Care Med 2009, 179: 48-53. 10.1164/rccm.200806-969OCPubMed CentralView ArticlePubMedGoogle Scholar
  4. Truog RD, Campbell ML, Curtis JR, Haas CE, Luce JM, Rubenfeld GD, Rushton CH, Kaufman DC: Recommendations for end-of-life care in the intensive care unit: a consensus statement by the American College [corrected] of Critical Care Medicine. Crit Care Med 2008, 36: 953-963. 10.1097/CCM.0B013E3181659096View ArticlePubMedGoogle Scholar
  5. Dellinger RP, Levy MM, Carlet JM, Bion J, Parker MM, Jaeschke R, Reinhart K, Angus DC, Brun-Buisson C, Beale R, Calandra T, Dhainaut JF, Gerlach H, Harvey M, Marini JJ, Marshall J, Ranieri M, Ramsay G, Sevransky J, Thompson BT, Townsend S, Vender JS, Zimmerman JL, Vincent JL: Surviving Sepsis Campaign: International guidelines for management of severe sepsis and septic shock: 2008. Crit Care Med 2008, 36: 296-327. 10.1097/01.CCM.0000298158.12101.41View ArticlePubMedGoogle Scholar
  6. Cook D, Rocker G, Marshall J, Sjokvist P, Dodek P, Griffith L, Freitag A, Varon J, Bradley C, Levy M, Finfer S, Hamielec C, McMullin J, Weaver B, Walter S, Guyatt G: Withdrawal of mechanical ventilation in anticipation of death in the intensive care unit. N Engl J Med 2003, 349: 1123-1132. 10.1056/NEJMoa030083View ArticlePubMedGoogle Scholar
  7. Sinuff T, Cook DJ, Rocker GM, Griffith LE, Walter SD, Fisher MM, Dodek PM, Sjokvist P, McDonald E, Marshall JC, Kraus PA, Levy MM, Lazar NM, Guyatt GH: DNR directives are established early in mechanically ventilated intensive care unit patients. Can J Anaesth 2004, 51: 1034-1041. 10.1007/BF03018494View ArticlePubMedGoogle Scholar
  8. Rocker G, Cook D, Sjokvist P, Weaver B, Finfer S, McDonald E, Marshall J, Kirby A, Levy M, Dodek P, Heyland D, Guyatt G: Clinician predictions of intensive care unit mortality. Crit Care Med 2004, 32: 1149-1154. 10.1097/01.CCM.0000126402.51524.52View ArticlePubMedGoogle Scholar
  9. O'Brien JM Jr, Welsh CH, Fish RH, Ancukiewicz M, Kramer AM: Excess body weight is not independently associated with outcome in mechanically ventilated patients with acute lung injury. Ann Intern Med 2004, 140: 338-345.View ArticlePubMedGoogle Scholar
  10. O'Brien JM Jr, Phillips GS, Ali NA, Lucarelli M, Marsh CB, Lemeshow S: Body mass index is independently associated with hospital mortality in mechanically ventilated adults with acute lung injury. Crit Care Med 2006, 34: 738-744.PubMed CentralPubMedGoogle Scholar
  11. Teachman BA, Brownell KD: Implicit anti-fat bias among health professionals: is anyone immune? Int J Obes Relat Metab Disord 2001, 25: 1525-1531. 10.1038/sj.ijo.0801745View ArticlePubMedGoogle Scholar
  12. Kaminsky J, Gadaleta D: A study of discrimination within the medical community as viewed by obese patients. Obes Surg 2002, 12: 14-18. 10.1381/096089202321144513View ArticlePubMedGoogle Scholar
  13. Akinnusi ME, Pineda LA, El Solh AA: Effect of obesity on intensive care morbidity and mortality: a meta-analysis. Crit Care Med 2008, 36: 151-158. 10.1097/01.CCM.0000297885.60037.6EView ArticlePubMedGoogle Scholar
  14. Hogue CW Jr, Stearns JD, Colantuoni E, Robinson KA, Stierer T, Mitter N, Pronovost PJ, Needham DM: The impact of obesity on outcomes after critical illness: a meta-analysis. Intensive Care Med 2009, 35: 1152-1170. 10.1007/s00134-009-1424-5View ArticlePubMedGoogle Scholar
  15. Hamel MB, Davis RB, Teno JM, Knaus WA, Lynn J, Harrell F Jr, Galanos AN, Wu AW, Phillips RS: Older age, aggressiveness of care, and survival for seriously ill, hospitalized adults. SUPPORT Investigators. Study to Understand Prognoses and Preferences for Outcomes and Risks of Treatments. Ann Intern Med 1999, 131: 721-728.View ArticlePubMedGoogle Scholar
  16. Scott WJ, Howington J, Feigenberg S, Movsas B, Pisters K: Treatment of non-small cell lung cancer stage I and stage II: ACCP evidence-based clinical practice guidelines (2nd edition). Chest 2007, 132: 234S-242S. 10.1378/chest.07-1378View ArticlePubMedGoogle Scholar
  17. Knaus WA: APACHE 1978–2001: the development of a quality assurance system based on prognosis: milestones and personal reflections. Arch Surg 2002, 137: 37-41. 10.1001/archsurg.137.1.37View ArticlePubMedGoogle Scholar
  18. Lemeshow S, Klar J, Teres D: Outcome prediction for individual intensive care patients: useful, misused, or abused? Intensive Care Med 1995, 21: 770-776. 10.1007/BF01704747View ArticlePubMedGoogle Scholar
  19. Sinuff T, Adhikari NK, Cook DJ, Schunemann HJ, Griffith LE, Rocker G, Walter SD: Mortality predictions in the intensive care unit: comparing physicians with scoring systems. Crit Care Med 2006, 34: 878-885. 10.1097/01.CCM.0000201881.58644.41View ArticlePubMedGoogle Scholar
  20. Poses RM, Bekes C, Copare FJ, Scott WE: The answer to "What are my chances, doctor?" depends on whom is asked: prognostic disagreement and inaccuracy for critically ill patients. Crit Care Med 1989, 17: 827-833. 10.1097/00003246-198908000-00021View ArticlePubMedGoogle Scholar
  21. Chen YY, Connors AF Jr, Garland A: Effect of decisions to withhold life support on prolonged survival. Chest 2008, 133: 1312-1318. 10.1378/chest.07-1500View ArticlePubMedGoogle Scholar
  22. Adam AK, Soubani AO: Outcome and prognostic factors of lung cancer patients admitted to the medical intensive care unit. Eur Respir J 2008, 31: 47-53. 10.1183/09031936.00031607View ArticlePubMedGoogle Scholar
  23. Reichner CA, Thompson JA, O'Brien S, Kuru T, Anderson ED: Outcome and code status of lung cancer patients admitted to the medical ICU. Chest 2006, 130: 719-723. 10.1378/chest.130.3.719View ArticlePubMedGoogle Scholar
  24. Lin YC, Tsai YH, Huang CC, Hsu KH, Wang SW, Tsao TC, Lin MC: Outcome of lung cancer patients with acute respiratory failure requiring mechanical ventilation. Respir Med 2004, 98: 43-51. 10.1016/j.rmed.2003.07.009View ArticlePubMedGoogle Scholar
  25. den Boer S, de Keizer NF, de Jonge E: Performance of prognostic models in critically ill cancer patients – a review. Crit Care 2005, 9: R458-R463. 10.1186/cc3765PubMed CentralView ArticlePubMedGoogle Scholar
  26. Walter SD, Cook DJ, Guyatt GH, Spanier A, Jaeschke R, Todd TR, Streiner DL: Confidence in life-support decisions in the intensive care unit: a survey of healthcare workers. Canadian Critical Care Trials Group. Crit Care Med 1998, 26: 44-49. 10.1097/00003246-199801000-00015View ArticlePubMedGoogle Scholar
  27. Diette GB, Patino CM, Merriman B, Paulin L, Riekert K, Okelo S, Thompson K, Krishnan JA, Quartey R, Perez-Williams D, Rand C: Patient factors that physicians use to assign asthma treatment. Arch Intern Med 2007, 167: 1360-1366. 10.1001/archinte.167.13.1360View ArticlePubMedGoogle Scholar
  28. Poses RM, Bekes C, Copare FJ, Scott WE: What difference do two days make? The inertia of physicians' sequential prognostic judgments for critically ill patients. Med Decis Making 1990, 10: 6-14. 10.1177/0272989X9001000103View ArticlePubMedGoogle Scholar
  29. Ventilation with lower tidal volumes as compared with traditional tidal volumes for acute lung injury and the acute respiratory distress syndrome. The Acute Respiratory Distress Syndrome Network N Engl J Med 2000, 342: 1301-1308. 10.1056/NEJM200005043421801Google Scholar
  30. Bernard GR, Macias WL, Joyce DE, Williams MD, Bailey J, Vincent JL: Safety assessment of drotrecogin alfa (activated) in the treatment of adult patients with severe sepsis. Crit Care 2003, 7: 155-163. 10.1186/cc2167PubMed CentralView ArticlePubMedGoogle Scholar
  31. Sprung CL, Maia P, Bulow HH, Ricou B, Armaganidis A, Baras M, Wennberg E, Reinhart K, Cohen SL, Fries DR, Nakos G, Thijs LG: The importance of religious affiliation and culture on end-of-life decisions in European intensive care units. Intensive Care Med 2007, 33: 1732-1739. 10.1007/s00134-007-0693-0View ArticlePubMedGoogle Scholar
  32. Asch DA, Jedrziewski MK, Christakis NA: Response rates to mail surveys published in medical journals. J Clin Epidemiol 1997, 50: 1129-1136. 10.1016/S0895-4356(97)00126-1View ArticlePubMedGoogle Scholar
  33. Cummings SM, Savitz LA, Konrad TR: Reported response rates to mailed physician questionnaires. Health Serv Res 2001, 35: 1347-1355.PubMed CentralPubMedGoogle Scholar

Copyright

© O'Brien Jr et al.; licensee BioMed Central Ltd. 2009

This article is published under license to BioMed Central Ltd. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.