Identifying critically ill patients most likely to benefit from pre-emptive therapies will become increasingly important if therapies are to be used safely and cost-effectively. We sought to determine whether a predictive model could be constructed that would serve as a useful decision support tool for the pre-emptive management of intensive care unit (ICU)-related anemia.
Our cohort consisted of all ICU patients (n = 5,170) admitted to a large tertiary-care academic medical center during the period from 1 July 2000 to 30 June 2001. We divided the cohort into development (n = 3,619) and validation (n = 1,551) sets. Using a set of demographic and physiologic variables available within six hours of ICU admission, we developed models to predict patients who either received late transfusion or developed late anemia. We then constructed a point system to quantify, within six hours of ICU admission, the likelihood of developing late anemia.
Models showed good discrimination with receiver operating characteristic curve areas ranging from 0.72 to 0.77, although predicting late transfusion was consistently less accurate than predicting late anemia. A five-item point system predicted likelihood of late anemia as well as existing clinical trial inclusion criteria but resulted in pre-emptive intervention more than two days earlier.
A rule-based decision support tool using information available within six hours of ICU admission may lead to earlier and more appropriate use of blood-sparing strategies.
Most intensive care therapies are prescribed in response to the development of organ dysfunction or other unwanted events. For example, antibiotics are prescribed when a patient develops sepsis, or tidal volumes are lowered when a patient develops acute respiratory distress syndrome (ARDS) . In other arenas of medicine, therapies are much more commonly provided to populations who are at risk of unwanted events, in the hope of future benefit. For example, antihypertensives and lipid-lowering agents are given to prevent cardiovascular disease long before any hint of the disease has ever manifested. It is likely that similar pre-emptive strategies will reap gains in the intensive care unit (ICU). For example, it seems that use of low tidal volumes in patients at risk of ARDS reduces the likelihood of their subsequently developing ARDS . However, critically ill patients are fragile and complex, so broad use of early therapies with a theoretical hope of avoiding or mitigating downstream consequences will probably need to be balanced against the costs and risks of the therapies themselves. Thus, strategies to identify the candidates most likely to benefit will become increasingly important if therapies are to be used safely and cost-effectively.
In this article we chose anemia of critical illness as an illustrative example of a condition that one might want to manage pre-emptively. Trials are currently ongoing to assess whether treatment with recombinant human erythropoietin (rHuEPO) can mitigate anemia-associated morbidity or mortality. If the outcome of these trials is positive, we will be faced with an interesting challenge. Specifically, we will have a therapy that is expensive, associated with small but potentially important side effects, unlikely to offer any benefit to patients who have short ICU courses and are at no risk of anemia of critical illness, and yet only able to produce significant improvements in serum hemoglobin levels after several days of treatment. Therefore, to be most effective it must be given early in the ICU course, and to be most efficient it must be given only to the subset of ICU patients who will end up incurring a prolonged hospital stay and developing anemia. The dilemma of finding the right subset of patients before they have developed the outcome of interest is a classic decision-making problem, with similarities to other situations that may soon arise in critical care, such as early tracheostomy  or the prophylactic use of antidepressants .
The goal of this paper was to determine whether a predictive model could be constructed that would serve as a useful decision support tool for the pre-emptive management of ICU-related anemia. We stress that this exercise is only theoretical at this point, because there is insufficient evidence to recommend rHuEPO as therapy for anemia of critical illness. However, we believe the exercise sheds light on the potential opportunities for similar models for other pre-emptive management strategies in the ICU.
Materials and methods
Our approach was to build and validate a model-based rule that used information available within six hours of ICU admission to predict the likelihood that a patient would have a prolonged hospital course complicated by anemia. We then compared the potential consequences of a prescription strategy that relied on the rule with those of a strategy that uses the existing clinical trial inclusion criteria  and with those of a broad-based ICU prescription strategy.
Patients and data sources
For our study cohort we selected all ICU patients admitted during the period from 1 July 2000 to 30 June 2001 at the University of Pittsburgh Medical Center, a hospital with more than 120 ICU beds caring for adult medical, surgical, trauma, neurologic, and solid-organ transplant patients. We obtained information from several computerized databases, including administrative discharge data, laboratory data, itemized billing records, and a detailed bedside database containing the following prospectively collected data for all ICU patients: demographics, admission diagnosis, physiologic data, and interventions (medications, fluids and blood products, mechanical ventilation, and renal replacement). To preserve patient confidentiality, all data were de-identified and linked by an honest broker. The data were cleaned (purged of inconsistent and/or nonsense values), organized, and merged to create files for the analysis. We then generated composite variables, such as the presence of organ dysfunction, from detailed physiologic data. Data were manipulated as a relational database in Microsoft FoxPro 6.0. The University of Pittsburgh Institutional Review Board approved the study.
Modeling and statistical analyses
The cohort was divided into development (two-thirds of cohort) and validation (one-third of cohort) sets. Using a set of demographic and physiologic variables available within six hours of ICU admission, we developed two predictive models using logistic regression. The first model was developed to identify patients who received 'late transfusion', namely those who received red cell transfusion(s) 7 days or more after ICU admission. The second model was developed to identify patients who developed 'late anemia', namely those whose actual or predicted hemoglobin reached 7 g/dl or less at seven days or more after ICU admission. The threshold of 7 g/dl was based on the randomized controlled trial by Hebert and colleagues , which found a lower hospital mortality for subjects transfused below this value compared with a more liberal (10 g/dl) transfusion threshold. For patients with pre-transfusion hemoglobin concentrations of more than 7 g/dl, we extrapolated these values to determine who would have reached 7 g/dl by using the mean of individual slopes of hemoglobin concentrations. Individual slopes were constructed from the calculated day-to-day decrease in hemoglobin in patients not receiving transfusions .
Independent predictors eligible for inclusion in the two models were as follows: age (less than 65 years, 65 years or more), gender, race (white, black or other), comorbidity , body mass index, medical or surgical admission, emergency surgery, trauma, and the following data obtained within six hours of ICU admission: minimum hemoglobin, red cell transfusion (yes/no), creatinine (less than 1.6 mg/dl, 1.6 mg/dl or more), international normalized ratio (1.9 or less, more than 1.9), arterial blood lactate (1.5 mg/dl or less, more than 1.5 mg/dl), and need for mechanical ventilation or vasopressors. Model discrimination and calibration were tested by using the receiver operator characteristic (ROC) curve area and the Hosmer–Lemeshow test. In secondary analyses, we developed models using data available within 24, 48, and 72 hours of ICU admission. Using the 6-hour model for late anemia, we constructed a point system to quantify, on the first ICU day, the likelihood of their reaching a hemoglobin value of 7 g/dl or less at 7 days or more after ICU admission and therefore being a suitable candidate for a blood-sparing strategy such as rHuEPO. We then compared the potential consequences of a prescription strategy that relied on the rule with those of a strategy that uses the existing clinical trial inclusion criteria (ICU length of stay 3 days or more, age 18 years or more, and hematocrit less than 38%)  and with those of a broad-based ICU prescription strategy. All data analyses were performed and models developed with SPSS 11.5 (Chicago, IL, USA). We assumed statistical significance for differences between groups or model coefficients at p < 0.05.
A total of 5,170 ICU patients were admitted from 1 July 2000 to 30 June 2001 (Table 1), with 887 (17.2%) and 1,034 (20.0%) in the late transfusion and late anemia groups, respectively. There was a fair degree of overlap between the two groups (Figure 1). Within six hours of ICU admission, patients in either group were significantly more likely than those who did not belong to each group to have at least one comorbidity, lower body mass index, surgical admission type, transfusion, mechanical ventilation, inotrope use, and higher creatine, international normalized ratio, and lactate (all p < 0.05). In addition, those in the late transfusion group were less likely to have undergone emergent surgery, whereas those in the late anemia had lower initial hemoglobin values and were more likely to have been less than 65 years of age and admitted for trauma.
Model development and validation
The population was divided into development (n = 3,619) and validation (n = 1,551) cohorts. Independent predictors for late transfusions and late anemia are provided in Table 2. The ROC curve areas and Hosmer–Lemeshow test p values for the development and validation cohorts for late transfusion were 0.75 (p = 0.06) and 0.72 (p = 0.46), respectively, and the corresponding values for late anemia were 0.77 (p = 0.01) and 0.74 (p = 0.05). The ROC curve for the development cohort of the late anemia model is depicted in Figure 2a. This figure also illustrates the correspondence between sensitivity and the fraction of the study population that exceeds the positivity threshold and is therefore considered at risk for late anemia for a given sensitivity level. Higher sensitivity results in lower specificity, corresponding to a higher proportion of the population being considered at risk. For example, a desired sensitivity of 60% entails declaring a threshold for positivity corresponding to a probability of late anemia of 22% or more (T'), resulting in 34% of the population exceeding this predicted probability and thus being considered at risk for late anemia (P') and potential candidates for intervention. For a desired sensitivity of 90%, the corresponding probability of late anemia would be 10% or more, resulting in the judging of 64% of the population to be at risk. Higher discrimination; in other words a greater ROC curve area, would allow a smaller proportion of the population to be considered at risk for a given sensitivity. In secondary analyses we constructed models that included the same risk factors but evaluated within 24, 48, and 72 hours of ICU admission. These models showed modest improvements in discrimination (Table 3), yet they resulted in progressively greater delays in identifying patients at risk of late transfusion or late anemia. It is noteworthy that predicting late transfusion was consistently less accurate than predicting late anemia.
Development of an ICU anemia score
We wished to improve the practicality and possible clinical utility of a prediction rule for patients at risk for late anemia who might benefit from a blood-sparing strategy. Using the late anemia model we constructed a point system to calculate, within 6 hours of ICU admission, the likelihood of reaching the outcome of interest, a hemoglobin value of 7 g/dl or less at 7 days or more after ICU admission. We assumed that a baseline hemoglobin of 14 g/dl or more and the absence of any risk predictors would amount to a minimum score of zero and thus a minimum chance for reaching the outcome. The maximal score of 51 corresponds to a baseline hemoglobin of 7 and the presence of all risk factors. Interestingly, this maximal score corresponds to a predicted probability of outcome of 0.81 as opposed to 1.00, reflecting that patients with an excessive burden are likely to die before 7 days. The ICU anemia point scheme, described in Table 4, demonstrated good discrimination, with an ROC curve area of 0.74. The correspondence between the sensitivity of the model in identifying patients that suffered late anemia and the corresponding number of points is shown in Figures 2b and 3.
Recommending a specific threshold beyond which patients should be considered at risk for late anemia and thus as candidates for a blood-sparing strategy depends on the cost and possible side effects of this strategy. Willingness to broaden the criteria to implement the strategy is more forthcoming with a low risk/low cost strategy. From Figure 2b, one might consider as candidates patients with ICU anemia scores in excess of 12 (more than 90% sensitivity; 64% of the population are candidates). Costlier or riskier strategies would dictate higher thresholds (for example an ICU anemia score of more than 21; more than 60% sensitivity; 34% of the population are candidates).
Potential consequences of point-based system and other prescriptive strategies
On the assumption that the proposed intervention is rHuEPO, treatment of all comers regardless of baseline or clinical characteristics resulted in 100% of patients who needed the intervention getting it, but also 100% of patients who did not (Table 5 and Figure 2b, point A). Application of the Corwin trial inclusion criteria  to patients in our cohort correctly targeted for treatment 77.4% (800/1,034) of those who were destined to develop late anemia (Table 5). However, it also resulted in the treatment of 35.5% (1,470/4,136) of patients who did not develop late anemia. When plotted against the ICU anemia score ROC curve, the prediction offered by the Corwin criteria (Figure 2b, point B) falls near the curve, indicating that the Corwin criteria perform as well as the point-based system. However, the Corwin criteria require a patient to be in the ICU for 3 days before initiating treatment, resulting in a delay of more than 2 days in treatment initiation in comparison with the point-based system, which depends only on data available within 6 hours of ICU admission.
Using information available within six hours of ICU admission, we were able predict the likelihood of late anemia with reasonable accuracy, suggesting a window of opportunity for the early identification of patients who might benefit from transfusion-sparing strategies. Additionally, the application of predictors in the form of a rule-based decision support tool resulted in a much larger proportion of patients 'appropriately' receiving intervention as opposed to more broad-based use and earlier intervention than with existing trial inclusion criteria . Although this exercise was theoretical, it has the potential to increase the safe and cost-effective use of rHuEPO and other transfusion-sparing strategies. Furthermore, we believe it is illustrative of the potential opportunities for similar models for other pre-emptive management strategies in the ICU.
The identification of candidates most likely to benefit from an early intervention targeting late sequelae involves a trade-off between accuracy and timeliness. In our models, including risk factors from within 24, 48, or 72 hours of ICU admission, as opposed to only the first 6 hours, improved discrimination, but at a cost of a delays in starting the intervention. Similarly, using existing clinical trial inclusion criteria to select patients for intervention led to a delay of more than 2 days in treatment. For an intervention such as rHuEPO, which takes days to begin to show an effect, a delay of 1 or 2 days may be unacceptable. For less time-sensitive pre-emptive strategies, such delays may not be an issue. The importance of timeliness must be balanced with that of accuracy, which becomes more important as th erisk and cost of a particular intervention increase. Clearly, identification strategies will need to be tailored for each intervention.
Predicting late transfusion was consistently less accurate than predicting late anemia. Given the substantial variation in transfusion practices  it should not be surprising that predicting transfusion was difficult. As transfusion practices become more standardized and restrictive transfusion thresholds the norm, the ability to predict transfusion is likely to become easier. However, as we have witnessed at our own institution, even with strict enforcement of transfusion thresholds, there are still many apparently unjustified transfusions that continue to add unnecessary variation.
There are several limitations that deserve consideration. Because of the retrospective nature of this study, we were not able to determine why patients received blood transfusion. Because so few hemoglobin concentrations ever fell to 7 g/dl before transfusion, we had to extrapolate pre-transfusion hemoglobin values to determine who would have reached 7 g/dl by using the mean of individual slopes of hemoglobin concentrations. Although it might have been more accurate to develop a predictive model using a large group of patients whose hemoglobin concentrations actually fell to the threshold, such a group does not exist, even among the 5,170 ICU patients in our study. Importantly, the mean hemoglobin slope after the acute phase (more than 3 days) used for this extrapolation (about 0.22 g/dl per day) was consistent with what others have reported previously  and are therefore likely to be a reasonable approximation. To develop our models we relied on information obtained from several computerized databases, an approach that would miss potential predictors that were not recorded in these databases. This was a single-center study at a tertiary-care academic medical center. It is possible that predictors of late anemia at this institution might vary from those of other institutions, especially those with significantly different case-mix or ICU practices. In light of this and because this exercise was only intended to be theoretical at this point, the ICU anemia point system should be validated in other cohorts before being applied to routine patient care.
A rule-based decision support tool using information available within six hours of ICU admission could lead to more appropriate and timely use of transfusion-sparing interventions. In the future, strategies to identify critically ill patients most likely to benefit from pre-emptive therapies will become increasingly important if therapies are to be used safely and cost-effectively.
Identifying critically ill patients most likely to benefit from pre-emptive therapies will become increasingly important if therapies are to be used safely and cost-effectively.
By using information available within 6 hours of ICU admission, we were able predict the likelihood of late anemia with reasonable accuracy, suggesting a window of opportunity for the early identification of patients who might benefit from transfusion-sparing strategies.
The application of predictors in the form of a rule-based decision support tool resulted in a much larger proportion of patients 'appropriately' selected for intervention.
A rule-based method has the potential to increase the safe and cost-effective use of rHuEPO and other transfusion-sparing strategies in critically ill patients admitted to the ICU.
ARDS = acute respiratory distress syndrome
ICU = intensive care unit
rHuEPO = recombinant human erythropoietin
ROC = receiver operating characteristic.
The Acute Respiratory Distress Syndrome Network: Ventilation with lower tidal volumes as compared with traditional tidal volumes for acute lung injury and the acute respiratory distress syndrome.N Engl J Med 2000, 342: 1301-1308. 10.1056/NEJM200005043421801
Gajic O, Dara SI, Mendez JL, Adesanya AO, Festic E, Caples SM, Rana R, St Sauver JL, Lymp JF, Afessa B, et al.: Ventilator-associated lung injury in patients without acute lung injury at the onset of mechanical ventilation.Crit Care Med 2004, 32: 1817-1824. 10.1097/01.CCM.0000133019.52531.30
Rumbak MJ, Newton M, Truncale T, Schwartz SW, Adams JW, Hazard PB: A prospective, randomized, study comparing early percutaneous dilational tracheotomy to prolonged translaryngeal intubation (delayed tracheotomy) in critically ill medical patients.Crit Care Med 2004, 32: 1689-1694. 10.1097/01.CCM.0000134835.05161.B6
Corwin HL, Gettinger A, Pearl RG, Fink MP, Levy MM, Shapiro MJ, Corwin MJ, Colton T: Efficacy of recombinant human erythropoietin in critically ill patients: a randomized controlled trial.JAMA 2002, 288: 2827-2835. 10.1001/jama.288.22.2827
Hebert PC, Wells G, Blajchman MA, Marshall J, Martin C, Pagliarello G, Tweeddale M, Schweitzer I, Yetisir E: A multicenter, randomized, controlled clinical trial of transfusion requirements in critical care. Transfusion Requirements in Critical Care Investigators, Canadian Critical Care Trials Group.N Engl J Med 1999, 340: 409-417. 10.1056/NEJM199902113400601
This work was supported by a research grant from Amgen, Inc.
Authors and Affiliations
The CRISMA Laboratory (Clinical Research, Investigation, and Systems Modeling of Acute Illness), Department of Critical Care Medicine, University of Pittsburgh School of Medicine, 3550 Terrace Street, Pittsburgh, PA, 15261, USA
Eric B Milbrandt, Gilles Clermont, Javier Martinez, Alex Kersten, Malik T Rahim & Derek C Angus
This work was supported by a research grant from Amgen, Inc. The authors were solely responsible for study conceptualization, analysis, and manuscript preparation without input or oversight from the sponsor.
EBM participated in the design of the study, acquisition, analysis and interpretation of the data, and drafted the manuscript. GC participated in the design of the study, acquisition, analysis and interpretation of the data, and helped draft the manuscript. JM, AK, and MTR participated in the acquisition, analysis and interpretation of the data and helped draft the manuscript. DCA conceived of the study, participated in its design and coordination, and helped draft the manuscript. All authors read and approved the final manuscript.
Authors’ original submitted files for images
Below are the links to the authors’ original submitted files for images.