Skip to content


Open Access

Duration of red blood cell storage is associated with increased incidence of deep vein thrombosis and in hospital mortality in patients with traumatic injuries

  • Philip C Spinella1, 2Email author,
  • Christopher L Carroll1,
  • Ilene Staff3,
  • Ronald Gross4,
  • Jacqueline Mc Quay4,
  • Lauren Keibel1,
  • Charles E Wade2 and
  • John B Holcomb5
Critical Care200913:R151

Received: 12 June 2009

Accepted: 22 September 2009

Published: 22 September 2009



In critically ill patients the relationship between the storage age of red blood cells (RBCs) transfused and outcomes are controversial. To determine if duration of RBC storage is associated with adverse outcomes we studied critically ill trauma patients requiring transfusion.


This retrospective cohort study included patients with traumatic injuries transfused ≥5 RBC units. Patients transfused ≥ 1 unit of RBCs with a maximum storage age of up to 27 days were compared with those transfused 1 or more RBC units with a maximum storage age of ≥ 28 days. These study groups were also matched by RBC amount (+/- 1 unit) transfused. Primary outcomes were deep vein thrombosis and in-hospital mortality.


Two hundred and two patients were studied with 101 in both decreased and increased RBC age groups. No differences in admission vital signs, laboratory values, use of DVT prophylaxis, blood products or Injury Severity Scores were measured between study groups. In the decreased compared with increased RBC storage age groups, deep vein thrombosis occurred in 16.7% vs 34.5%, (P = 0.006), and mortality was 13.9% vs 26.7%, (P = 0.02), respectively. Patients transfused RBCs of increased storage age had an independent association with mortality, OR (95% CI), 4.0 (1.34 - 11.61), (P = 0.01), and had an increased incidence of death from multi-organ failure compared with the decreased RBC age group, 16% vs 7%, respectively, (P = 0.037).


In trauma patients transfused ≥5 units of RBCs, transfusion of RBCs ≥ 28 days of storage may be associated with deep vein thrombosis and death from multi-organ failure.


Deep Vein ThrombosisInjury Severity ScoreDeep Vein Thrombosis ProphylaxisBlood Group TypeAdmission Vital Sign


In 2004, 29 million units of blood components were transfused in the US [1]. Due to advances in testing for infectious agents, the risk of transmitted diseases associated with blood products continues to dramatically decrease [1]. However, there are still significant risks associated with red blood cell (RBC) transfusion [28]. In particular, an increased volume of RBC transfusion has been associated or independently associated with adverse outcomes, including sepsis, deep vein thrombosis (DVT), multi-organ failure, and death [28]. A meta-analysis that included 270, 000 patients found that the risks of RBC transfusion were greater than the benefits in 42 of the 45 studies examined [9]. Additionally, a recent large prospective randomized controlled study in critically ill patients reported as a secondary outcome that in-hospital mortality was related to the amount of RBCs transfused [10].

Several investigators have attempted to determine reasons for the association between RBC transfusion and poor outcomes. A plausible biologic explanation is that lesions occurring to RBCs during prolonged storage contribute to these poor outcomes. Stored RBCs have been associated with inflammatory injury, immunomodulation, altered tissue perfusion, and impaired vasoregulation [26]. In vitro studies also document increased risk of hypercoagulation with aged RBCs [11, 12]. In addition, transfusion of RBCs stored for greater than 14 to 28 days has been linked to poor outcomes [24, 6]. However, the studies supporting the association between RBC storage and poor outcomes are mainly retrospective or prospective cohort studies, and a few studies have failed to find an association [1318]. As a result, the theory that prolonged storage of RBCs lead to poor outcomes remains controversial [19].

We suspect that poor outcomes associated with the transfusion of RBCs stored for a prolonged period may be due, in part, to an increased inflammatory and hypercoagulable state induced by 'old RBCs' in critically ill patients. Patients with significant traumatic injuries develop a hyper-inflammatory and hypercoagulable state [20]. The pro-inflammatory and immunomodulatory nature of old RBCs [21, 22] may further promote a hypercoagulable state [23, 24]. DVT may be promoted in patients who are in a hypercoagulable state and multi-organ failure (MOF) is well known to occur via hypercoagulable mechanisms. We therefore hypothesized that the transfusion of old RBCs to critically ill trauma patients would be associated with an increased incidence of DVT and in-hospital mortality. A secondary hypothesis was that death secondary to MOF would be increased for patients transfused old RBCs.

Materials and methods

This study was approved by the Institutional Review Board at Hartford Hospital, Hartford, CT, USA. We performed a retrospective cohort study of patients aged 16 years or older admitted to the Hartford Hospital intensive care unit (ICU) with traumatic injuries who received five or more units of RBCs during the hospital admission between 2004 and 2007. Patients who died in the emergency or operating room prior to ICU admission were excluded.

Data were retrospectively analyzed from prospectively populated hospital databases and patient charts. To ensure adequate follow up or to account for deaths that occurred in patients discharged prior to 180 days from admission, the social security index and Hartford Hospital databases were used to determine if there were any deaths prior to this time.

In addition to mortality, information collected included patient age, race, sex, ABO blood type, admission vital signs and laboratory values, Glasgow Coma Score (GCS), Injury Severity Score (ISS), total units of RBCs given during the entire hospitalization, plasma, apheresis platelets, cryoprecipitate, percentage of RBCs that were leukoreduced, mechanism of injury, use of DVT prophylaxis, ICU free days, and cause of death. The GCS recorded was the lower value recorded by either emergency medical providers pre-hospital or by providers in the emergency department. Race was determined by the trauma registrar and recorded in the hospital database by the following categories: white, black, Hispanic, Asian, Pacific Islander, or other. Mechanism of injury was categorized as either blunt or penetrating injury.

The incidence of DVT was determined by reviewing ultrasound results for DVT screening tests that are routinely performed on days 2 to 3 of admission for all trauma patients in the ICU. In addition to these empiric screens, if a DVT was diagnosed later in the admission due to clinical suspicion it was also included in our analysis. A DVT was defined as a thrombus that was detected by ultrasound in a deep vein. Superficial venous thrombi were not included. All forms of DVT prophylaxis were recorded including intravenous and subcutaneous heparin, subcutaneous enoxaparin, and pneumatic compression devices. The frequency of DVT prophylaxis was then compared between RBC storage age study groups. The ISS was calculated by trained staff within the Hartford Hospital Trauma Program according to the methods described by the Association for the Advancement of Automotive Medicine Abbreviated Injury Scale, 1998 Revision. Cause of death was determined by chart review and was categorized as either death due to hemorrhage, primary central nervous system (CNS) injury, or MOF. MOF was defined as two or more organ failures at the time of death. Organ failure at time of death was defined as follows: cardiac failure as requiring vasoactive agents, pulmonary failure as requiring mechanical ventilation with radiographic evidence of lung pathology, CNS failure as GCS less than 6, and renal failure as requiring dialysis or serum creatinine more than 3 mg/dl. Patients with traumatic brain injuries who remained intubated at time of death without evidence of lung injury or who were on minimal mechanical ventilator settings were determined to have died secondary to primary CNS injury and not MOF. The cause of death and organ failure at time of death was determined by chart review by a single reviewer (PCS) who was blinded to patient RBC age category and all other variables recorded for the study patients. This was accomplished by this reviewer being blinded to the database and just reviewing death certificates and patient charts. Organ failure scores such as the Sequential Organ Failure Assessment or Marshall Multi Organ Dysfunction Score were not able to be calculated from our database.

Data analysis

We defined our study groups according to the maximum storage age of RBCs. Previous studies that used either non prestorage leukocyte reduced or prestorage leukocyte reduced RBCs have reported that RBCs above (mean or maximum) 14 to 28 days were associated with adverse events or outcomes [11, 13, 2531]. Clinical studies have also reported on univariate analysis that MOF and mortality have been associated with the transfusion of RBCs of 30 and 25 days, respectively [25, 26]. Therefore, as a result of our blood bank issuing RBCs that were both not prestorage leukocyte reduced and were leukocyte reduced during the time period of the study, we a priori decided to categorize patients according to a maximum RBC storage age of 14 or more, 21 or more, and 28 or more days. The primary groups analyzed are defined by a maximum RBC age of less than 28 days or 28 or more days, unless otherwise noted. To ensure equal amounts of RBCs transfused we matched all study groups within +/- one unit of total RBCs transfused. This was accomplished by a computerized random sampling program ("SAMPLE", SPSS, Chicago, IL, USA). The matching of patients by RBC volume was performed for each maximum RBC age analyzed (14, 21 and 28 days).

We defined study groups according to maximum RBC age, rather than mean RBC age, because the mean can obscure potential effects of older RBCs [32]. We categorized transfusion amount as 5 or more, and 10 or more units of RBCs. This was based on previous findings demonstrating that mortality dramatic increases after five or more units of RBCs have been transfused to patients with traumatic injuries [33]. To determine if there was an increased size effect with increased injury, we decided to analyze patients transfused 10 or more units of RBCs because RBC volume is associated with severity of illness [19].

The primary outcomes were DVT, and in-hospital mortality. Non-parametric and parametric data are presented as median (interquartile range) or mean (standard error of mean), respectively. The Wilcoxon Rank-sum test was used for comparison of non-parametric continuous data. The Fisher Exact or Chi Squared test was used for comparison of categorical data as appropriate. Variables with a P value of less than 0.1 on univariate analysis with in-hospital mortality were considered for inclusion for the multivariate logistic regression analysis. A best-fit model was determined by using changes in the log likelihood between models to determine which variables produced the most accurate model. The model with the highest chi squared statistic per degree of freedom was reported. A survival analysis at 180 days from admission was performed with a Kaplan Meier curve and Log Rank test. Statistical analysis performed with SPSS 15.0 (Chicago, IL, USA).


There were 270 patients identified who were admitted to the ICU with traumatic injuries and were transfused 5 or more units of RBCs. There were 202 patients who were able to be matched within 1 unit of RBC amount transfused according to the cut-off point of 28 days of RBC storage. Admission variables, ISS and outcomes were similar between the 202 patients included in the analysis and the 68 patients excluded as a result of not being able to match them with patients in the other treatment group (data not shown). In this cohort of patients who received 5 or more units of RBCs and matched by RBC amount (Figure 1), patient age, sex, race, admission vital signs and laboratory values, amount of blood products transfused, percentage leukoreduced RBCs, and ISS were similar between patients receiving RBCs of decreased and increased storage age (Table 1). Most of the patients (163 of 202 or 81%) received both prestorage leukoreduced and non-leukoreduced RBCs. There were only 39 of 202 (19%) patients who received 100% leukoreduced RBCs. The percentage of prestorage leukoreduced RBCs of all RBCs transfused was similar between RBC storage age groups (Table 1), and there was no relation between percentage of leukoreduced RBCs and mortality by chi squared analysis (Table 1) nor by logistic regression analysis with percent leukocyte reduction treated as a continuous variable (odds ratio (OR) = 1, 95% confidence interval (CI) = 0.99 to 1.01; P = 0.8). There were similar percentages of patients in the decreased and increased RBC storage groups who received plasma; 41.6% (42 of 101) vs 45.5% (46 of 101); platelets 17.8% (18 of 101) vs 24.8% (25 of 101), and cryoprecipitate 9.9% (10 of 101) vs 6.9% (7 of 101; P < 0.05). No patients in either study group received recombinant activated factor VII (rFVIIa). Blunt injury was less common in the decreased RBC storage age group compared with the increased RBC age group, 89% vs. 96%, respectively, (P = 0.05). Mechanism of injury was not associated with mortality on univariate analysis nor did it meet criteria for inclusion in the multivariate logistic regression analysis. The distribution of patient ABO blood group types was not similar between both study groups. Patients in the decreased RBC age group had a higher incidence of blood group type O and those in the increased RBC age group had a higher incidence of blood group type B (Table 2). No statistical differences were measured for patients with blood group types A and AB between study groups (Table 2). The maximum RBC storage age was (median, interquartile range) 19 days (16 to 24) and 34 (31 to 38) for decreased and increased RBC age groups, respectively (P < 0.001).
Figure 1
Figure 1

Frequency of patients transfused by total amount of RBCs for both study groups. RBC = red blood cells.

Table 1

Comparison of variables between patients transfused RBCs of decreased and increased storage age for patients transfused 5 or more units of RBCs


Decreased RBC age group (n = 101)

Increased RBC age group (n = 101)



48.0 (27.0 to 60.5)

45.0 (27.0 to 63.0)



78/101 (77.2%)

73/101 (72.3%)


Race (W, B, H, AP, O)%

(76.2, 6.9, 9.9, 2.0, 5.0)

(82.2, 5.9, 8.9, 0, 3.0)


Blunt injury

90/101 (89.1%)

97/101 (96.0%)


Glasgow Coma Score

14.0 (3.0 to 15.0)

14.0 (3.0 to 15.0)


Systolic blood pressure

126.0 (103.0 to 141.0)

123.0 (99.3 to 143.0)


Heart rate

100.0 (80.0 to 120.0)

99 (79.5 to 120.0)


Temperature (F)

96.5 (95.6 to 97.4)

96.5 (95.2 to 98.0)



21.0 (19.0 to 23.0)

21 (19.3 to 23.8)



7.3 (7.2 to 7.4)

7.3 (7.2 to 7.4)


Prothrombin time (seconds)

13.0 (12.2 to 14.5)

13.2 (12.2 to 14.2)


Hematocrit (%)

36.9 (32.9 to 39.2)

36.1 (31.1 to 39.6)


Heparin IV (%)*

14/101 (13.9%)

19/101 (18.8%)


Heparin SC (%)*

48/101 (47.5%)

51(101) (50.1%)


Enoxaparin SC (%)*

21/101 (20.8%)

25/101 (24.8%)


Pneumatic compression device (%)*

79/101 (78.2%)

72/101 (71.3%)


Long bone fracture (%)

46/101 (45.5%)

48/101 (47.5%)


Spinal cord injury (%)

5/101 (5.0%)

10/101 (9.9%)


RBC amount (Units)

9.0 (6.0 to 12.5) [10.5, 6.0]

9.0 (6.0 to 12.0) [10.4, 5.9]


RBC leukoreduced%

50.0 (25.8 to 85.7)

62.5 (37.3 to 83.3)


Maximum RBC storage age (days)

19.0 (16.0 to 24.0)

34.0 (31.0 to 38.0)

< 0.001

Median RBC storage age (days)

14.0 (11.0 to 17.0)

20.5 (15.5 to 26.0)

< 0.001

FFP (Units)

0.0 (0.0 to 4.0) [2.5]

0.0 (0.0 to 4.0) [2.5]


aPLT (Units)

0.0 (0.0 to 0.0) [0.2]

0.0 (0.0 to 0.5) [0.2]


Cryoprecipitate (Units)

0.0 (0.0 to 0.0) [.1]

0.0 (0.0 to 0.0) [0.1]


Injury Severity Score

24.0 (14.0 to 34.0)

24 (13.5 to 33.5)


Data presented as median (interquartile range [mean] or as percentages

* indicates deep vein thrombosis prophylaxis methods prescribed

AP: Asian/Pacific Islander; aPLT: apheresis platelets; B: black; FFP: fresh frozen plasma; GCS: Glasgow Coma Score; H: hispanic; IV: intravenous; O: Other; RBC: red blood cell; SC: subcutaneous; W: white.

DVT prophylaxis was initiated in 93.1% (94 of 101) of patients in the decreased RBC age group compared with 89.1% (90 of 101) in the increased RBC age group (P = 0.46). There were no differences between the methods of prophylaxis between the two groups (Table 1). There were 183 of 202 (91%) of patients screened for DVT with 5 of 101 (5%) not screened in the decreased RBC age group and 14 of 101 (14%) not screened in the increased RBC age group. These 19 patients not screened for DVT had similar ISS compared with the 183 screened for DVT. Additionally, for these 19 patients without DVT screening performed, the five patients transfused RBCs of decreased storage age had similar ISS compared with the 14 patients transfused RBCs of increased storage age. ABO blood group types were similar between patients who did and did not develop DVT (P = 0.69; Table 2). In the 183 patients screened for DVT, the incidence of DVT was higher in the increased compared with the decreased RBC age group, 34.5% vs 16.7%, respectively, (P = 0.006; Table 1). The median day of DVT diagnosis was not different between increased and decreased RBC age groups, 8 days (6 to 14) vs 10 days (7 to 19), respectively (P = 0.58). When alternative definitions of old RBCs were used, the transfusion of one or more units of RBCs 21 or more days old was associated with increased DVT and there was an association that approached significance with the transfusion of 1 or more units of RBCs 14 or more days old (Table 3).
Table 2

Comparisons of ABO blood groups for study groups and outcomes measured

Blood group

Decreased RBC age group (n = 101)

Increased RBC age group*

(n = 101)

- DVT (%) (n = 137)

+ DVT (%)

(n = 46)

Survived (%)

(n = 161)

Died (%)

(n = 41)

A (n = 72)













B (n = 38)



27.7% *










AB (n = 12)













O (n = 80)



27.7% *










* indicates P value of 0.001 for comparison of ABO blood groups between decreased and increased red blood cell (RBC) age group (chi-squared test). DVT: deep vein thrombosis.

In-hospital mortality was increased for those who received RBCs of increased (maximum RBC age 28 or more days) compared with decreased (maximum RBC age of less than 28 days) RBC age, 27 of 101 (26.7%) vs. 14 of 101 (13.9%), respectively (P = 0.02; Table 3). Additionally, patients in the increased RBC age group had an increased incidence and rate of death out to 180 days (Kaplan-Meier statistic; Figure 2). Survival rates were similar according to ABO blood group types (P = 0.39; Table 2). When the number of transfused RBC units 28 or more days old was analyzed to determine how many are required to measure an association with increased mortality, the transfusion of just 1 to 2 units of RBCs 28 or more days old was associated with increased in-hospital mortality (Figure 3). The mean (± standard error of the mean) ICU-free days were also increased in the patients transfused RBCs of decreased storage age compared with the increased RBC age group, 64.2 ± 2.9 vs. 54.5 ± 3.6 days, respectively (P = 0.036). Although the absolute mortality rate increased as the cut off of RBC age lengthened from 14 to 28 days of storage there was no statistical difference between groups when defined at 14 and 21 days of storage (Table 3).
Figure 2
Figure 2

Kaplan Meier Curve of trauma associated survival over 180 days for patients transfused fresh and old RBCs. RBC: red blood cells.

Figure 3
Figure 3

The relation between in-hospital mortality and the amount of RBC units transfused at 28 or more days of storage in patients transfused 5 or more units of RBCs. RBC: red blood cells.

Table 3

Relation of RBC storage age and outcomes for patients transfused 5 or more units of RBCs and matched for RBC amount between study groups

Outcome and maximum RBC age used to determine increased RBC age group

Patient number

Decreased RBC age


RBC age

Absolute difference in outcome (%)




≥ 14 days


12.0% (3/25)

32.0% (8/25)



≥ 21 days


17.1% (14/82)

31.2% (24/77)



≥ 28 days


16.7 (16/96)

34.5% (30/87)





≥ 14 days


17.9% (5/28)

21.4 (6/28)



≥ 21 days


18.2% (16/88)

25.0% (22/88)



≥ 28 days


13.9% (14/101)

26.7% (27/101)



* see text for explanation of difference in patient numbers between number of patients analyzed for deep vein thrombosis (DVT) and mortality outcomes according to maximum red blood cell (RBC) age used to determine study groups.

On multivariate logistic regression, in-hospital mortality was independently associated with the transfusion of older RBCs for patients transfused 5 or more units of RBCs (OR = 4, 95% CI = 1.34 to 11.61; P = 0.01). Mortality was also independently associated with patient age, ISS, lower GCS, and total amount of cryoprecipitate transfused (Table 4). The incidence of death from MOF was increased for patients transfused RBCs of increased compared with decreased age, 16% vs 7%, respectively (P = 0.037; Table 5).
Table 4

Multi-variate logistic regression for in-hospital mortality


OR (95% CI)


Age (Years)

1.05 (1.01 to 1.08)


Cryoprecipitate (units)

12.9 (2.24 to 73.64)



0.89 (0.79 to 0.99)



1.08 (1.03 to 1.12)


Increased RBC age group

4.0 (1.34 to 11.61)


The area under the curve (95% confidence interval (CI)) for this regression analysis was 0.85 (0.77 to 0.92).

GCS: Glasgow Coma Score; ISS: Injury Severity Score; OR: odds ratio; RBC: red blood cell.

Table 5

Comparison of cause of death between study groups

Cause of death

Decreased RBC age group (n = 101)

Increased RBC age group (n = 101)



1/101 (1%)

1/101 (1%)



6/101 (6%)

10/101 (10%)


Multi-organ failure

7/101 (7%)

16/101 (16%)


CNS: central nervous system; RBC: red blood cell.

There were 94 patients matched by RBC units transfused who received 10 or more units of RBCs. In this cohort, there were no differences in patient age, admission vital signs and laboratory values, amount of blood products transfused, percentage of leukoreduced RBCs, and ISS between patients receiving RBCs of decreased and increased storage age (data not shown). The maximum RBC storage age (median, interquartile range) was 20 days (18 to 24) vs 34 (31 to 38) for decreased and increased RBC storage age groups, respectively (P< 0.001). Of the 83 of 94 (88%) patients who were screened for DVT, the incidence of DVT was higher in the increased (maximum RBC age 28 or more days) compared with the decreased RBC age group, 17 of 39 (43.6%) vs. 7 of 44 (15.9%), respectively (P = 0.006). Mortality was increased for those who received RBCs of increased compared with decreased storage age, 18 of 47 (38.3%) vs 6 of 47 (12.8%; P = 0.009). On multivariate logistic regression, in-hospital mortality was independently associated with the transfusion of RBCs of increased age (OR = 8.9, 95% CI = 2 to 40; P = 0.004). The incidence of death from MOF was increased in the patients transfused RBCs of increased compared with decreased age, 11 of 47 (22%) vs. 3 of 47 (6%), respectively (P = 0.02). The mean (± standard error of the mean) ICU-free days were raised in the decreased compared with the increased RBC age group, 59.8 ± 4.2 vs. 41.6 ± 5.2 days, respectively (P = 0.008)


This is the first study to report an independent association between the transfusion of RBCs of increased storage age (maximum RBC age 28 or more days) with increased in-hospital mortality for critically ill trauma patients transfused similar total amounts of RBCs. Death as a result of MOF was increased and ICU-free days were decreased for patients transfused RBCs of increased age. Our results also indicate that critically ill patients transfused just 1 to 2 units of old RBCs (28 or more days of storage) was associated with increased mortality. This suggests that even relatively small number of old RBC transfusions may be harmful in critically ill trauma patients. Finally, an association was measured between RBC of increased storage age (maximum RBC age 21 or more days) with DVT, which has not been previously reported.

The incidence of DVT was numerically increased with the transfusion of RBCs of 14 or more, 21 or more, and 28 or more days old with similar absolute differences in DVT incidence. Statistical significance occurred only for groups defined at 21 or more and 28 or more days of age. This is in contrast to an increasing absolute difference in mortality as the definition of old RBCs increased with statistical significance only for the groups defined at a maximum of 28 days of storage. This analysis was limited by less patients in the 14 or more and 21 days RBC age groups. In contrast, these comparisons were strengthened by the matching of the amount of RBCs transfused between all groups of increased and decreased RBC age compared in this study (≥ 14, ≥ 21, and ≥ 28 days).

Although our study was not designed to investigate mechanisms associated with findings, our hypothesis was based on previous literature reviewed by Park and colleagues regarding the interplay between inflammation and hypercoagulation [20] and the literature supporting old RBCs are hyper-inflammatory, immunomodulatory, and impair microvascular perfusion and vasoregulation [26]. Old RBCs have been demonstrated to increase polymorphonuclear cell activation, superoxide anion and IL-8 concentrations [21, 22], which may be a result of pro-inflammatory bioactive lipids, which increase with RBC storage time [5, 34]. In fact, bioactive lipids that accumulate with storage time have recently been associated in a laboratory study with increased thrombin generation. In these prestorage leukoreduced RBCs increased thrombin generation occurred after 31 days of storage in AS-1 solution [12]. Another recent publication indicates RBC storage time is associated with increased generation of procoagulant phospholipids [11]. Immunomodulation and increased risk of sepsis independently associated with old RBCs will also increase these risks [25, 27, 3538]. We theorize that the hyper-inflammatory and hypercoagulable state associated with trauma is potentiated by the pro-inflammatory and immunomodulatory effects of old RBCs [21, 22, 35, 37, 39], which then increases the risk of DVT and death as a result of MOF via hypercoagulation and diffuse endothelial injury (Figure 4).
Figure 4
Figure 4

Flow diagram of describing potential mechanism of how old RBCs increase risk of multi-organ failure via inflammatory and coagulation pathways. ARDS: acute respiratory distress syndrome; DVT: deep vein thrombosis; MI: myocardial infarction; RBC: red blood cells.

We compared ABO blood group types between study groups because a previous report indicates that patients with type A blood have increased concentrations of factor VIII and von Willebrand's factor, which was associated with increased risk of DVT [40]. In our analysis, although there was not an equal distribution of patient ABO blood groups between study groups, we did not measure any relation between patient blood type and incidence of DVT or in-hospital mortality. Therefore, although it is important to determine the effect of ABO blood type on risk of thromboembolic events in future analyses, there was no apparent effect on either DVT or mortality in our study.

Previous studies reported an independent association between the transfusion of old RBCs and increased risk of sepsis, MOF, and death in all types of critically ill patients [2527, 32, 41, 42]. However, a consistent criticism of some of these studies is that there were not equal amounts of RBCs transfused in the study groups. The concern regarding the amount of RBCs transfused per study group is related to the concept that RBC amount itself strongly correlates with injury severity and can never be adequately adjusted for with multivariate logistic regression [9]. Our findings may have increased validity compared with previous studies as a result of our method of specifically matching patients by the amount (± 1 units) of RBCs transfused. Previous reports have also indicated that RBC transfusion volume was associated with DVT [7, 8]. These studies did not take into account the storage age of RBCs.

The recent study by Weinberg and colleagues in trauma patients analyzed the number of units greater than 14 days of age and reported that for patients transfused 6 or more units of pre-storage leukoreduced RBCs that the odds ratio for death was increased for those who were transfused 1 to 2 units greater than 14 days old and that the odds of death were higher for patients who were transfused 3 or more units of RBCs [32]. Although this study did not compare patient groups that specifically matched patients by amount of RBCs, their findings are consistent with the present study results. Not only is there consistency with increased mortality in patients transfused old RBCs there is also consistency in that both studies demonstrate it only takes 1 to 2 units of old RBCs to increase the odds of death and the size of the effect is greater for patients with increased injury indicated by the amount of RBCs transfused. In our analysis patients transfused 10 or more units of RBCs had an approximate doubling of the OR for mortality when compared with patients transfused 5 or more units of RBCs.

A major difference in our report compared with the study by Weinberg and colleagues is our definition of when RBCs become 'old'. The different methods of defining old RBCs used in various studies have made comparing results problematic. Previous definitions have included mean, median, and maximum RBC storage age in addition to the number of RBCs transfused above 14 and 21 days of age [2527, 32, 41]. The change from the use of non-prestorage leukocyte reduced RBCs to the transfusion of prestorage leukocyte reduced RBCs has also made it difficult to determine the optimal definition of old RBCs. Independent associations with the amount of non-prestorage leukocyte reduced RBCs more than 14 and 21 days old have been reported with sepsis [27]. The mean RBC storage age and the amount of non-prestorage leukocyte reduced RBCs of more than 14 and 21 days old have also been associated with increased MOF [26]. Optimally when comparing the effect of RBC storage age on outcomes the definition of fresh and old RBCs should not allow for mixing of RBC storage age between groups as was done in the study by Koch and colleagues [41]. In this study of more than 6000 patients, the fresh RBC group was defined as those who were only transfused RBCs of 14 days of storage or less and the old RBC group received only RBCs of greater than 14 days of storage. In smaller retrospective studies that are not large enough to have complete separation of fresh and old RBCs, it is more appropriate to use the maximum RBC age transfused than mean RBC age to define if the patient received fresh or old blood. This is because the adverse effects of RBCs have been measured with relatively small amounts transfused [10, 32, 41]. When mean RBC age is used to define patients who received fresh or old RBCs this method allows for the youngest RBCs to balance out or negate the contribution of storage age from the oldest unit transfused. For example a patient who receives 2 units at 40 days old and 8 units at 10 days old will have a mean of 16 days whereas for a patient who receives 10 units at 16 days of storage the mean will be 16 days. The patient transfused 2 units at 40 days would theoretically be at increased risk but using the mean RBC age to define fresh vs old RBC patient groups does not identify this difference whereas using maximum RBC age does.

Additionally, an individual patient's severity of illness may influence the clinical effect of old RBCs. It is our theory that the most critically ill patients will be most affected by older RBCs. This concept is supported by the study performed by Weinberg and colleagues [32], where patients who were sicker had increased OR with mortality with 'old' RBCs, and also in our study where patients transfused 10 or more units of RBCs had an increased OR for mortality compared with patients with decreased injury who only received 5 or more units of RBCs.

A source of criticism of previous studies is the inclusion of patients who received non-leukocyte reduced RBCs. A recent study by Weinberg and colleagues only included patients who received prestorage leukoreduced RBCs and their results still demonstrated an increased risk of death with the use of RBCs more than 14 days of storage for patients transfused 6 or more units of RBCs [32]. In addition, while our study groups received a mixture and similar proportions of both non-leukoreduced and leukoreduced RBCs, the percentage of leukoreduced RBC units was not associated with survival on univariate or multivariate logistic regression analysis.

The clinical benefits of prestorage leukoreduced RBCs are controversial. Although there are some benefits to their use [43], it is our belief that universal leukoreduction cannot mitigate all the adverse effects of prolonged RBC storage in critically ill patients. For example, the deformability and nitric oxide mechanisms will very likely not be altered by leukoreduction nor will the proinflammatory effects of bioactive lipids that increase with storage time [34]. As has been suggested previously, perhaps the routine use of RBC washing for patients at risk of inflammatory and immunomodulatory injury should be considered when old RBCs need to be transfused [5]. The evidence that only one seven minute wash cycle is required to mitigate the proinflammatory effects of old RBCs [21] and the current development of large multi-unit RBC washing devices may make this approach a more viable option for patients requiring a large amount of RBCs rapidly.

During the time period of the study the typical practice at our trauma center did not include the frequent early use of plasma, platelets and cryoprecipitate as is described in Table 1 and there was no use at all of rFVIIa. Despite the low frequency of the use of these blood products the amount of cryoprecipitate was independently associated with in-hospital mortality. A potential explanation for these findings is that the use of cryoprecipitate was used very late in the resuscitation of patients when the patient was already in a state of irreversible shock and high risk of death secondary to hemorrhage. These findings are in contrast to recently published US military data indicating that the early use of procoagulant blood components to include cryoprecipitate and plasma are independently associated with improved survival [4446]. As we are unable to determine when specifically cryoprecipitate was transfused in our study, we cannot easily explain these results.

Our study was limited primarily by its retrospective nature. As such, limitations include possible selection bias and the potential for not adequately adjusting for unmeasured confounding variables. As a result our findings can only be hypothesis generating and are not intended to be interpreted as hypothesis testing. A significant limitation of our study was the inability to match RBC volume according to the timing of RBCs transfused. RBCs transfused after the development of DVT could not have influenced the development of DVT, although this risk should be equal in both RBC age groups studied. However, storage age of RBCs administered to the patients in this study was not chosen specifically by anyone. The age of RBC administered was according to blood bank policy and was consistent during the study period. Therefore, the risk of selection bias regarding the age of RBCs transfused is small. Although we did not include all potential confounders such as time from injury to operative control of bleeding, ICU practices etc., we were able to include a large number of variables that have been associated with mortality in trauma and our regression analyses were strong according to the high area under the curve measured in the model. The only difference noted in the primary patient population was an increased incidence of penetrating injury in the decreased RBC age group. However, the mechanism of injury was not associated with mortality and therefore is not a confounding variable on RBC age and mortality. Another potential limitation is that DVT screening did not occur for all patients included in the study. DVT screening occurred in 91% of patients included. Some may have been transferred out of the ICU before it was ordered, some may have died before it could be performed, and others may not have received one due to physician error in not ordering one. The timing of DVT screening was also not uniform or standardized. This may have introduced sampling bias. Although the use of each method of DVT prophylaxis method was similar between study groups, the inability to compare the timing of DVT prophylaxis initiation from admission is another limitation. Finally, our analysis of DVT was limited by our inability to adjust for confounding variables.

As the literature continues to demonstrate that older RBCs are potentially harmful in critically ill patients, and there is biologic plausibility, consistency, and size effect, well-designed prospective controlled trials to test this hypothesis must be performed. The clinical effects of the storage lesion and the precise mechanisms of how they potentially cause adverse effects need further study. Blood banks do not routinely record the storage age of RBCs transfused. This needs to become standard to facilitate the study of age of RBC on outcomes. Blood banking methods or alternative storage solutions also need to be studied to determine if these potential adverse effects can be mitigated. Furthermore, the criteria for licensing current and future storage solutions should also include the monitoring or testing of the many potential adverse effects of the storage lesion. Finally, study is needed in human subjects to determine if stored RBCs are able to perfuse the microvasculature tissue and increase oxygen delivery and consumption for critically ill patients with shock.


In trauma patients transfused 5 or more units of RBCs, DVT, and in-hospital mortality was increased with the transfusion of old RBCs when compared with a group of patients of similar severity of injury who were transfused RBCs of decreased storage age. After adjustment for other variables associated with mortality there was an independent association with the transfusion of older RBCs with in-hospital mortality. The increased risk of mortality was associated with the transfusion of just 1 to 2 units of RBCs greater than 28 days of storage and could be accounted for by increased MOF. As there is no evidence that RBCs of increased storage age improve microvascular delivery of oxygen and consumption for patients in a shock state and there is a substantial amount of evidence that indicates they may increase injury in critically ill patients, the preferential use of fresh RBCs can be appropriate if local inventory allows for this without substantially increasing RBC waste. Prospective randomized study in this population is needed.

Key messages

For patients with traumatic injuries transfused 5 or more units of RBCs and alive on ICU admission:

  • DVT is associated with the transfusion of RBCs of 21 or more days of storage in this study population.

  • Mortality at 30 days is independently associated with the transfusion of RBCs of 28 or more days of storage in this study population.

  • Mortality at 30 days was increased with 1 to 2 units of RBCs of 28 or more days of storage in this study population.

  • This is the first study to specifically match study groups by the volume of RBCs transfused, which eliminates the potential confounding effect of RBC volume transfused on DVT and mortality.

  • Future studies should investigate the potential affect of RBC storage age on thrombotic mechanisms.



confidence interval


central nervous system


deep vein thrombosis


Glasgow Coma Score


intensive care unit




Injury Severity Score


multi-organ failure


odds ratio


red blood cell


recombinant activated factor VII.



This study was funded by a grant from the Department of Research, Hartford Hospital.

Authors’ Affiliations

Department of Pediatrics, Connecticut Children's Medical Center, Hartford, USA
Department of Combat Casualty Care Research, United States Army Institute of Surgical Research, Fort Sam Houston, USA
Department of Research, Hartford Hospital, Hartford, USA
Department of Surgery and Emergency Medicine, Hartford Hospital, Hartford, USA
Department of Acute Care Surgery, University of Texas Health Science Center, Houston, USA


  1. AAAB: Facts about blood and blood banking.[]
  2. Ho J, Sibbald WJ, Chin-Yee IH: Effects of storage on efficacy of red cell transfusion: when is it not safe? Crit Care Med 2003,31(12 Suppl):S687-697. 10.1097/01.CCM.0000099349.17094.A3View ArticlePubMedGoogle Scholar
  3. Napolitano LM, Corwin HL: Efficacy of red blood cell transfusion in the critically ill. Crit Care Clin 2004,20(2):255-268. 10.1016/j.ccc.2003.12.002View ArticlePubMedGoogle Scholar
  4. Tinmouth A, Fergusson D, Yee IC, Hebert PC: Clinical consequences of red cell storage in the critically ill. Transfusion 2006,46(11):2014-2027. 10.1111/j.1537-2995.2006.01026.xView ArticlePubMedGoogle Scholar
  5. Silliman CC, Moore EE, Johnson JL, Gonzalez RJ, Biffl WL: Transfusion of the injured patient: proceed with caution. Shock 2004,21(4):291-299. 10.1097/00024382-200404000-00001View ArticlePubMedGoogle Scholar
  6. Spinella PC, Perkins JP, Grathwohl KG, Repine TB, Beekley AC, Sebesta J, Azarow KA, Holcomb JB: The risks associated with fresh whole blood and RBC transfusions in a combat support hospital. Crit Care Med 2007, 35: 2576-2581. 10.1097/01.CCM.0000285996.65226.A9View ArticlePubMedGoogle Scholar
  7. Knudson MM, Collins JA, Goodman SB, McCrory DW: Thromboembolism following multiple trauma. J Trauma 1992,32(1):2-11. 10.1097/00005373-199201000-00002View ArticlePubMedGoogle Scholar
  8. Geerts WH, Code KI, Jay RM, Chen E, Szalai JP: A prospective study of venous thromboembolism after major trauma. New Engl J Med 1994,331(24):1601-1606. 10.1056/NEJM199412153312401View ArticlePubMedGoogle Scholar
  9. Marik PE, Corwin HL: Efficacy of red blood cell transfusion in the critically ill: a systematic review of the literature. Crit Care Med 2008,36(9):2667-2674. 10.1097/CCM.0b013e3181844677View ArticlePubMedGoogle Scholar
  10. Hebert PC, Wells G, Blajchman MA, Marshall J, Martin C, Pagliarello G, Tweeddale M, Schweitzer I, Yetisir E: A multicenter, randomized, controlled clinical trial of transfusion requirements in critical care. Transfusion Requirements in Critical Care Investigators, Canadian Critical Care Trials Group. N Engl J Med 1999,340(6):409-417. 10.1056/NEJM199902113400601View ArticlePubMedGoogle Scholar
  11. Cardo LJ, Hmel P, Wilder D: Stored packed red blood cells contain a procoagulant phospholipid reducible by leukodepletion filters and washing. Transfus Apher Sci 2008,38(2):141-147. 10.1016/j.transci.2007.09.006View ArticlePubMedGoogle Scholar
  12. Sweeney J, Kouttab N, Kurtis J: Stored red blood cell supernatant facilitates thrombin generation. Transfusion 2009, in press.Google Scholar
  13. Leal-Noval SR, Jara-Lopez I, Garcia-Garmendia JL, Marin-Niebla A, Herruzo-Aviles A, Camacho-Larana P, Loscertales J: Influence of erythrocyte concentrate storage time on postsurgical morbidity in cardiac surgery patients. Anesthesiology 2003,98(4):815-822. 10.1097/00000542-200304000-00005View ArticlePubMedGoogle Scholar
  14. Vamvakas EC, Carven JH: Transfusion and postoperative pneumonia in coronary artery bypass graft surgery: effect of the length of storage of transfused red cells. Transfusion 1999,39(7):701-710. 10.1046/j.1537-2995.1999.39070701.xView ArticlePubMedGoogle Scholar
  15. Walsh TS, McArdle F, McLellan SA, Maciver C, Maginnis M, Prescott RJ, McClelland DB: Does the storage time of transfused red blood cells influence regional or global indexes of tissue oxygenation in anemic critically ill patients? Crit Care Med 2004,32(2):364-371. 10.1097/01.CCM.0000108878.23703.E0View ArticlePubMedGoogle Scholar
  16. Watering L, Lorinser J, Versteegh M, Westendord R, Brand A: Effects of storage time of red blood cell transfusions on the prognosis of coronary artery bypass graft patients. Transfusion 2006,46(10):1712-1718. 10.1111/j.1537-2995.2006.00958.xView ArticlePubMedGoogle Scholar
  17. Weiskopf RB, Feiner J, Hopf H, Lieberman J, Finlay HE, Quah C, Kramer JH, Bostrom A, Toy P: Fresh blood and aged stored blood are equally efficacious in immediately reversing anemia-induced brain oxygenation deficits in humans. Anesthesiology 2006,104(5):911-920. 10.1097/00000542-200605000-00005View ArticlePubMedGoogle Scholar
  18. Vamvakas EC, Carven JH: Length of storage of transfused red cells and postoperative morbidity in patients undergoing coronary artery bypass graft surgery. Transfusion 2000,40(1):101-109. 10.1046/j.1537-2995.2000.40010101.xView ArticlePubMedGoogle Scholar
  19. Zimrin AB, Hess JR: Current issues relating to the transfusion of stored red blood cells. Vox sanguinis 2009,96(2):93-103. 10.1111/j.1423-0410.2008.01117.xView ArticlePubMedGoogle Scholar
  20. Park MS, Salinas J, Wade CE, Wang J, Martini W, Pusateri AE, Merrill GA, Chung K, Wolf SE, Holcomb JB: Combining early coagulation and inflammatory status improves prediction of mortality in burned and nonburned trauma patients. J Trauma 2008,64(2 Suppl):S188-194. 10.1097/TA.0b013e318160a5a3View ArticlePubMedGoogle Scholar
  21. Biffl WL, Moore EE, Offner PJ, Ciesla DJ, Gonzalez RJ, Silliman CC: Plasma from aged stored red blood cells delays neutrophil apoptosis and primes for cytotoxicity: abrogation by poststorage washing but not prestorage leukoreduction. J Trauma 2001,50(3):426-431. discussion 432 10.1097/00005373-200103000-00005View ArticlePubMedGoogle Scholar
  22. Zallen G, Moore EE, Ciesla DJ, Brown M, Biffl WL, Silliman CC: Stored red blood cells selectively activate human neutrophils to release IL-8 and secretory PLA2. Shock 2000,13(1):29-33. 10.1097/00024382-200013010-00006View ArticlePubMedGoogle Scholar
  23. Kaufmann CR, Dwyer KM, Crews JD, Dols SJ, Trask AL: Usefulness of thrombelastography in assessment of trauma patient coagulation. J Trauma 1997,42(4):716-720. discussion 720-712 10.1097/00005373-199704000-00023View ArticlePubMedGoogle Scholar
  24. Park MS, Martini WZ, Dubick MA, Salinas J, Butenas S, Kheirabadi BS, Pusateri AE, Vos JA, Guymon MA, Wolf SE, Mann KG, Holcomb JB: Thromboelastography as a better indicator of hypercoagulable state after injury than prothrombin time or activated partial thromboplastin time. J Trauma 2009,67(2):266-276. 10.1097/TA.0b013e3181ae6f1cPubMed CentralView ArticlePubMedGoogle Scholar
  25. Purdy FR, Tweeddale MG, Merrick PM: Association of mortality with age of blood transfused in septic ICU patients. Can J Anaesth 1997,44(12):1256-1261. 10.1007/BF03012772View ArticlePubMedGoogle Scholar
  26. Zallen G, Offner PJ, Moore EE, Blackwell J, Ciesla DJ, Gabriel J, Denny C, Silliman CC: Age of transfused blood is an independent risk factor for postinjury multiple organ failure. Am J Surg 1999,178(6):570-572. 10.1016/S0002-9610(99)00239-1View ArticlePubMedGoogle Scholar
  27. Offner PJ, Moore EE, Biffl WL, Johnson JL, Silliman CC: Increased rate of infection associated with transfusion of old blood after severe injury. Arch Surg 2002,137(6):711-716. discussion 716-717 10.1001/archsurg.137.6.711View ArticlePubMedGoogle Scholar
  28. de Korte D, Kleine M, Korsten HG, Verhoeven AJ: Prolonged maintenance of 2, 3-diphosphoglycerate acid and adenosine triphosphate in red blood cells during storage. Transfusion 2008,48(6):1081-1089. 10.1111/j.1537-2995.2008.01689.xView ArticlePubMedGoogle Scholar
  29. Tsai AG, Cabrales P, Intaglietta M: Microvascular perfusion upon exchange transfusion with stored red blood cells in normovolemic anemic conditions. Transfusion 2004,44(11):1626-1634. 10.1111/j.0041-1132.2004.04128.xView ArticlePubMedGoogle Scholar
  30. Sweeney J, Kouttab N, Kurtis J: Stored red cell supernatant facilitates thrombin generation. Transfusion 2009, in press.Google Scholar
  31. Fitzgerald RD, Martin CM, Dietz GE, Doig GS, Potter RF, Sibbald WJ: Transfusing red blood cells stored in citrate phosphate dextrose adenine-1 for 28 days fails to improve tissue oxygenation in rats. Crit Care Med 1997,25(5):726-732. 10.1097/00003246-199705000-00004View ArticlePubMedGoogle Scholar
  32. Weinberg JA, McGwin G Jr, Griffin RL, Huynh VQ, Cherry SA 3rd, Marques MB, Reiff DA, Kerby JD, Rue LW 3rd: Age of transfused blood: an independent predictor of mortality despite universal leukoreduction. J Trauma 2008,65(2):279-282. discussion 282-274 10.1097/TA.0b013e31817c9687View ArticlePubMedGoogle Scholar
  33. Spinella PC, Perkins JG, Holcomb JB: Risks associated with fresh whole blood and red blood cell transfusions in a combat support hospital. Crit Care Med 2007.,35(12): A152(suppl)Google Scholar
  34. Silliman CC, Clay KL, Thurman GW, Johnson CA, Ambruso DR: Partial characterization of lipids that develop during the routine storage of blood and prime the neutrophil NADPH oxidase. J Lab Clin Med 1994,124(5):684-694.PubMed CentralPubMedGoogle Scholar
  35. Blajchman MA: Immunomodulation and blood transfusion. Am J Ther 2002,9(5):389-395. 10.1097/00045391-200209000-00005View ArticlePubMedGoogle Scholar
  36. Hill GE, Frawley WH, Griffith KE, Forestner JE, Minei JP: Allogeneic blood transfusion increases the risk of postoperative bacterial infection: a meta-analysis. J Trauma 2003,54(5):908-914. 10.1097/01.TA.0000022460.21283.53View ArticlePubMedGoogle Scholar
  37. Raghavan M, Marik PE: Anemia, allogenic blood transfusion, and immunomodulation in the critically ill. Chest 2005,127(1):295-307. 10.1378/chest.127.1.295View ArticlePubMedGoogle Scholar
  38. Nimah M, Brilli RJ: Coagulation dysfunction in sepsis and multiple organ system failure. Crit Care Clin 2003,19(3):441-458. 10.1016/S0749-0704(03)00008-3View ArticlePubMedGoogle Scholar
  39. Stack G, Baril L, Napychank P, Snyder EL: Cytokine generation in stored, white cell-reduced, and bacterially contaminated units of red cells. Transfusion 1995,35(3):199-203. 10.1046/j.1537-2995.1995.35395184274.xView ArticlePubMedGoogle Scholar
  40. Tirado I, Mateo J, Soria JM, Oliver A, Martinez-Sanchez E, Vallve C, Borrell M, Urrutia T, Fontcuberta J: The ABO blood group genotype and factor VIII levels as independent risk factors for venous thromboembolism. Thromb Haemost 2005,93(3):468-474.PubMedGoogle Scholar
  41. Koch CG, Li L, Sessler DI, Figueroa P, Hoeltge GA, Mihaljevic T, Blackstone EH: Duration of red-cell storage and complications after cardiac surgery. N Engl J Med 2008,358(12):1229-1239. 10.1056/NEJMoa070403View ArticlePubMedGoogle Scholar
  42. Robinson WP 3rd, Ahn J, Stiffler A, Rutherford EJ, Hurd H, Zarzaur BL, Baker CC, Meyer AA, Rich PB: Blood transfusion is an independent predictor of increased mortality in nonoperatively managed blunt hepatic and splenic injuries. J Trauma 2005,58(3):437-444. discussion 444-435 10.1097/01.TA.0000153935.18997.14View ArticlePubMedGoogle Scholar
  43. Blajchman MA: The clinical benefits of the leukoreduction of blood products. J Trauma 2006,60(6 Suppl):S83-90. 10.1097/01.ta.0000199537.09201.7bView ArticlePubMedGoogle Scholar
  44. Borgman M, Spinella PC, Perkins JG, Grathwohl KW, T R, Beekley AC, Wade CE, Holcomb JB: Blood product replacement affects survival in patients receiving massive transfusions at a combat support hospital. J Trauma 2007,63(4):805-813. 10.1097/TA.0b013e3181271ba3View ArticlePubMedGoogle Scholar
  45. Spinella PC, Perkins JG, Grathwohl KW, Beekley AC, Niles SE, McLaughlin DF, Wade CE, Holcomb JB: Effect of plasma and red blood cell transfusions on survival in patients with combat related traumatic injuries. J Trauma 2008,64(2 Suppl):S69-77. discussion S77-68 10.1097/TA.0b013e318160ba2fView ArticlePubMedGoogle Scholar
  46. Stinger HK, Spinella PC, Perkins JG, Grathwohl KW, Salinas J, Martini WZ, Hess JR, Dubick MA, Simon CD, Beekley AC, Wolf SE, Wade CE, Holcomb JB: The ratio of fibrinogen to red cells transfused affects survival in casualties receiving massive transfusions at an army combat support hospital. J Trauma 2008,64(2 Suppl):S79-85. discussion S85 10.1097/TA.0b013e318160a57bView ArticlePubMedGoogle Scholar


© Spinella et al.; licensee BioMed Central Ltd. 2009

This article is published under license to BioMed Central Ltd. This is an open access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.