Open Access

Influence of enrollment sequence effect on observed outcomes in the ADDRESS and PROWESS studies of drotrecogin alfa (activated) in patients with severe sepsis

  • Pierre-François Laterre1Email author,
  • William L Macias2,
  • Jonathan Janes3,
  • Mark D Williams2,
  • David R Nelson2,
  • Amand RJ Girbes4,
  • Jean-François Dhainaut5 and
  • Edward Abraham6
Critical Care200812:R117

https://doi.org/10.1186/cc7011

Received: 14 April 2008

Accepted: 11 September 2008

Published: 11 September 2008

Abstract

Introduction

We performed a study to determine whether an enrollment sequence effect noted in the PROWESS (recombinant human activated Protein C Worldwide Evaluation in Severe Sepsis) trial exists in the ADDRESS (Administration of Drotrecogin Alfa [Activated] [DrotAA] in Early Stage Severe Sepsis) trial.

Methods

We evaluated prospectively defined subgroups from two large phase 3 clinical trials: ADDRESS, which included 516 sites in 34 countries, and PROWESS, which included 164 sites in 11 countries. ADDRESS consisted of patients with severe sepsis at low risk of death not indicated for treatment with DrotAA. PROWESS consisted of patients with severe sepsis with one or more organ dysfunctions. DrotAA (24 μg/kg per hour) or placebo was infused for 96 hours.

Results

In ADDRESS and PROWESS, there was a statistically significant interaction between the DrotAA treatment effect and the sequence in which patients were enrolled. In both trials, higher mortality was associated with DrotAA use in the subgroup of patients enrolled first at study sites. Compared with placebo, PROWESS mortality was lower with DrotAA treatment for the second and subsequent patients enrolled, whereas in ADDRESS, mortality remained higher for the second patient enrolled but thereafter was lower for DrotAA-treated patients. Comparison of patients enrolled first with subsequent patients enrolled indicated that the characteristics of patients changed. Subsequently enrolled patients were treated earlier, were less likely to suffer nonserious bleeds (ADDRESS), and experienced fewer protocol violations (PROWESS).

Conclusions

Analyses suggest that an enrollment sequence effect was present in the ADDRESS and PROWESS trials. Analysis of this effect on outcomes suggests that it is most apparent in patients at lower risk of death. In PROWESS, this effect appeared to be associated with a reduction of the DrotAA treatment effect for the first patients enrolled at each site. In ADDRESS, this effect may have contributed to early termination of the study. The finding of an enrollment sequence effect in two separate trials suggests that trial designs, site selection and training, data collection and monitoring, and statistical analysis plans may need to be adjusted for these potentially confounding events.

Trial Registration

ADDRESS trial registration number: NCT00568737. PROWESS was completed before trial registration was required.

Introduction

The Protein C Worldwide Evaluation in Severe Sepsis (PROWESS) study demonstrated that drotrecogin alfa (activated) (DrotAA) reduced mortality in patients with severe sepsis [1]. Subgroup analyses suggested heterogeneity in the observed treatment effect for some subgroups, including those defined by baseline Acute Physiology and Chronic Health Evaluation (APACHE) II score, by protocol violation status, and by the sequence of enrollment at a study site [2, 3]. Within these subgroups, the observed reduction in mortality associated with DrotAA was larger for patients with higher APACHE II scores, with no violation of the protocol, and who comprised the second and subsequent patients enrolled at a study site [3]. The latter two observations suggested that a learning curve appeared to be present within PROWESS such that the ability to demonstrate efficacy improved with increasing site experience with the study protocol [3].

Based on subgroup analyses of PROWESS, regulatory agencies approved the use of DrotAA in patients at higher risk of death as defined, for example, by an APACHE II score of greater than or equal to 25 or multiple-organ dysfunction (MOD) [4, 5]. As a condition for approval, the US Food and Drug Administration required the sponsor to conduct a randomized placebo-controlled trial of DrotAA in the nonindicated population of severe sepsis patients at lower risk of death (the Administration of Drotrecogin Alfa [Activated] in Early Stage Severe Sepsis [ADDRESS] study) [6, 7]. Based on the estimated placebo mortality rate in this lower-severity population, the ADDRESS study planned to enroll approximately 11,400 severe sepsis patients at 1,000 investigative sites in 35 countries. The ADDRESS study was prematurely terminated at the recommendation of the safety monitoring board because of a low likelihood of meeting the prospectively defined objective of demonstrating a significant reduction in the risk of 28-day all-cause mortality with DrotAA [7].

As a potential learning curve was present in the PROWESS trial and because the ADDRESS trial would require approximately 1,000 investigative sites, many of which were without prior clinical trial experience, prospectively defined analyses were included in the ADDRESS statistical analysis plan to assess the influence of any learning curve on the observed outcomes. We report the results of these analyses and additional exploratory analyses of both the PROWESS and ADDRESS databases. We discuss the results of these analyses in the context of their implication on the design and conduct of future clinical trials in patients with severe sepsis.

Materials and methods

Both PROWESS and ADDRESS were randomized double-blind placebo-controlled studies evaluating the efficacy (28-day mortality) of DrotAA (Xigris®; Eli Lilly and Company, Indianapolis, IN, USA) given as an intravenous infusion (24 μg/kg per hour) for 96 hours in patients with severe sepsis. Both studies were approved by the ethics committee of each individual participating center, and written informed consent was obtained from each patient or next of kin. In PROWESS, patients were at a greater risk of death [1] than in ADDRESS [7] (placebo 28-day mortality 30.8% versus 17.0%, respectively). For ADDRESS, the study enrolled patients with severe sepsis not indicated for treatment with DrotAA under the applicable label in the country in which the patient was enrolled. Severe sepsis was defined as the presence of a known or suspected infection and at least one sepsis-induced organ dysfunction. The population indicated for DrotAA varied from country to country but was generally defined as patients with severe sepsis with MOD and/or an APACHE II score of greater than or equal to 25. Randomization was stratified by site and within site by heparin use.

Statistical analyses

In the PROWESS study, the prospectively defined analysis to assess the influence of site enrollment on the observed treatment effect was an analysis of treatment effects that potentially differed across subgroup strata (using Breslow-Day tests). Potential interactions were identified for subgroups defined by presence versus absence of a significant protocol violation (P = 0.07), original versus amended protocol (P = 0.08), and APACHE II quartile at baseline (P = 0.09). Further examination of these interactions led to post hoc analyses of within-site sequence effects, as previously described [3]. Based on the post hoc significance of an interaction related to sequence, ADDRESS included a prospectively defined analysis to assess the influence of site enrollment on the observed treatment effect which was an analysis of 28-day mortality in the subgroups of first patients enrolled at each investigative site compared with the second and subsequent patients enrolled at each site (using Breslow-Day tests). The treatment effect was further assessed by analysis of mortality by the number of patients (1 to 4, 5 to 8, 9 to 12, and more than 12 patients) enrolled per site. Chi-square tests were used to compare mortality rates between treated and placebo patients.

Results

At the time of termination, 2,640 patients had been enrolled in the ADDRESS study at 516 centers in 34 countries. Mortality data at day 28 were available for 2,613 patients (placebo, n = 1,297; DrotAA, n = 1,316). There was no statistical difference between the placebo and DrotAA groups in 28-day all-cause mortality (placebo, 17.0%; DrotAA, 18.5%; P = 0.34) (Table 1). Based on a prospectively defined analysis, there was a significant treatment-by-sequence of enrollment interaction for the first patient enrolled at each site compared with all subsequently enrolled patients at that site (P = 0.04). Mortality at 28 days was higher for DrotAA patients compared with placebo patients in the subgroup of patients who comprised the first patients enrolled at each study site (22.3% versus 14.5%). Mortality rates were similar between treatment groups for the second and subsequent patients enrolled at each study site. Similar to what had previously been reported for PROWESS [3], treatment effect assessed by enrollment sequence grouped by block size (four patients per block) is listed in Table 2. In ADDRESS, mortality was higher for DrotAA patients compared with placebo in the first block, similar to placebo in the second block, and lower than placebo in the third and subsequent blocks. The median number of patients enrolled per site was eight. A treatment-by-enrollment sequence was observed at both small (≤8 patients) and high (>8 patients) enrolling sites (data not shown).
Table 1

Twenty-eight-day mortality for all patients enrolled in ADDRESS and for sequence subgroups

 

Drotrecogin alfa (activated)

Placebo

Relative risk

95% CI

Breslow-Day P value

 

Number

Died (percentage)

Number

Died (percentage)

   

All randomly assigned patients

1,316

243 (18.47)

1,297

221 (17.04)

1.08

0.92, 1.28

 

Patient classification

      

0.04

   First patient only

260

58 (22.31)

249

36 (14.46)

1.54

1.06, 2.25

 

   Excluding first patient

1,056

185 (17.52)

1,048

185 (17.65)

0.99

0.82, 1.19

 

ADDRESS, ADministration of DRotrecogin alfa (activated) in Early Stage Severe Sepsis; CI, confidence interval.

In PROWESS, a statistically significant treatment-by-enrollment interaction was also observed (P = 0.007) (Table 2). However, in PROWESS, mortality was lower for DrotAA patients compared with placebo in all randomization blocks, although the difference was larger in the second and subsequent blocks of patients. The relative risk associated with DrotAA was similar between ADDRESS and PROWESS for patients enrolled in the third and subsequent blocks. Additionally, patients enrolled in the third and subsequent blocks represented 45.5% of all PROWESS patients (n = 769/1,690) and only 21.4% of ADDRESS patients (n = 558/2,613).
Table 2

Mortality rates and relative risks for drotrecogin alfa (activated) by enrollment sequence within a site: ADDRESS and PROWESS

 

ADDRESS

PROWESS

Enrollment sequence within a site

Placebo

DrotAA

 

Placebo

DrotAA

 
 

Number

Mortality percentage

Number

Mortality percentage

RR (95% CI)

Number

Mortality percentage

Number

Mortality percentage

RR (95% CI)

1st to 4th patients

727

15.5%

741

20.1%

1.29 (1.04, 1.62)

279

31.5%

280

28.6%

0.91 (0.71, 1.17)

5th to 8th patients

303

17.5%

284

17.3%

0.99 (0.69, 1.40)

179

29.6%

183

24.4%

0.81 (0.58, 1.14)

9th to 12th patients

132

19.7%

142

16.2%

0.82 (0.49, 1.37)

128

25.8%

121

20.7%

0.80 (0.51, 1.26)

>12th patients

135

21.5%

149

14.8%

0.69 (0.42, 1.14)

254

33.5%

266

22.9%

0.69 (0.52, 0.91)

ADDRESS, ADministration of DRotrecogin alfa (activated) in Early Stage Severe Sepsis; CI, confidence interval; DrotAA, drotrecogin alfa (activated); PROWESS, Protein C Worldwide Evaluation in Severe Sepsis; RR, relative risk.

In PROWESS, randomization was stratified only by site, resulting in a uniform block size of four at each site. However, in ADDRESS, randomization was also stratified by baseline heparin use, so there was no uniform block size for randomization, thus the first four patients (in theory) could have all received DrotAA or all placebo or some other combination. Thus, further exploratory analyses were performed by subgroups in which the first through fourth patients enrolled at each site were excluded from the analysis. In ADDRESS (Figure 1a), 28-day mortality was lower for DrotAA patients compared with placebo patients in the subgroup of patients excluding the first two patients enrolled at a site (16.6% versus 18.4%). These data are similar to those in PROWESS in which higher mortality was observed for DrotAA patients compared with placebo patients who comprised the first enrolled patients at each site (n = 164 patients, 26.2% versus 20.0%). However, this 'first patient' effect was relatively small compared with the remaining patients in the study (Figure 1b).
Figure 1

Twenty-eight-day mortality in all randomly assigned patients in ADDRESS (a) and PROWESS (b) with no patients removed from the analysis and also with the first through fourth patients enrolled at each site removed. Note: 0 represents the results for the entire population, and 1 through 4 correspond to the analysis with the first through fourth patients from each site removed. ADDRESS, ADministration of DRotrecogin alfa (activated) in Early Stage Severe Sepsis; DrotAA, drotrecogin alfa (activated); PBO, placebo; PROWESS, Protein C Worldwide Evaluation in Severe Sepsis.

To explore interactions between mortality and the sequence of patient enrollment, ADDRESS patients were divided into subgroups comprising the first two enrolled patients at each site (≤2 subgroup, n = 904) and those comprising the third and subsequently enrolled patients (≥3 subgroup, n = 1,709). For patients in the ≤2 subgroup, 28-day mortality rates were 21.9% and 15.8% for DrotAA and placebo patients, respectively. In the ≥3 subgroup, 28-day mortality rates were 16.6% and 18.4% for DrotAA and placebo patients, respectively. Baseline characteristics for these subpopulations are shown in Table 3. Compared with patients in the ≤2 subgroup, patients in the ≥3 subgroup site tended to be enrolled in countries other than the US and Canada (indicating that 'patients per site' rates were generally lower in the US and Canada compared with the rest of the world), had lower acute physiology scores, and were more likely to have chronic health points, to have undergone recent surgery, and to have received prophylactic-dose heparin. These patients were also less likely to have community-acquired infections. Additionally, time from documented first organ dysfunction to start of study drug administration was shorter for third and subsequent patients enrolled at a site compared with the first two patients enrolled. In PROWESS, approximately 90% of patients started study drug within 24 hours [1], whereas in ADDRESS this was only 50%.
Table 3

Baseline characteristics for ADDRESS sequence subgroups

Variable

First 2 patients (n = 916)

3rd and subsequent patients (n = 1,724)

P value

Male, number (percentage)

517 (56.4%)

999 (57.9%)

0.46

Age in years, mean ± SD

58.3 ± 16.8

58.9 ± 16.6

0.36

Region, number (percentage)

  

<0.001

   Europe

265 (28.9%)

581 (33.7%)

 

   US and Canada

480 (52.4%)

681 (39.5%)

 

   Other countries

171 (18.7%

462 (26.8%)

 

Racial origin, number (percentage)

  

0.001

   African descent

67 (7.3%)

98 (5.7%)

 

   Caucasian

694 (75.8%)

1,221 (70.8%)

 

   Hispanic

57 (6.2%)

144 (8.4%)

 

   Asian

62 (6.7%)

168 (9.7%)

 

   Other

36 (3.9%)

93 (5.4%)

 

   Number of organ failures, mean ± SD

1.5 ± 0.7

1.4 ± 0.7

0.19

   APACHE II score, mean ± SD

18.3 ± 5.7

18.1 ± 5.9

0.22

   Acute physiology score, mean ± SD

14.1 ± 5.4

13.7 ± 5.4

0.07

   Number of patients with chronic health points (percentage)

221 (24.1%)

476 (27.6%)

0.08

   Recent surgery, number (percentage)

321 (35.0%)

681 (39.5%)

0.06

   Time from first organ failure to start of study drug in hours, mean ± SD

24.1 ± 13.6

21.8 ± 13.7

<0.001

   Community-acquired infection, number (percentage)

688 (75.1%)

1,235 (71.6%)

0.06

   Heparin use at baseline, number (percentage)

504 (55.0%)

1,047 (60.7%)

0.005

Frequencies were analyzed using a chi-square test. Means were analyzed using a type III sum squares analysis of variance on the ranks. ADDRESS, ADministration of DRotrecogin alfa (activated) in Early Stage Severe Sepsis; APACHE, Acute Physiology and Chronic Health Evaluation; SD, standard deviation.

The frequencies of serious bleeding events and any bleeding events in ADDRESS were also compared between the ≤2 and ≥3 subgroups (Table 4). The frequencies of serious bleeding events were similar between the ≤2 and ≥3 subgroups for both DrotAA and placebo patients. For DrotAA patients, a statistically significantly higher percentage of patients in the ≤2 subgroup experienced 'any bleeding' and 'any bleeding during the infusion' compared with DrotAA patients in the ≥3 subgroup. A lower percentage of DrotAA patients in the ≥3 subgroup experienced a transfusion compared with patients in the ≤2 subgroup (P = 0.13). A similar pattern for bleeding events and transfusions was observed in placebo patients, but the differences between the ≤2 and ≥3 subgroups did not reach statistical significance.
Table 4

Summary of adverse events by sequence of enrollment (ADDRESS)

Variable

Drotrecogin alfa (activated)

Placebo

 

≤2nd patient (n = 459)

≥3rd patient (n = 858)

P value

≤2nd patient (n = 422)

≥3rd patient (n = 851)

P value

Patients with ≥1 SBE

16 (5.5%)

35 (4.1%)

0.60

11 (2.5%)

17 (2.0%)

0.57

Patients with ≥1 SBE during infusion

9 (2.0%)

22 (2.6%)

0.49

6 (1.4%)

9 (1.1%)

0.63

Patients with ≥1 BE

61 (13.3%)

82 (9.6%)

0.04

33 (7.5%)

50 (5.9%)

0.27

Patients with ≥1 of any BE during infusion

53 (11.5%)

69 (8.0%)

0.04

28 (6.3%)

41 (4.8%)

0.25

Patients requiring any blood transfusion

38 (8.3%)

52 (6.1%)

0.13

19 (4.3%)

25 (2.9%)

0.20

Patients with ≥1 BE and who did not survive

28 (6.1%)

29 (3.4%)

0.02

7 (1.7%)

16 (1.9%)

0.68

'≤2nd patient' refers to the first two patients enrolled. '≥3rd patient' refers to the third and subsequent patients enrolled. Frequencies were analyzed using a chi-square test. ADDRESS, ADministration of DRotrecogin alfa (activated) in Early Stage Severe Sepsis; BE, bleeding event reported as an adverse event; SBE, bleeding event reported as a serious adverse event.

Mortality for patients experiencing any bleeding event in ADDRESS was higher than for patients not experiencing a bleeding complication, irrespective of treatment group. For placebo patients, 28-day mortality rates were 27.7% (95% confidence interval [CI] 18.1% to 37.3%) and 16.3% (95% CI 14.2% to 18.4%) for patients who did and did not experience a bleeding complication, respectively. For DrotAA, 28-day mortality rates were 40.1% (95% CI 32.1% to 48.2%) and 15.8% (95% CI 13.8% to 17.9%) for patients who did and did not experience a bleeding complication, respectively. The percentage of patients who experienced a bleeding event and subsequently died was significantly less for DrotAA patients in the ≥3 subgroup compared with the 2 subgroup (3.4% versus 6.1%; P = 0.02), whereas there was no difference between subgroups in placebo patients (1.9% versus 1.7%; P = 0.68) (Table 4). An analysis of baseline characteristics for patients who experienced any bleeding event indicated that these patients were more severely ill compared with patients who did not have a bleeding event (data not shown). Consequently, the presence of a bleeding complication could have been a marker for higher disease severity; hence, these patients might have been expected to have higher mortality.

Sequence effect in selected subgroups

As statistically significant treatment-by-sequence of enrollment interactions were present in both ADDRESS and PROWESS, selected subgroups from both studies were also examined. Figure 2 displays 28-day mortality for patients with MOD enrolled in ADDRESS or PROWESS for subgroups in which the first through fourth patients enrolled at each site were excluded from the analysis. A treatment-by-sequence of enrollment interaction was present in the ADDRESS (P = 0.006) but not in the PROWESS MOD subpopulations. A similar analysis was performed for the APACHE II score ≥25 subgroup (Figure 3). As with the MOD subgroup in PROWESS, no treatment-by-sequence of enrollment interaction was present in PROWESS patients with APACHE II scores ≥25. However, in the ADDRESS subgroup, we observed a significant interaction (P = 0.01) that appeared to result from an increase in the mortality rate for placebo patients, despite the absence of an increase in the APACHE II score as sites enrolled more patients. There was not a parallel increase in mortality for the DrotAA patients.
Figure 2

Twenty-eight-day mortality in all randomly assigned patients with multiple organ dysfunction in ADDRESS (a) and PROWESS (b) with no patients removed from the analysis and also with the first through fourth patients enrolled at each site removed. Note: 0 represents the results for the entire population, and 1 through 4 correspond to the analysis with the first through fourth patients from each site removed. ADDRESS, ADministration of DRotrecogin alfa (activated) in Early Stage Severe Sepsis; DrotAA, drotrecogin alfa (activated); MOD, multiple organ dysfunction; PBO, placebo; PROWESS, Protein C Worldwide Evaluation in Severe Sepsis.

Figure 3

Twenty-eight-day mortality in all randomly assigned patients with an APACHE II score of greater than or equal to 25 in ADDRESS (a) and PROWESS (b) with no patients removed from the analysis and also with the first through fourth patients enrolled at each site removed. Note: 0 represents the results for the entire population, and 1 through 4 correspond to the analysis with the first through fourth patients from each site removed. ADDRESS, ADministration of DRotrecogin alfa (activated) in Early Stage Severe Sepsis; APACHE, Acute Physiology and Chronic Health Evaluation; DrotAA, drotrecogin alfa (activated); PBO, placebo; PROWESS, Protein C Worldwide Evaluation in Severe Sepsis.

Similar analyses were conducted for subpopulations defined as single-organ dysfunction surgical patients (Figure 4) and single-organ dysfunction medical patients (Figure 5). For single-organ dysfunction surgical patients, no treatment-by-sequence of enrollment interaction was present in either the ADDRESS or the PROWESS study. However, mortality was higher for DrotAA patients compared with placebo patients in ADDRESS (20.7% versus 14.1%, n = 636) and was similar to placebo patients in PROWESS (Figure 4). In ADDRESS, an analysis by type of single-organ dysfunction in surgical patients suggested that the higher mortality observed in DrotAA compared with placebo patients was due to those patients with isolated respiratory failure (21.1% versus 10.4%; P < 0.05). Strong trends for treatment interactions were present in the single-organ dysfunction medical patients in both the ADDRESS (P = 0.051) and PROWESS (P = 0.058) trials (Figure 5). As the above results suggested that any treatment-by-sequence of enrollment interaction might be most apparent in lower-risk patients, the entire ADDRESS population was compared with the lower-risk population in PROWESS as defined by an APACHE II score of less than 25 (Figure 6). A treatment-by-sequence of enrollment interaction was present in ADDRESS (P = 0.04) and only a trend in PROWESS (P = 0.11).
Figure 4

Twenty-eight-day mortality in all randomly assigned surgical patients with single organ dysfunction in ADDRESS (a) and PROWESS (b) with no patients removed from the analysis and also with the first through fourth patients enrolled at each site removed. Note: 0 represents the results for the entire population, and 1 through 4 correspond to the analysis with the first through fourth patients from each site removed. ADDRESS, ADministration of DRotrecogin alfa (activated) in Early Stage Severe Sepsis; DrotAA, drotrecogin alfa (activated); OD, organ dysfunction; PBO, placebo; PROWESS, Protein C Worldwide Evaluation in Severe Sepsis.

Figure 5

Twenty-eight-day mortality in all randomly assigned medical patients with single organ dysfunction in ADDRESS (c) and PROWESS (d) with no patients removed from the analysis and also with the first through fourth patients enrolled at each site removed. Note: 0 represents the results for the entire population, and 1 through 4 correspond to the analysis with the first through fourth patients from each site removed. ADDRESS, ADministration of DRotrecogin alfa (activated) in Early Stage Severe Sepsis; DrotAA, drotrecogin alfa (activated); OD, organ dysfunction; PBO, placebo; PROWESS, Protein C Worldwide Evaluation in Severe Sepsis.

Figure 6

Twenty-eight-day mortality in all randomly assigned patients in ADDRESS (a) and in PROWESS (b) with an APACHE II score of less than 25, with no patients removed from the analysis, and also with the first through fourth patients enrolled at each site removed. Note: 0 represents the results for the entire population, and 1 through 4 correspond to the analysis with the first through fourth patients from each site removed. ADDRESS, ADministration of DRotrecogin alfa (activated) in Early Stage Severe Sepsis; APACHE, Acute Physiology and Chronic Health Evaluation; DrotAA, drotrecogin alfa (activated); PBO, placebo; PROWESS, Protein C Worldwide Evaluation in Severe Sepsis.

Discussion

In both PROWESS and ADDRESS, there appeared to be an influence of enrollment sequence within a site on the observed treatment effect with DrotAA. Analyses of the ADDRESS and PROWESS studies suggest that the enrollment sequence effect or 'learning curve' may be largely confined to the populations with a lower risk of death and may relate to difficulties in diagnosis, timely diagnosis, or a situation in which small absolute reductions in mortality may not overcome the complications of therapy.

In ADDRESS, as in the PROWESS study, there was a statistically significant interaction between the observed treatment effect associated with DrotAA and the sequence in which patients were enrolled at study sites. In both trials, greater mortality was found with DrotAA use within the subgroup that comprised the first patients enrolled at study sites. In PROWESS, mortality was lower with DrotAA treatment for the second and subsequent patients enrolled at a site. However, in ADDRESS, mortality remained higher with DrotAA compared with placebo for the second patients enrolled. Thereafter, mortality was lower for DrotAA-treated patients compared with placebo.

A comparison of baseline characteristics for patients in ADDRESS who comprised the first two patients enrolled at a site compared with the third and subsequent patients indicated that there were changes in the characteristics of patients enrolled in the study as the site gained experience with the protocol. Specifically, differences were observed between the subgroups in the racial origin of patients, acute physiology scores, percentage of patients with chronic health points, surgical status, and heparin use. The mean time from the onset of first organ failure to treatment with study drug decreased from 24.1 to 21.8 hours for the first two patients compared with subsequent patients (P < 0.001). The observed relative risk for DrotAA was similar between studies for patients enrolled in the third and subsequent patient blocks (0.82 versus 0.80 and 0.69 versus 0.69 for patients in the third block and subsequent blocks, respectively). An analysis of outcomes within clinically relevant subgroups suggested that any sequence effect is contained within the population of patients at lower risk of death. These data suggest that a potential sequence effect was present in the ADDRESS study as was observed in PROWESS.

An analysis of bleeding complications by sequence of enrollment in ADDRESS also indicated that sites enrolled patients at lower risk of bleeding in both the DrotAA and placebo groups. The percentage of patients experiencing any bleeding complications declined in both treatment groups as sites enrolled more patients into the study. Patients in the ≥3 subgroup who received DrotAA received fewer transfusions compared with those patients in the ≤2 subgroup. In both treatment groups, mortality was higher for patients experiencing any bleeding complications compared with those who did not have a bleeding event. As bleeding events were more common in the DrotAA groups, there were more patients in this group compared with placebo who experienced any bleeding event and did not survive. Patients experiencing a bleeding event were more severely ill than those who did not experience a bleeding event, so a direct contribution to the cause of death could not be determined but certainly cannot be excluded.

Determining the influence of sequential changes in patient characteristics on the outcome of ADDRESS is difficult. Previous data have suggested that the observed treatment effect associated with DrotAA is larger if treatment is initiated within 24 hours of first organ failure [8]. The presence of a sequence effect in low-risk patients only may suggest the difficulty of making the diagnosis of severe sepsis in this clinical setting. The influence of protocol violations on outcome could not be assessed because less than 50% of patients were monitored. However, protocol violations reduced the observed treatment effect in PROWESS [3]. Fewer bleeding complications and transfusions in DrotAA patients comprising the third and subsequently enrolled subgroup could have contributed to the lower observed mortality. However, an important aspect of these data is that the population of patients enrolled in a clinical study may change as sites gain experience with a protocol and patient identification, leading to change in the observed treatment effect associated with an intervention as sites enroll more patients in the study. These observations have important implications for the design of future trials. For example, the ongoing PROWESS shock study (a randomized placebo-controlled trial of DrotAA in adults with persistent shock) is using an academic coordinating center to approve every patient enrolled into the study. In addition, this trial has robust source data verification and documentation of protocol violations which will allow for re-education if the sites demonstrate inadequate understanding of the protocol.

Learning curves have been described in other clinical trials and in clinical practice. Halm and colleagues [9] performed a literature review of studies examining the independent relationship between hospital or physician volume and clinical outcomes. They found that, overall, 71% of all studies of hospital volume and 69% of studies of physician volume reported statistically significant associations between higher volume and better outcomes. The strongest associations were found for AIDS treatment and for surgery on pancreatic cancer, esophageal cancer, abdominal aortic aneurysms, and pediatric cardiac problems. However, it is not always clear what exactly is responsible for these different outcomes.

While it is perhaps easier to explain a sequence effect in relation to surgical procedures, as well as AIDS treatments noted earlier, there have been a number of studies investigating the relationship between outcome and clinical experience with the use of tissue-type plasminogen activator (tPA) in ischemic stroke. Heuschmann and colleagues [10] found that data from the German Stroke Registers Study Group indicated that in-hospital mortality of ischemic stroke patients after tPA use varied among hospitals with different experience in tPA treatment in routine clinical practice. There have been additional studies from different countries indicating that tPA should be administered by experienced physicians in hospitals with expertise in acute stroke care and that published treatment guidelines should be followed [1115]. Similar to the findings associated with following tPA treatment guidelines, adherence to the American College of Critical Care Medicine-Pediatric Advanced Life Support guidelines for hemodynamic support for newborns and children in septic shock has been noted to be associated with improved survival [16].

Data analyses based on secondary objectives in a clinical study and post hoc exploratory analyses suffer from concerns over the play of chance in multiple comparisons as well as reporting bias. Primary and secondary analyses have been reported for both the PROWESS and ADDRESS studies. The initial report of a treatment-by-enrollment interaction in PROWESS was considered hypothesis-generating. However, this finding in a subsequent trial suggests that such a sequence effect may have important implications for study design and data interpretation. Nevertheless, such findings do not alter the overall conclusions of these studies.

The exact cause of the 'enrollment sequence' remains uncertain. We have demonstrated associated changes in patient characteristics, but it is not clear whether these or other 'unrecorded' differences may be responsible. Changes in patient management, particularly in relation to DrotAA administration, could also be important, but not all of the details of patient management were collected. Two factors that could be important are time to treatment, which has previously been linked to improved outcome with DrotAA treatment [17], and the frequency of bleeding events. We hypothesize that the 'sequence effect' may be more apparent in populations with lower mortality risk because the absolute mortality reduction is anticipated to be less in this population and thus it may be more difficult to demonstrate efficacy in this situation unless treatment conditions are optimized. The 'enrollment sequence' may thus have been more apparent in ADDRESS because much of this population was at low risk.

Conclusion

Analysis of the ADDRESS and PROWESS studies suggests that an enrollment sequence effect was present in both studies. In PROWESS, this enrollment sequence effect reduced the observed treatment effect associated with DrotAA. In ADDRESS, this enrollment sequence effect may have contributed to the decision to recommend early termination of the study for futility. The finding of such an enrollment sequence effect in two separate clinical trials in severe sepsis suggests that trial designs, site training, data collection and monitoring, and statistical analysis plans need to be adjusted for these potential confounders. A list of potential recommendations to assist in the design of future clinical trials is shown in Table 5.
Table 5

Potential recommendations to assist in the design of future clinical trials

Recommendation

Potential benefits

Potential issues

More extensive inclusion and exclusion criteria that are more descriptive of the population to be enrolled

Less opportunity for patient variability and sites having to 'learn as they go'

Lower likelihood of extrapolating efficacy observed in the clinical trial to effectiveness in clinical practice

Standardize the major facets of severe sepsis concomitant care

Reduced variability as caring for patients with severe sepsis may be a more complex 'procedure' than many commonly performed surgical procedures

May be questions related to the applicability of the study results to a more general severe sepsis population, in which concomitant care has not been standardized

Given the heterogeneity of severe sepsis patients, different populations of patients may require unique sets of inclusion and exclusion criteria (for example, medical patients and surgical patients).

Optimizes inclusion and exclusion criteria without the extra time and resources that would be needed to run two separate studies

May be issues with interpretation of data and powering if the treatment effect differs significantly between the two populations, in which two separate studies may be preferable

Use a clinical coordinating center to assist study sites in enrollment of eligible patients.

Helps to optimize protocol compliance

May be questions related to the applicability of the study results to a more general severe sepsis population

Site selection should be based on having good clinical trial and critical care experience.

Helps to minimize variability and (potentially) protocol violations

May be questions related to the applicability of the study results to a more general severe sepsis population

The use of severity scoring systems in clinical trials may require training and validation of the training to ensure proper collection of severity of illness information.

Helps to ensure the collection of accurate data

Additional time and resources required

Given the potential influence of site experience on outcomes, randomization stratified by site should be considered a requirement for studies in complex disease states.

Helps to minimize effect of differences between sites and enrollment sequence effect

May limit ability to stratify by additional parameters

Planned enrollment per site should be based on the block size used for randomization. Expected enrollment per site should be at least two full blocks of patients.

Helps to minimize enrollment sequence effect

May only be able to have larger sites in the study, raising questions related to generalizability of the results

Futility stopping rules should incorporate the potential for learning curves to obscure a beneficial treatment effect in the early stages of a clinical trial.

Helps to avoid type II error

May be a delay in identifying futility signals if no enrollment sequence effect is present

Statistical analysis plans should explore the potential for learning curves within the clinical trial dataset.

Prospectively defined analyses have greater weight and may help to explain study findings

Additional workload

Clinical studies should have a prospectively defined monitoring plan. Source data verification and documentation of protocol violations should be performed on the first few patients enrolled at a site until the site demonstrates adequate understanding of the protocol.

Helps to minimize protocol violations

Additional time and resources required

Key messages

  • In the ADDRESS and PROWESS studies, a significant interaction between drotrecogin alfa (activated) treatment effect and the sequence in which patients were enrolled was observed.

  • Characteristics of patients enrolled first changed compared with subsequent patients, including earlier treatment and fewer protocol violations in the latter group.

  • The enrollment sequence effect observed in two separate trials suggests the need for analyses adjusted for confounding events.

  • Future study designs in severe sepsis should include site selection and training, close monitoring of strict protocol adhesion, sufficient recruitment per site, and a clinical coordinating center to ensure adequate patient selection.

Abbreviations

ADDRESS: 

Administration of Drotrecogin Alfa (Activated) in Early Stage Severe Sepsis

APACHE: 

Acute Physiology and Chronic Health Evaluation

CI: 

confidence interval

DrotAA: 

drotrecogin alfa (activated)

MOD: 

multiple-organ dysfunction

PROWESS: 

Protein C Worldwide Evaluation in Severe Sepsis

tPA: 

tissue-type plasminogen activator.

Declarations

Acknowledgements

We would like to acknowledge Karen Shields for her editorial assistance and David Sundin, who provided technical, administrative, and editorial services on behalf of Eli Lilly and Company in preparation of the manuscript.

Authors’ Affiliations

(1)
St Luc University Hospital
(2)
Lilly Research Laboratories, Eli Lilly and Company
(3)
Eli Lilly Erl Wood Manor
(4)
Department of Intensive Care, VU University Medical Center
(5)
Paris Descartes University
(6)
University of Alabama at Birmingham School of Medicine

References

  1. Bernard GR, Vincent JL, Laterre PF, LaRosa SP, Dhainaut JF, Lopez-Rodriguez A, Steingrub JS, Garber GE, Helterbrand JD, Ely EW, Fisher CJ, Recombinant Human Activated Protein C Worldwide Evaluation in Severe Sepsis (PROWESS) Study Group: Efficacy and safety of recombinant human activated protein C for severe sepsis. N Engl J Med 2001, 344: 699-709. 10.1056/NEJM200103083441001View ArticlePubMedGoogle Scholar
  2. Ely EW, Laterre PF, Angus DC, Helterbrand JD, Levy H, Dhainaut JF, Vincent JL, Macias WL, Bernard GR, PROWESS Investigators: Drotrecogin alfa (activated) administration across clinically important subgroups of patients with severe sepsis. Crit Care Med 2003, 31: 12-19. 10.1097/00003246-200301000-00002View ArticlePubMedGoogle Scholar
  3. Macias WL, Vallet B, Bernard GR, Vincent JL, Laterre PF, Nelson DR, Derchak PA, Dhainaut JF: Sources of variability on the estimate of treatment effect in the PROWESS trial: implications for the design and conduct of future studies in severe sepsis. Crit Care Med 2004, 32: 2385-2391. 10.1097/01.CCM.0000147440.71142.ACView ArticlePubMedGoogle Scholar
  4. EPARs for authorised medicinal products for human use – Xigris[http://www.emea.europa.eu/humandocs/Humans/EPAR/xigris/Xigris.htm]
  5. Xigris™ Drotrecogin alfa (activated) package insert[http://www.fda.gov/cder/foi/label/2001/droteli112101LB.pdf]
  6. Drotrecogin alfa (activated) Product Approval Information – Licensing Action[http://www.fda.gov/cder/foi/appletter/2001/droteli112101L.htm]
  7. Abraham E, Laterre PF, Garg R, Levy H, Talwar D, Trzaskoma BL, Francois B, Guy JS, Bruckmann M, Rea-Neto A, Rossaint R, Perrotin D, Sablotzki A, Arkins N, Utterback BG, Macias WL, Administration of Drotrecogin alfa (activated) in Early Stage Severe Sepsis (ADDRESS) Study Group: Drotrecogin alfa (activated) for adults with severe sepsis and a low risk of death. N Engl J Med 2005, 353: 1332-1341. 10.1056/NEJMoa050935View ArticlePubMedGoogle Scholar
  8. Vincent JL, Angus DC, Artigas A, Kalil A, Basson BR, Jamal HH, Johnson G 3rd, Bernard GR, Recombinant Human Activated Protein C Worldwide Evaluation in Severe Sepsis (PROWESS) Study Group: Effects of drotrecogin alfa (activated) on organ dysfunction in the PROWESS trial. Crit Care Med 2003, 31: 834-840. 10.1097/01.CCM.0000051515.56179.E1View ArticlePubMedGoogle Scholar
  9. Halm EA, Lee C, Chassin MR: Is volume related to outcome in health care? A systematic review and methodologic critique of the literature. Ann Intern Med 2002, 137: 511-520.View ArticlePubMedGoogle Scholar
  10. Heuschmann PU, Berger K, Misselwitz B, Hermanek P, Leffmann C, Adelmann M, Buecker-Nott HJ, Rother J, Neundoerfer B, Kalominsky-Rabas PL, for the German Stroke Registers Study Group and for the Competence Net Stroke: Frequency of thrombolytic therapy in patients with acute ischemic stroke and the risk of in-hospital mortality. Stroke 2003, 34: 1106-1113. 10.1161/01.STR.0000065198.80347.C5View ArticlePubMedGoogle Scholar
  11. Katzan IL, Furlan AJ, Lloyd LE, Frank JI, Harper DL, Hinchey JA, Hammel JP, Qu A, Sila CA: Use of tissue-type plasminogen activator for acute ischemic stroke: the Cleveland area experience. JAMA 2000, 283: 1151-1158. 10.1001/jama.283.9.1151View ArticlePubMedGoogle Scholar
  12. Gladstone DJ, Black SE: Update on intravenous tissue plasminogen activator for acute stroke: from clinical trials to clinical practice. CMAJ 2001, 165: 311-317.PubMed CentralPubMedGoogle Scholar
  13. Grotta JC, Burgin WS, El-Mitwalli A, Long M, Campbell M, Morgenstern LB, Malkoff M, Alexandrov AV: Intravenous tissue-type plasminogen activator therapy for ischemic stroke: Houston experience 1996 to 2000. Arch Neurol 2001, 58: 2009-2013. 10.1001/archneur.58.12.2009View ArticlePubMedGoogle Scholar
  14. Graham GD: Tissue plasminogen activator for acute ischemic stroke in clinical practice: a meta-analysis of safety data. Stroke 2003, 34: 2847-2850. 10.1161/01.STR.0000101752.23813.C3View ArticlePubMedGoogle Scholar
  15. Szoeke CE, Parsons MW, Butcher KS, Baird TA, Mitchell PJ, Fox SE, Davis SM: Acute stroke thrombolysis with intravenous tissue plasminogen activator in an Australian tertiary hospital. Med J Aust 2003, 178: 324-328.PubMedGoogle Scholar
  16. Han YY, Carcillo JA, Dragotta MA, Bills DM, Watson RS, Westerman ME, Orr RA: Early reversal of pediatric-neonatal septic shock by community physicians is associated with improved outcome. Pediatrics 2003, 112: 793-799. 10.1542/peds.112.4.793View ArticlePubMedGoogle Scholar
  17. Vincent JL, Bernard GR, Beale R, Doig C, Putensen C, Dhainaut JF, Artigas A, Fumagalli R, Macias W, Wright T, Wong K, Sundin DP, Turlo MA, Janes J: Drotrecogin alfa (activated) treatment in severe sepsis from the global open-label trial ENHANCE: further evidence for survival and safety and implications for early treatment. Crit Care Med 2005, 33: 2266-2277. 10.1097/01.CCM.0000181729.46010.83View ArticlePubMedGoogle Scholar

Copyright

© Laterre et al.; licensee BioMed Central Ltd. 2008

This article is published under license to BioMed Central Ltd. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.