Implementation of sepsis bundles in public hospitals in Brazil: a prospective study with heterogeneous results

Abstract

Background

Public hospitals in emerging countries pose a challenge to quality improvement initiatives in sepsis. Our objective was to evaluate the results of a quality improvement initiative in sepsis in a network of public institutions and to assess potential differences between institutions that did or did not achieve a reduction in mortality.

Methods

We conducted a prospective study of patients with sepsis or septic shock. We collected baseline data on compliance with the Surviving Sepsis Campaign 6-h bundles and mortality. Afterward, we initiated a multifaceted quality improvement initiative for patients with sepsis or septic shock in all hospital sectors. The primary outcome was hospital mortality over time. The secondary outcomes were the time to sepsis diagnosis and compliance with the entire 6-h bundles throughout the intervention. We defined successful institutions as those where the mortality rates decreased significantly over time, using a logistic regression model. We analyzed differences over time in the secondary outcomes by comparing the successful institutions with the nonsuccessful ones. We assessed the predictors of in-hospital mortality using logistic regression models. All tests were two-sided, and a p value less than 0.05 indicated statistical significance.

Results

We included 3435 patients from the emergency departments (50.7%), wards (34.1%), and intensive care units (15.2%) of 9 institutions. Throughout the intervention, there was an overall reduction in the risk of death, in the proportion of septic shock, and the time to sepsis diagnosis, as well as an improvement in compliance with the 6-h bundle. The time to sepsis diagnosis, but not the compliance with bundles, was associated with a reduction in the risk of death. However, there was a significant reduction in mortality in only two institutions. The reduction in the time to sepsis diagnosis was greater in the successful institutions. By contrast, the nonsuccessful sites had a greater increase in compliance with the 6-h bundle.

Conclusions

Quality improvement initiatives reduced sepsis mortality in public Brazilian institutions, although not in all of them. Early recognition seems to be a more relevant factor than compliance with the 6-h bundle.

Background

Although sepsis is still a leading cause of mortality worldwide, mortality rates are decreasing [1], especially in developed countries [2, 3]. However, low- and middle-income countries (LMICs) are responsible for a significant portion of the sepsis burden [4], and mortality rates are still very high in these settings [5,6,7,8]. Quality improvement initiatives are successful in reducing fatality rates in high-income countries [9], but there are only a few reports of such initiatives in LMICs [10,11,12,13,14,15]. In Brazil, some studies have already shown higher mortality rates in public institutions than in private ones [16, 17]; also, public institutions have worse compliance with treatment quality indicators, and mortality reductions in quality improvement initiatives seem not to be sustained [16, 18].

Public hospitals in emerging countries pose a challenge to quality improvement initiatives. These facilities usually have considerable limitations, including infrastructure issues; low availability of resources [19]; low availability of intensive care unit (ICU) beds [20]; a shortage of healthcare professionals [21]; inadequate staff qualification and high turnover [22, 23]; and overcrowding, especially in the emergency department (ED). One of the main barriers is the low awareness of sepsis [23], leading to late recognition with a consequent delay in intervention and increased lethality [16, 24]. Therefore, one of the main goals of any quality improvement initiative should be to train hospital staff to identify those patients in earlier phases. Early recognition and training are associated with a reduction in the severity of illness among patients with sepsis, as well as with lower severity scores and less organ dysfunction at the time of the sepsis diagnosis [15].

Therefore, we aimed to assess whether a quality improvement initiative in a network of public hospitals in Brazil would decrease mortality and the factors that would be associated with this reduction. On the basis of our previous experience in public hospitals [18], we anticipated that the success of the initiative would be heterogeneous among the hospitals; therefore, our secondary objective was to identify the quality indicators associated with a reduction in mortality rates.

Methods

We conducted a prospective study in a network of nine public hospitals with a central administration. The institutions were instructed to include clinical and sepsis management variables from all patients admitted to the ED, wards, or ICUs with a diagnosis of sepsis or septic shock in a centralized electronic database [25]. We defined sepsis as the presence of infection complicated by acute organ dysfunction (previously called severe sepsis) [17], as detailed in the Additional file. Septic shock was defined by the presence of refractory hypotension with the need for vasopressors. Patients could be included in the database only once in the same hospital admission. Patients receiving end-of-life care were excluded. The research and ethics committee of Universidade Federal de São Paulo approved the study (1387/10) and waived the need for informed consent because of the nature of the study.

Intervention

The quality improvement initiative was conducted as suggested by the Latin American Sepsis Institute (LASI) [15]. It was divided into two phases. All institutions were asked to select a case manager responsible for the project. After training, each case manager was instructed to collect data on all cases of sepsis and septic shock based on an active search for those patients in the main hospital areas over 3 months as a baseline. The LASI suggested strategies for the active search, such as performing an audit of all prescribed antibiotics. Data on the patients’ characteristics, compliance with the Surviving Sepsis Campaign (SSC) 6-h bundle (Additional file 1: Table S1), and hospital mortality were collected. To assess compliance, we considered the moment of sepsis recognition by the healthcare team, instead of the time of organ dysfunction onset, as time zero. During this baseline period, all institutions were also asked to create a sepsis team. The institutions had to develop treatment protocols, a guide for empiric antibiotic therapy, and a screening tool based on the presence of signs of systemic inflammatory response syndrome or organ dysfunction (reduced level of consciousness, dyspnea or oxygen requirement, hypotension, or low urine output) that could be used in the ED, wards, and ICUs. They were also asked to establish a routine for laboratory tests, allowing for more rapid reporting of results, as well as a routine for timely supply of antibiotics to all hospital sectors.

In the second phase, we started a continuous education program that aimed to train all nurses and physicians. All hospital staff were invited to participate in an 8-h course provided by the LASI with basic knowledge about sepsis recognition and treatment. Video lectures and an e-learning course were available at the LASI website. Educational materials, such as flowcharts, panels, and folders, were widely distributed in all institutions. Data collection was continued throughout the 2-year intervention, and each institution received a quarterly performance report from the LASI. The report summarized compliance with the bundle items, as well as hospital mortality data for each hospital sector, and provided benchmarking with the network and the entire LASI database. The results were discussed with the sepsis team and other healthcare professionals in quarterly joint meetings with the LASI team; at these meetings, opportunities for improvement and new strategies for optimization were discussed. Although LASI provided all tools and information to allow a similar intervention in all sites, ultimately local implementation was highly dependent on each institution.

Data collection

All information was collected using dedicated software that was specifically developed for this project, and there were automatic checks of data entry to improve the data completeness and consistency. All data were input prospectively by the case manager of the institution. The entire database was confidential, and each institution had access to only its own data.

We collected data on the patients’ demographics; sepsis characteristics; location at organ dysfunction onset; time to sepsis diagnosis; and severity of illness as assessed by the Acute Physiology and Chronic Health Evaluation II (APACHE II) score, Sequential Organ Failure Assessment (SOFA) score, and total number of organs with dysfunction at the time of the sepsis diagnosis. We also collected data on the compliance with the SSC 6-h bundle. We registered ICU admission in only the first 24 h of the sepsis diagnosis. Patients were followed until hospital discharge.

Outcomes

The primary outcome was hospital mortality over time. The predefined secondary outcomes were the time to sepsis diagnosis and compliance with the entire 6-h bundles throughout the intervention. As a secondary outcome, we also considered compliance with the 6-h bundle items that are required for all patients, including lactate sampling, blood cultures, and antibiotic administration. On the basis of our previous study [15], we hypothesized that the implementation would improve sepsis awareness with a reduction in the time to sepsis diagnosis and disease severity. Therefore, we included the percentage of sepsis in all patients, APACHE II score, SOFA score, and total number of organs with dysfunction as secondary outcomes.

Definitions

On the basis of our previous experience with the quality improvement program in the LASI [18], we expected that the mortality rates would not decrease in some institutions. Thus, we defined successful institutions as those where the mortality rates decreased significantly over time, and those in which the mortality rates did not drop were defined as nonsuccessful institutions. We used logistic regression with quarter and institution as explanatory variables and death as a dependent variable. The institutions that presented a statistically significant reduction in mortality OR according to quarter change were considered as the successful institutions.

Statistical analysis

The categorical variables are expressed as absolute numbers and percentages, and the continuous variables are reported as measures of central tendency and dispersion according to their distribution (mean [SD] or median [IQR]) as assessed by the Kolmogorov-Smirnov test.

Primary and secondary outcomes

We analyzed the primary and secondary outcomes throughout the intervention using generalized linear models to assess their trend over quarter changes in the entire population [26]. Generalized linear models were used, considering both the quarter and institutions as explanatory variables, accounting for the cluster effect. We also aimed to analyze differences over time in both the primary and secondary outcomes by comparing the successful institutions with the nonsuccessful ones. We also calculated the rate of decline (quarter mean time to sepsis diagnosis/quarter mean time to sepsis diagnosis + 1 in the next quarter), using generalized linear models with gamma distribution and log-link function to evaluate if the rate of decline was different between the successful and nonsuccessful institutions.

Prognostic factors associated with mortality

In the univariate analysis to assess the prognostic factors for mortality, we used Fisher’s exact test or the chi-square test for categorical data and the Mann-Whitney U test for continuous data without a normal distribution. We assessed the predictors of in-hospital mortality using logistic regression models with random effects at the institution level to account for the clustering effect [26]. We included in the model all variables with a p value < 0.05 in the univariate analysis and institution type (successful and nonsuccessful); however, we did not include lactate ≥ 4 mmol/L, owing to excessive missing data and the individual dysfunctions because they were already assessed in the SOFA score. Because we considered compliance with fluids and vasopressors a variable with special interest, we ran a second model including this variable. We also ran a sensitivity analysis excluding the institutions that did not include patients in the last quarters of the intervention. We present the general OR with 95% CI for those variables and, when there was a significant difference between institutions, individually according to the institution type. All tests were two-sided, and a p value < 0.05 indicated statistical significance. Analyses were performed using R 3.2.2 software (2014; R Core Team, Vienna, Austria).

Results

From September 2010 to August 2012, we included 3435 patients from the 9 participating hospitals. The baseline characteristics of the patients throughout the intervention are available in Table 1, and the individual results are provided in Additional file 1: Table S2. There was no major change in the patients’ characteristics, except for discreet changes in the frequency of some comorbidities, the urinary tract as a source of infection, and the location at sepsis presentation.

Table 1 Study population characteristics throughout the intervention

There was a significant reduction in the mortality rates over time with p < 0.0001 (Table 2). This reduction was significant for patients with sepsis and septic shock. There was also an improvement in the secondary outcomes with a significant reduction in the time to sepsis diagnosis and an increased percentage of patients diagnosed with sepsis instead of septic shock. Concomitantly, there was a reduction in the disease severity, as measured by the APACHE II score, SOFA score, and total number of organs with dysfunction. The compliance with the first three required items of the 6-h bundle, as well as compliance with the entire bundle, also increased significantly.

Table 2 Outcomes throughout the intervention in the whole population and according to the success of the institutions

In the logistic regression model, the time to sepsis diagnosis was independently associated with increased mortality (Table 3) in both institution types. By contrast, neither compliance with the entire 6-h bundle nor compliance with antibiotics or fluids and vasopressors (Additional file 1: Table S3) was associated with a reduction in the risk of death. The results of the sensitivity analysis excluding the institutions that did not include patients until the last quarter of the intervention showed similar results (Additional file 1: Table S4). The results of the univariate analysis are available in Additional file 1: Table S5. Other classical variables were also independently associated with a higher risk of death, such as age, cancer and illness severity, and presenting with sepsis in the wards.

Table 3 Factors associated with mortality in the whole population and according to the success of the institutions by multivariate analysis

Only two institutions had a significant reduction in mortality rates along the quarters (Additional file 1: Table S6 and Figure S1). The reduction in mortality rates can be seen in Fig. 1a. The baseline characteristics of the patients are available in Additional file 1: Table S7. There were some differences in the patient profiles. The percentage of patients from the ED was higher in the successful groups, but these patients also had a higher frequency of comorbidities. These sites were analyzed together (successful institutions) and compared with the institutions without success (nonsuccessful institutions), as shown in Table 2. Although the time to sepsis diagnosis was significantly reduced in both groups, the reduction was greater at the successful sites (Fig. 1b). Although the time curves decayed in parallel, the absolute numbers in the last quarter were smaller in the successful ones. The median time to sepsis diagnosis was still 2.0 (1.0–7.5) in the nonsuccessful hospitals, which was significantly different from the successful ones (0.8 [0.4–2.0], p < 0.0001). We also calculated the mean ratio for one-trimester difference for both types of institutions. In the successful ones, the ratio of decline was 0.777 (0.752–0.804, p < 0.0001), indicating that there was, on average, a 23% of decline in the time to sepsis diagnosis in each quarter of the intervention. By contrast, the ratio of decline in the nonsuccessful hospitals was 0.868 (0.675–1.116, p = 0.270), indicating that the mean ratio of decline did not differ between the quarters. We also found a greater increase in the proportion of patients with sepsis than in those with septic shock at the successful institutions. By contrast, compared with the successful sites, the nonsuccessful sites had a significant increase in compliance with the bundles for both the required items and the entire 6-h bundle (Fig. 1c).

Fig. 1

Changes in outcomes throughout the intervention. a Hospital mortality. b Time to sepsis diagnosis. c Full compliance with the 6-h bundle. Generalized linear models for comparison between the two types of institutions: mortality (p < 0.0001), time to sepsis diagnosis (p < 0.0001), and full compliance with the 6-h bundle (p = 0.0160)

Discussion

Our results show that the implementation of a sepsis protocol in public institutions in an emerging country led to an overall reduction in the risk of death, proportion of septic shock cases, time to sepsis diagnosis, and improvement in compliance with all 6-h bundle items. These results might be associated with earlier sepsis recognition rather than compliance with treatment indicators. Our findings also suggest that the impact of the implementation of sepsis protocols in public institutions in an emerging country is variable and that mortality reduction is achieved only in some institutions. The successful institutions seemed to be able to reduce the time to sepsis diagnosis more aggressively than the nonsuccessful ones.

Quality improvement initiatives in emerging countries face many challenges [27]. We observed that time to sepsis diagnosis was independently associated with a reduction in the risk of death, although compliance with the 6-h bundle with antibiotics or fluids was not. Although previous studies have demonstrated that compliance with the 6-h bundle and antibiotics was associated with a mortality reduction [15, 28], we were not able to replicate these findings. One explanation is that compliance in this study was measured with time zero beginning at the time of sepsis diagnosis, as acknowledged in the patients’ charts, and not at the time of organ dysfunction onset. Any potential effect of compliance on mortality would be better assessed if compliance were measured according to organ dysfunction onset and not in relation to the moment of sepsis diagnosis. Therefore, compliance alone, not considering the time to sepsis diagnosis, will hardly be associated with mortality reduction in settings where late recognition is frequent. Unfortunately, time zero for calculating compliance is not consistently reported in quality improvement studies. Our findings reinforce the key role of raising awareness, because delayed, although adequate, treatment might not result in optimal survival rates. Another potential explanation is that among the bundle items, some are direct interventions such as antibiotics and others are diagnostic tools, and their impact on mortality will be highly dependent on physicians’ management and not directly on compliance, such as lactate sampling or blood cultures, and we did not assess the adequacy of treatment in a broader way.

As previously shown [29], we observed a trend toward a reduction in disease severity throughout the study period, although none of the hospitals changed their admission profiles. The reduction in the time to sepsis diagnosis could have contributed to a higher percentage of patients being diagnosed with sepsis instead of in the latter stages of septic shock. The early diagnosis also possibly contributed to earlier treatment and therefore could have reduced the odds of progression to septic shock. This hypothesis is reinforced by the finding of a significant reduction in the SOFA score and number of organs with dysfunction. We might argue that a modification in the institution profile could have contributed to these findings. However, there was no relevant change in the baseline characteristics of the patients, except by discreet and irrelevant oscillations in the frequency of some comorbidities. Some differences might have been influenced by our intervention. For example, the increased inclusion of ED patients may indicate a higher awareness of the ED staff. The reduction in the frequency of pneumonia as the source of infection may be a consequence of training and, consequently, the correct diagnosis of respiratory dysfunction secondary to other sources.

The old sepsis definition was used in this study because it was a quality improvement initiative carried out before the new Sepsis 3.0 guidelines definition was available. However, because the Sepsis 3 guidelines define sepsis as the presence of a life-threatening organ dysfunction caused by infection, the definition we used is compatible with this broad concept. The variation in SOFA score, which is the current clinical criterion to define organ dysfunction, is not useful in quality improvement initiatives, because it would require calculations and also, in some cases, time to meet the criteria, which might result in delay in care. As recognized by the Sepsis 3 task force, this was not the intention of the new definition [30]. Thus, the SSC guidelines, on which this quality improvement initiative is based, did not change their criteria even after the launching of the definitions.

Although data from resource-limited settings are scarce, mortality reduction is not a consistent or sustained finding [15, 18, 31]. Our findings support this heterogeneity because only two institutions reduced their mortality rates despite the consistent global improvement in other quality indicators. In the successful institutions, there was a greater reduction in the time to sepsis diagnosis, reinforcing the results of our multivariate analysis. Unfortunately, our study did not address other factors that might influence the feasibility of quality improvement initiatives and help to explain this finding. Difficulties in implementing sepsis protocols have already been reported and include a lack of dedicated staff in the protocol; low availability of resources [32]; shortage of medical and nursing staff [21]; and low compliance with basic principles of quality care, such as continuous training strategies, which is associated with a high professional turnover rate [22], contributing to an inadequate safety culture and low quality of care [33]. These institutional characteristics were not addressed in our study. Also, we did not assess other institutional factors that might be associated with higher sepsis mortality rates, such as the availability of ICU beds or the percentage of patients transferred from other facilities.

Our study has some strengths. First, we reported data from a network of public institutions with a central administration, which is an original approach for a quality improvement initiative in an emerging country. Our results can be generalized to a significant proportion of the Brazilian public healthcare system and potentially to other emerging countries. Second, our implementation strategy was well planned with use of previously validated quality indicators, and we assessed patients in all hospital sectors instead of restricting inclusion to ICU patients. Third, the length of the intervention was sufficiently long to assess the persistence of the effects. However, our study also has some limitations. First, although the LASI provided all tools and information to allow for a similar intervention in all sites, local implementation was highly dependent on the local conditions. We did not measure the percentage of training staff in each site or the number of individually organized meetings. As in most quality improvement initiatives, this step is difficult to measure. Although the audit and feedback provided by the LASI was continuous throughout the intervention, differences in training capacities might have influenced the success of the institutions. The screening process could have been slightly different in some of the hospitals, such that patients might have been missed. Second, we did not monitor the quality of data collection with on-site verification of source documents, although we implemented central monitoring of data for completeness and consistency. Third, the before-after study design precludes any assessment of causality because there was no randomization process or control group. Fourth, we did not measure other concurrent quality improvement processes that might have influenced the results in the successful institutions. In addition, as previously described, we did not assess other information that could explain the differences between institutions, such as the availability of resources, staffing, ICU availability, and other general institutional quality indicators. Organizational factors in each of the institutions, such as nurse/patient ratio, physician and resident staffing hours, the presence of multidisciplinary rounds, the use of protocols in the ICU, or the use of checklists, are not described and might have influenced the results. Fifty, one of the successful sites had baseline mortality rates lower than the other institutions. However, the impact of having a lower baseline mortality rate in a quality improvement initiative is not clear, because this does not necessarily imply that there is a better chance of reducing mortality. We can argue that hospitals with lower mortality are better organized and thus are better settings for quality improvement initiatives. By contrast, this might also mean that the basic adjustments in quality improvement have already been done and that further improvement would be harder to achieve. Along these lines, settings with higher baseline mortality rates might succeed in adjusting basic processes, which will lower their mortality rates.

Conclusions

A multifaceted approach for sepsis treatment increased compliance and awareness in public institutions in Brazil. However, the impact on mortality was variable. Early recognition seems to be a more relevant factor than compliance with the 6-h bundle in terms of improving the chances of survival.