Incidence and outcome of inappropriate in-hospital empiric antibiotics for severe infection: a systematic review and meta-analysis

Introduction The aims of this study were to explore the incidence of in-hospital inappropriate empiric antibiotic use in patients with severe infection and to identify its relationship with patient outcomes. Methods Medline (from 2004 to 2014) was systematically searched by using predefined inclusion criteria. Reference lists of retrieved articles were screened for additional relevant studies. The systematic review included original articles reporting a quantitative measure of the association between the use of (in)appropriate empiric antibiotics in patients with severe in-hospital infections and their outcomes. A meta-analysis, using a random-effects model, was conducted to quantify the effect on mortality by using risk ratios. Results In total, 27 individual articles fulfilled the inclusion criteria. The percentage of inappropriate empiric antibiotic use ranged from 14.1% to 78.9% (Q1-Q3: 28.1% to 57.8%); 13 of 27 studies (48.1%) described an incidence of 50% or more. A meta-analysis for 30-day mortality and in-hospital mortality showed risk ratios of 0.71 (95% confidence interval 0.62 to 0.82) and 0.67 (95% confidence interval 0.56 to 0.80), respectively. Studies with outcome parameter 28-day and 60-day mortality reported significantly (P ≤0.02) higher mortality rates in patients receiving inappropriate antibiotics. Two studies assessed the total costs, which were significantly higher in both studies (P ≤0.01). Conclusions This systematic review with meta-analysis provides evidence that inappropriate use of empiric antibiotics increases 30-day and in-hospital mortality in patients with a severe infection. Electronic supplementary material The online version of this article (doi:10.1186/s13054-015-0795-y) contains supplementary material, which is available to authorized users.


Introduction
Infections are among the top three leading causes of death worldwide [1]. Septicaemia and pneumonia combined are the sixth most common causes of death in the United States [2]. Bloodstream infections (BSIs) are associated with substantial morbidity, mortality, and health-care costs [3]. Sepsis is one of the leading causes of death in the critically ill, with a mortality rate of 28% to 55% [4]. Antibiotics are the mainstay of treatment for these serious infections [5]. Antibiotic treatment for moderate to severe infections has to start early and, in the absence of evidence on the causative pathogen or its sensitivity to antibiotics, is often guided by empirical evidence [6].
Given the high incidence of infections and the not well-established relationship between empiric (I)AAT and clinical outcome [19][20][21][22], it is necessary to synthese the best available evidence. This systematic review with meta-analysis was conducted to synthesize the best available evidence regarding (1) the definition, (2) the incidence, and (3) the outcome of empiric IAAT.

Data sources and search strategy
Quantitative studies on the association between the use of empiric (I)AAT in patients with a severe infection and their outcome in public or private general hospital settings were searched in Medline. Studies published in the last 10 years (20 August 2004 to 20 August 2014) were selected as critical illness management changes continuously and earlier and earlier studies may be less relevant for current practice. The following Medical Subject Headings (MeSH) search terms and free-text terms were used either individually or in combination: 'antibiotic', 'infection', 'appropriate', 'inappropriate', 'adequate', 'inadequate', 'outcome', 'mortality', 'survival rate', 'cost', and 'length of stay'. Only studies published in English, Dutch, German, or French were included. Reference lists of retrieved articles were hand-searched for additional relevant studies. A detailed description of the search strategy is included in the Additional file 1: digital content.

Eligibility criteria Study design
Potentially included study designs included randomized controlled trials (RCTs), non-randomized controlled trials, controlled before-after studies, interrupted time series, and repeated measures studies. Only studies reporting a quantitative evaluation regarding the association between the use of AAT or IAAT in patients with a severe infection and their outcomes within the hospital setting were included. The studies use (I)AAT as the independent variable and outcome-measured as mortality, hospital length of stay (LOS), and costs-as the dependent variable. Studies that recruited less than 75 patients were excluded because the research team assumes that these studies bear the risk to be underpowered.

Patients
The included patients were adults (at least 18 years old) with a severe infection. For this review, pneumonia, BSI or bacteraemia, sepsis, severe sepsis, or septic shock were considered severe infections. Studies specifically focused on meningitis, endocarditis or infections in burn and transplant patients were excluded as the literature showed that treatment effects are expected to largely deviate from any common effect.

Intervention
The intervention of interest concerned empiric AAT versus IAAT. Empiric antibiotic therapy is defined as all nondefinitive therapy and refers to antibiotics given prior to the result of the final culture and the antibiotic sensitivity tests [23]. Studies that did not specify the used definition of AAT or IAAT were excluded. Studies comparing two or more types of antibiotics were excluded.

Outcome
Outcomes were assessed in terms of mortality, hospital LOS, and costs.

Study appraisal
Two reviewers (KM and AL) independently performed the initial scan of titles and abstracts of all retrieved citations by using standardized screening forms. Both reviewers documented the reasons for exclusion. Full-text copies of all potentially relevant studies were obtained and further checked for inclusion. Any discrepancies between reviewers were resolved by discussion. Continuing disagreements were settled by a third reviewer (NC or AV). Additional sources that had been cross-referenced from the Medline search results were included if they met the criteria above. The quality of the articles was evaluated by using the Downs and Black quality assessment method, which is a list of 27 criteria to evaluate both randomized and non-randomized trials [24]. This scale assesses study reporting, external validity, internal validity, and power of non-randomized studies and has been ranked in the top six quality assessment scales suitable for use in systematic reviews [25,26]. As had been done in other reviews using the Downs and Black scale [27][28][29], the tool was modified slightly for use in this particular review. Specifically, the scoring for question 27 dealing with statistical power was simplified to a choice of awarding either 1 point or 0 points, depending on whether there was sufficient power to detect a clinically important effect. The criterion was that to detect a 10% difference, assuming power of 0.90 and alpha of 0.05. The Downs and Black scores were grouped into the following 4 quality levels: excellent (26 to 28), good (20 to 25), fair (15 to 19) and poor (less than 14) [29]. Only articles with a quality level of good or excellent were retained.

Data extraction
Data extraction was completed independently by two reviewers (KM and AL), who used a standardized data collection form. The following data were extracted and reported: (1) data on study setting and patient population as possible confounding factors, (2) definition and incidence of the (I)AAT, and (3) definition and measurement of outcome variables (in terms of mortality, hospital LOS, and costs among patients given AAT versus IAAT). In case of disagreement between the two reviewers, a third reviewer (NC or AV) extracted the data.

Study characteristics
For every included study, descriptive data on the study setting (that is, study design, geographic location of the study, baseline characteristics, study years, and sample size) and patient characteristics (that is, source of infection and severity scale) were collected.

Definition and measuring incidence of (I)AAT
We reviewed how empiric (I)AAT was defined and measured. We assessed which evidence-based elements, such as therapy dose, route, and timing, were evaluated. Empiric antibiotic therapy is defined as all non-definitive therapy and refers to antibiotics given prior to the result of the final culture and the antibiotic sensitivity tests [23].

Measurement of the dependent variable
The outcome was measured as mortality, LOS, and costs for patients given empirical (I)AAT. The time span of mortality assessment was also registered.

Data analysis
Data were analyzed by using R (a language and environment for statistical computing) [30]. All reported P values were two-sided; P <0.05 was considered to indicate statistical significance. A random-effects meta-analysis using the DerSimonian-Laird estimator obtained risk ratios (RRs) and 95% confidence intervals (CIs) for mortality rate reductions [31]. Heterogeneity of the study results was assessed by using the Cochran Q test and the Higgins I 2 test. The following thresholds were used to quantify heterogeneity: P <0.10 in Cochran's Q test and I 2 ≤ 25% for low, 25% < I 2 < 50% for moderate, and I 2 ≥ 50% for high. Funnel plots assessed publication bias. Sensitivity analysis identified heterogeneous studies that influenced the metaanalysis. Meta-regression was used to examine the impact of study characteristics on study effect size and heterogeneity.

Data on definition and measurement of (I)AAT
Data on the definition and the incidence of (I)AAT were presented in Table 2 The percentage of empiric IAAT showed an enormous range from 14.1% to 78.9% (median of 49, 26%, interquartile range 28.1% to 57.8%). The magnitude of this range can be explained in part by the differences in the definitions, settings, diseases, and infectious agents. Because of this considerable heterogeneity, it may be misleading to quote an average value for the incidence. However, 13 (48.1%) of these 27 studies described an incidence of IAAT of 50% or more.

Measurement of the dependent variable
Outcome was measured as mortality, LOS, and costs. A meta-analysis was conducted to quantify the effect of appropriateness in empiric antibiotics on mortality. The number of studies that assess the total LOS [48,55], LOS after infection onset [36,45], and the costs [37,48] were very small. Therefore, these results are presented in a descriptive manner only.

LOS and costs
Four studies reported the effect on LOS: total LOS [48,55] or LOS after the onset of infection [36,45]. In one of the two studies [45], the mean LOS after infection onset was significantly (P = 0.022) higher in the group sepsis patients with IAAT. This indicates that IAAT independently increased the median attributable LOS by 2 days. However, the study by Marschall et al. [36] found no significant differences in LOS post-onset (P = 0.09) in patients with Gram-negative bacteraemia. Appropriately treated patients with ventilator-associated pneumonia had a significantly shorter total LOS (P = 0.022) [48]. Nevertheless, Chen et al. [55] found no differences in the total LOS of patients with community-onset bloodstream infections.  The costs were assessed in only two studies [37,48]. The total costs for patients with IAAT were significantly higher in both studies (P ≤0.01).

Discussion
The incidence of patients with severe infections is substantial. Previous studies confirmed-as proven by the low number needed to treat-that correct antibiotic treatment is a crucial determinant of therapeutic success [66]. Therefore, a systematic review with meta-analysis was conducted to investigate the incidence and consequences of IAAT on the outcome in hospitalized patients with infection. Definitions and criteria items used to denote (I)AAT varied substantially between studies. However, most definitions included the criterion 'matching with the in vitro susceptibility' or 'intermediate or full in vitro resistance'. The timing of administration of the antibiotics was taken into account in only 71% of the definitions. Timing of admission is, however, an important aspect of adequate antibiotic therapy. In patients with septic shock, each hour of delay in antimicrobial therapy is associated with an average decrease in survival of 7.6% [13]. Rivers et al. [67] showed that early goal-directed therapy provides significant benefits with respect to outcome in patients with severe sepsis and septic shock. For patients with Staphylococcus aureus bacteraemia, the breakpoint between delayed and early treatment was 44.75 hours, and delayed treatment was found to be an independent predictor of infection-related mortality [68]. Based on this heterogeneity in the definitions, it was impossible to estimate the overall incidence of IAAT. However, IAAT ranged from 14.1% to 78.9%, and 46.4% of studies described an incidence of IAAT of 50% or more. Given this high incidence, health-care professionals must become aware of this problem. Moreover, in an era of rising antimicrobial resistance rates, choosing empiric AAT is an increasing challenge. The meta-analysis, involving 13,014 patients, suggests that the empiric AAT reduces 30-day mortality (RR 0.71, 95% CI 0.62 to 0.82) and in-hospital mortality (RR 0.67, 95% CI 0.56 to 0.80). In addition, empiric AAT positively affects LOS and costs.
Strengths of this study include the comprehensive search strategy, the methodological quality assessment, and the random-effects model analysis combined with meta-regression. Besides the methodological strengths, the study has limitations. First, the present findings should be interpreted in the context of the included   studies and their limitations: the heterogeneity in patients' characteristics, definitions of IAAT, and the time span of outcome assessment. Second, the lack of RCTs is this review could be seen as a major limitation. The lack of RCTs regarding this topic stems from obvious ethical constraints. Given the methodological heterogeneity of the included (retro-and prospective) observational studies, an overall meta-analysis was impossible. Meta-analysis was performed for 30-day and in-hospital mortality only. Third, several potential biasing and confounding elements might have hampered this metaanalysis. The reported diseases and the diagnosis process, the study quality quantified by the Downs and Black instrument, the quality of the health-care systems in the different countries, and the definitions of adequate antibiotic therapy had a marked influence on the metaanalysis of in-hospital mortality. Nevertheless, we aggregated all reported diseases to avoid a small numbers problem. Probably the cleanest data for assessing the impact of (I)AAT would be for bacteremia, as this is the infection that can most accurately be defined. Fourth, this analysis does not cover all areas, such as fungemiae. However, this limitation creates opportunities for further research. Fifthly, we used the (criteria of the) definitions used in the included studies. Most of the studies approached the definition one-sided and used only the criteria 'matching with the culture' and 'according to the guidelines'. However, appropriateness of antibiotic treatment is related not only to the substance itself but also to dosing or administration route (or both) of the antibiotic. Finally, during this review, we focused on (in)appropriate antibiotic therapy. Off course, inappropriate therapy is not only determined by the antibiotic used. Further research could focus on other aspects of (in)appropriate therapy.

Conclusions
This systematic review demonstrates a very high incidence of IAAT in patients with severe bacterial infection, such as BSI, pneumonia, sepsis, or septic shock. Accurate empirical treatment of these severe infections is not a simple process seen in currently reported rates of IAAT. Meta-analysis provides evidence that empiric inappropriate use of empiric antibiotics increases 30-day and in-hospital mortality in these patients. Clinicians should be aware of this problem, and further improvement actions should be taken. Inappropriate antibiotic treatment stems from several causes, mainly due to resistance; therefore, it is not easy to find the most appropriate treatment option. As long as general recommendations about antibiotic stewardship are missing, problems will remain. Computerized decision support, including complex and locally calibrated decision algorithms [69,70] or early molecular identification or both, might be helpful.