- Open Access
Judging quality of current septic shock definitions and criteria
Critical Care volume 19, Article number: 445 (2015)
Septic shock definitions are being revisited. We assess the feasibility, reliability, and validity characteristics of the current definitions and criteria of septic shock. Septic shock is conceptualised as cardiovascular dysfunction, tissue perfusion and cellular abnormalities caused by infection. Currently, for feasibility, septic shock is identified at the bedside by using either hypotension or a proxy for tissue perfusion/cellular abnormalities (e.g., hyperlactatemia). We propose that concurrent presence of cardiovascular dysfunction and perfusion/cellular abnormalities could improve validity of septic shock diagnosis, as we are more likely to identify a patient population with all elements of the illness concept. This epidemiological refinement should not affect clinical care and may aid study design to identify illness-specific biomarkers and interventions.
The illness concept, with its description (definition), clinical symptoms, signs and laboratory tests (criteria), and consequences (outcome), inform clinical reasoning and diagnosis. Diagnostic certainty is important for patients, clinicians and researchers as it helps to identify risk factors, select optimal treatment, devise new therapies, and prognosticate. In this context, septic shock is a clinical syndrome with protean clinical manifestations and biochemical abnormalities but with no universally accepted gold standard for diagnosis . In the absence of a gold standard test, the clinician-determined probability of the illness, the definition and the corresponding criteria are fundamental to diagnosing syndromes. After outlining the septic shock illness concept, this article evaluates how bedside operationalization of current definitions of septic shock definitions and their corresponding clinical criteria influence the validity, reliability, and feasibility attributes .
Infection triggers a series of host responses, dysregulation of which results in organ dysfunction (sepsis) [3–5]. Septic shock, the most severe form of sepsis, is highly complex, with concurrent myocardial, vascular (macro- and micro-), tissue perfusion and cellular level abnormalities. Myocardial abnormalities include both cardiomyocyte injury  and structural myocardial dysfunction [6, 7]. Vascular abnormalities affect the endothelium, sub-endothelial tissues, microcirculation and vasomotor tone. These alter systemic vascular resistance, increase endothelial permeability, impair tissue oxygen delivery (tissue hypoxia) and utilization (cellular dysoxia), and modify cellular metabolism (Fig. 1) [8–15].
Current definitions and criteria for septic shock
The 1992 and 2001 Sepsis Consensus Conference definitions [16, 17] and the definition used by the Surviving Sepsis Campaign (SSC) guidelines  currently provide the most-cited case definitions and criteria for making a diagnosis of septic shock. Yet all differ for both definition and criteria (e.g., use of different cutoffs for hypotension) and variable use of markers of impaired tissue perfusion (e.g., raised lactate, base deficit). There is also an implicit reliance upon clinician-driven interventions such as fluid therapy or inotropes. Thus, there are potentially varying combinations of clinical criteria and differences in the interventional targets used to diagnose septic shock. This heterogeneity highlights the limitations of current operationalization methods [19–21].
Judging quality attributes of septic shock definitions and criteria
Validity is the ability to capture what the investigator truly seeks to measure. Validity assessments can be either qualitative (face validity, content validity) or quantitative (criterion validity, construct validity).
“Face validity” refers to whether patients identified by the criteria appear to match the illness concept at face value. At present, hypotension is a core element of the criteria identifying septic shock. However, the presence of hypotension in the context of infection does not necessarily define septic shock. Hypotension in an infected patient could be related to pre-illness medications (e.g., antihypertensive), comorbidities (e.g., heart failure) or concurrent interventions (e.g., sedatives) or a combination of these. In other words, we remain uncertain whether hypotension as the only criterion truly represents septic shock, especially when used without caveats to address potential confounders (e.g., medications or heart failure). Furthermore, we cannot quantify this uncertainty, as there is no gold standard test for septic shock.
Content validity assesses whether the clinical criteria encompass all components of the illness. For septic shock, these components are cardiovascular dysfunction, hypoperfusion and cellular abnormalities. We highlighted above that hypotension alone is an incomplete proxy for cardiovascular dysfunction to define septic shock. Equally, imperfect proxies of tissue hypoperfusion such as base deficit, confusion or oliguria are inadequate in isolation to define septic shock . Similarly, we currently lack a reliable proxy to measure the complex cellular abnormalities that occur in septic shock. Hyperlactatemia is often used as a proxy; however, this is usually found in all types of shock  and the serum level is variably related to multiple confounders, including the rate of tissue utilization (lactate clearance), accelerated β-adrenergic-driven aerobic glycolysis, liver dysfunction and co-existing anaerobic glycolysis [11, 22–25].
Given the lack of a gold standard diagnostic test for septic shock, the performance characteristics of these variables (sensitivity, specificity, and predictive values) could be based on prognosis (e.g., short-term mortality), with or without a clinical reclassification risk assessment or blinded clinician adjudication . If we alter the existing clinical criteria, the reclassification risk refers to the proportion of patients reclassified into new risk-of-death categories . Levy et al., using the SSC dataset, gave a simple descriptive demonstration of this concept by using lactate as an additional marker for septic shock in a cohort of patients with an overall mortality of 38.4 % . The authors reported categories of shock as lactate of more than 4 mmol/l only (29.9 % mortality), vasopressor use only (36.7 % mortality), or a combination of lactate of more than 4 mmol/l plus vasopressor use (46.1 % mortality). A similar exercise conducted on data collected from English intensive care units (ICUs) reported mortality rates of 26.2 % for patients with an isolated lactate level of more than 4 mmol/l, 31.4 % for refractory hypotension only, and 55.5 % for the combination . Kaukonen et al. recently evaluated the performance characteristics of the four systemic inflammatory response syndrome (SIRS) criteria in identifying sepsis . They highlighted that 12 % of patients admitted to ICUs with organ dysfunction consequent to presumed infection were SIRS-negative (i.e., had fewer than two criteria of SIRS). Thus, if sepsis criteria are developed without requiring SIRS variables, different risk categories within patient populations may be identified (i.e., reclassification).
While cardiovascular organ dysfunction has long been central to the definition of septic shock, the criteria used to identify it remain variable. Levy et al.  specified the use of either Sepsis Organ Failure Assessment (SOFA) or multiple organ dysfunction score (MODS) to quantify organ dysfunction in patients with severe sepsis [30, 31]. However, these two scoring systems differ, particularly in terms of cardiovascular dysfunction criteria. The SOFA score uses both mean arterial pressure and the dose of vasoactive drugs being administered. By contrast, the MODS is solely physiology-based, using a pressure-adjusted heart rate parameter derived from heart rate, central venous pressure and mean arterial pressure to reflect fluid-unresponsive hypotension. Under the MODS scoring system, organ dysfunction is a continuum with no category differentiating dysfunction from failure [30–32]. Cardiovascular dysfunction is quantified by a continuous variable derived by using regression analyses to define the variables and their weights. In contrast, SOFA, which was developed by using expert opinion, views cardiovascular dysfunction as occurring in discrete steps, with a score of 1 or 2 being coded as organ “dysfunction” and 3 or 4 coded as organ “failure”. Thus, the cutoffs and relative weights of variables within these scores are unlikely to stratify cardiovascular dysfunction similarly .
Criterion and construct validity
Criterion validity encompasses both concurrent and predictive validities. Concurrent validity refers to the ability of the definition and criteria to discriminate groups, whereas predictive validity is the ability to predict (future) outcomes. A closely related concept is construct validity; this refers to an assessment of how well the definitions are converted into measurable criteria to identify septic shock in clinical practice.
There are many examples in the literature of poor criterion validity for current definitions and criteria of septic shock. For example, a cohort study of nearly 8000 patients defined by the 1992 criteria as having septic shock  reported an overall crude hospital mortality of 52.4 % . However, mortality ranged from 21.1 to 84.5 % when stratified on anatomical site of infection, and this variation persisted despite adjustments for confounders such as age, comorbidities and organism type. Given that cardiovascular dysfunction is the core criterion used for diagnosis, the predictive validity of current definitions of septic shock is weak, being heavily dependent on how it is operationalized at the bedside .
Outcome is also affected by how the individual components of septic shock (cardiovascular dysfunction, hypoperfusion and cellular abnormalities) are determined. Mortality in a single dataset varied from 45 to 60 % depending on what diagnostic criteria were applied . Similarly, hospital mortality rates in septic shock patients admitted to ICUs in Australia and New Zealand (171 ICUs; n = 6757) (ANZICS data, R. Bellomo, personal communication) and Italy (221 ICUs; n = 4959) (GiViTI data, Italian ICU registry; Margherita project, G. Bertolini, personal communication) during 2012 were reported as 22 and 57.9 %, respectively. The Australasian case definition was predicated upon an APACHE III (Acute Physiology and Chronic Health Evaluation III) diagnosis coding of infection plus recording of either a decrease in mean blood pressure of less than 65 mmHg or systolic blood pressure of less than 90 mmHg at any time within the first 24 h of admission. In contrast, the Italian criteria used those provided by the 2001 Consensus definitions . Even when data were extracted from representative national datasets by using the International Classification of Diseases coding system, the hospital mortality using the septic shock code was much higher in Germany (60.5 % in 2011)  than in the USA (42.1 % between 2004 and 2009) . Are outcomes in Germany much worse, or do the differences merely reflect coding practices or variable patient pathways (e.g., transfer to post-acute care hospitals) ?
Secular trends in septic shock outcomes are depicted in Fig. 2. Among other explanations , this improving trend in mortality may also represent an enhanced detection of a less severe “septic shock” population using hypotension alone as a criterion (i.e., stage migration, or the Will Rogers phenomenon ).
Reliability refers to agreement between observers and by the same observer during repeated measurements (i.e., consistency and reproducibility). With septic shock, some measurements, such as blood pressure and lactate levels, are likely to have reasonably high inter-observer, intra-observer and intra-subject reliability subject to certain caveats and exclusion of methodological errors. On the other hand, adequacy of fluid resuscitation and initiation of vasopressor therapy are far more subjective because of a lack of consensus on triggers and end-points. Potentially, we can test the reliability attributes of these variables (e.g., consistency within and between observers) by using test-retest reliability and the related kappa statistic.
Feasibility is a composite concept that depends on the purpose of the diagnosis; it is a compromise between validity and reliability . For a high-mortality condition such as septic shock, ease of detection is key. The SSC guidelines have increased feasibility by emphasizing cardiovascular dysfunction criteria for the diagnosis of septic shock . However, relaxing diagnostic criteria will almost certainly increase the rate of false-positive diagnoses. Conversely, if the complexity and number of criteria to be met for diagnosis are increased to improve validity, then feasibility will almost certainly be reduced. For example, the incidence of septic shock was halved from 9.1 to 4.4 % when liberal criteria (i.e., refractory hypotension) were replaced with restrictive criteria (i.e., refractory hypotension with non-cardiovascular dysfunction) .
How could we address these challenges?
The above discussion highlights variability in the criteria used to identify septic shock in published research. The European Society of Intensive Care Medicine-Society of Critical Care Medicine (ESICM-SCCM) Task Force on new definitions of sepsis is currently undertaking a systematic review of the literature to explore how these septic shock definitions and criteria are operationalized. Observational studies reporting the incidence and outcomes of septic shock are ideal target publications as randomized controlled trials often have trial-specific criteria. The second step in this process is to generate agreement on the updated illness concept which should reflect all three domains of biology (i.e., cardiovascular dysfunction, cellular abnormalities and evidence of impaired tissue perfusion). Based on this illness concept and by applying the principle of parsimony, the minimum number of bedside variables that have face and content validity could be determined. Inter-observer reliability of septic shock case detection could be improved by using a single validated criterion for cardiovascular dysfunction (e.g., need for vasopressor therapy to maintain target blood pressure) and a proxy for likely cellular and impaired tissue perfusion abnormality (e.g., lactate). These steps could be achieved with qualitative research methods, such as a Delphi process . An important research question in this context is whether septic shock illness criteria should have predictive validity . If we believe that septic shock is the most severe form of sepsis, then an argument could be made for having predictive validity as a characteristic of any updated septic shock criteria. This research question could be addressed by developing a parsimonious model using combinations and cutoffs of the variables for outcomes such as short-term mortality.
Septic shock conceptually comprises an illness with new onset or worsening cardiovascular dysfunction, impaired tissue perfusion and cellular abnormalities caused by infection. Individually, none of these abnormalities truly reflects the complex illness concept but they may do so in combination, potentially improving the validity and reliability of a diagnosis of septic shock. As there are no specific treatments for this condition, this epidemiological refinement will not affect clinical care and may aid the design of studies that can identify illness-specific biomarkers and interventions.
Intensive care unit
Multiple organ dysfunction score
Systemic inflammatory response syndrome
Sepsis Organ Failure Assessment
Surviving Sepsis Campaign
Shankar-Hari M, Deutschman CS, Singer M. Do we need a new definition of sepsis? Intensive Care Med. 2015;41:909–11.
Rubenfeld GD. Epidemiology of acute lung injury. Crit Care Med. 2003;31(4 Suppl):S276–84.
Xiao W, Mindrinos MN, Seok J, Cuschieri J, Cuenca AG, Gao H, et al. A genomic storm in critically injured humans. J Exp Med. 2011;208:2581–90.
Takasu O, Gaut JP, Watanabe E, To K, Fagley RE, Sato B, et al. Mechanisms of cardiac and renal dysfunction in patients dying of sepsis. Am J Respir Crit Care Med. 2013;187:509–17.
Marshall JC. Sepsis: rethinking the approach to clinical research. J Leukoc Biol. 2008;83:471–82.
Kimmoun A, Ducrocq N, Mory S, Delfosse R, Muller L, Perez P, et al. Cardiac contractile reserve parameters are related to prognosis in septic shock. Biomed Res Int. 2013;2013:930673.
Zanotti-Cavazzoni SL, Hollenberg SM. Cardiac dysfunction in severe sepsis and septic shock. Curr Opin Crit Care. 2009;15:392–7.
Vincent JL, De Backer D. Circulatory shock. N Engl J Med. 2013;369:1726–34.
Abraham E, Shoemaker WC, Bland RD, Cobo JC. Sequential cardiorespiratory patterns in septic shock. Crit Care Med. 1983;11:799–803.
Vincent JL, Gris P, Coffernils M, Leon M, Pinsky M, Reuse C, et al. Myocardial depression characterizes the fatal course of septic shock. Surgery. 1992;111:660–7.
Hotchkiss RS, Karl IE. Reevaluation of the role of cellular hypoxia and bioenergetic failure in sepsis. JAMA. 1992;267:1503–10.
Kreymann G, Grosser S, Buggisch P, Gottschall C, Matthaei S, Greten H. Oxygen consumption and resting metabolic rate in sepsis, sepsis syndrome, and septic shock. Crit Care Med. 1993;21:1012–9.
Carre JE, Orban JC, Re L, Felsmann K, Iffert W, Bauer M, et al. Survival in critical illness is associated with early activation of mitochondrial biogenesis. Am J Respir Crit Care Med. 2010;182:745–51.
Langley RJ, Tsalik EL, van Velkinburgh JC, Glickman SW, Rice BJ, Wang C, et al. An integrated clinico-metabolomic model improves prediction of death in sepsis. Sci Transl Med. 2013;5:195ra95.
De Backer D, Orbegozo Cortes D, Donadello K, Vincent JL. Pathophysiology of microcirculatory dysfunction and the pathogenesis of septic shock. Virulence. 2014;5:73–9.
Bone RC, Balk RA, Cerra FB, Dellinger RP, Fein AM, Knaus WA, et al. Definitions for sepsis and organ failure and guidelines for the use of innovative therapies in sepsis. The ACCP/SCCM Consensus Conference Committee. American College of Chest Physicians/Society of Critical Care Medicine. Chest. 1992;101:1644–55.
Levy MM, Fink MP, Marshall JC, Abraham E, Angus D, Cook D, et al. SCCM/ESICM/ACCP/ATS/SIS International Sepsis Definitions Conference. Crit Care Med. 2001;2003(31):1250–6.
Dellinger RP, Levy MM, Rhodes A, Annane D, Gerlach H, Opal SM, et al. Surviving sepsis campaign: international guidelines for management of severe sepsis and septic shock: 2012. Crit Care Med. 2013;41:580–637.
Iwashyna TJ, Govindan S. Did they just prove that a diagnosis of “septic shock” is meaningless? Am J Respir Crit Care Med. 2014;189:1156–7.
Leligdowicz A, Dodek PM, Norena M, Wong H, Kumar A, Kumar A, et al. Association between source of infection and hospital mortality in patients who have septic shock. Am J Respir Crit Care Med. 2014;189:1204–13.
Gaieski DF, Edwards JM, Kallan MJ, Carr BG. Benchmarking the incidence and mortality of severe sepsis in the United States. Crit Care Med. 2013;41:1167–74.
Levy B, Gibot S, Franck P, Cravoisy A, Bollaert PE. Relation between muscle Na + K+ ATPase activity and raised lactate concentrations in septic shock: a prospective study. Lancet. 2005;365:871–5.
Vander Heiden MG, Cantley LC, Thompson CB. Understanding the Warburg effect: the metabolic requirements of cell proliferation. Science. 2009;324:1029–33.
Tapia P, Soto D, Bruhn A, Alegria L, Jarufe N, Luengo C, et al. Impairment of exogenous lactate clearance in experimental hyperdynamic septic shock is not related to total liver hypoperfusion. Crit Care. 2015;19:188.
Gore DC, Jahoor F, Hibbert JM, DeMaria EJ. Lactic acidosis during sepsis is related to increased pyruvate production, not deficits in tissue oxygen availability. Ann Surg. 1996;224:97–102.
Cook NR. Statistical evaluation of prognostic versus diagnostic models: beyond the ROC curve. Clin Chem. 2008;54:17–23.
Levy MM, Dellinger RP, Townsend SR, Linde-Zwirble WT, Marshall JC, Bion J, et al. The Surviving Sepsis Campaign: results of an international guideline-based performance improvement program targeting severe sepsis. Intensive Care Med. 2010;36:222–31.
Mouncey PR, Osborn TM, Power GS, Harrison DA, Sadique MZ, Grieve RD, et al. Trial of early, goal-directed resuscitation for septic shock. N Engl J Med. 2015;372:1301–11.
Kaukonen KM, Bailey M, Pilcher D, Cooper DJ, Bellomo R. Systemic inflammatory response syndrome criteria in defining severe sepsis. N Engl J Med. 2015;372:1629–38.
Marshall JC, Cook DJ, Christou NV, Bernard GR, Sprung CL, Sibbald WJ. Multiple organ dysfunction score: a reliable descriptor of a complex clinical outcome. Crit Care Med. 1995;23:1638–52.
Vincent JL, Moreno R, Takala J, Willatts S, De Mendonca A, Bruining H, et al. The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure. On behalf of the Working Group on Sepsis-Related Problems of the European Society of Intensive Care Medicine. Intensive Care Med. 1996;22:707–10.
Cook R, Cook D, Tilley J, Lee K, Marshall J, Canadian Critical Care Trials Group. Multiple organ dysfunction: baseline and serial component scores. Crit Care Med. 2001;29:2046–50.
Vincent JL, Moreno R. Clinical review: scoring systems in the critically ill. Crit Care. 2010;14:207.
Klein Klouwenberg PM, Ong DS, Bonten MJ, Cremer OL. Classification of sepsis, severe sepsis and septic shock: the impact of minor variations in data capture and definition of SIRS criteria. Intensive Care Med. 2012;38:811–9.
Heublein S, Hartmann M, Hagel S, Hutagalung R, Brunkhorst FM. Epidemiology of sepsis in German hospitals derived from administrative databases. 6th International Congress on Sepsis and Multiorgan Dysfunction, September 4-6, Weimar. Infection. 2013;41:7–23.
Damuth E, Mitchell JA, Bartock JL, Roberts BW, Trzeciak S. Long-term survival of critically ill patients treated with prolonged mechanical ventilation: a systematic review and meta-analysis. Lancet Respir Med. 2015;3:544–53.
Rhee C, Gohil S, Klompas M. Regulatory mandates for sepsis care—reasons for caution. N Engl J Med. 2014;370:1673–6.
Feinstein AR, Sosin DM, Wells CK. The Will Rogers phenomenon. Stage migration and new diagnostic techniques as a source of misleading statistics for survival in cancer. N Engl J Med. 1985;312:1604–8.
Okoli C, Pawlowski SD. The Delphi method as a research tool: an example, design considerations and applications. Inform Manage. 2004;42:15–29.
MS-H and MS acknowledge the support of the UK National Institute for Health Research (NIHR) Biomedical Research Centre scheme. MS is a recipient of a UK NIHR Senior Investigator Fellowship.
The authors declare that they have no competing interests.
MS-H conceived and drafted the first version of the manuscript. MS revised the first draft. GB, DA, RB, and FMB provided additional data. All authors contributed to all subsequent critical revisions of the manuscript and have read and approved the final manuscript.
Authors are co-chairs (MS and CSD), members (DA and RB) and a co-opted member (MS-H) of the Sepsis Redefinitions Task Force supported by the European Society of Intensive Care Medicine and the Society for Critical Care Medicine.
About this article
Cite this article
Shankar-Hari, M., Bertolini, G., Brunkhorst, F.M. et al. Judging quality of current septic shock definitions and criteria. Crit Care 19, 445 (2015). https://doi.org/10.1186/s13054-015-1164-6