A Comparative Analysis of Sepsis Identification Methods in an Electronic Database

Alistair E. W. Johnson, DPhil; Jerome Aboab, MD, PhD; Jesse D. Raffa, PhD; Tom J. Pollard, PhD; Rodrigo O. Deliberato, MD, PhD; Leo A. Celi, MD, MPH; David J. Stone, MD


Crit Care Med. 2018;46(4):494-499. 

In This Article


Demographics of the population studied are provided (Table 1). Of the 11,791 patients, 75.2% (9,323) had a SOFA score greater than or equal to 2 during their first ICU day. Median age was 64.5 years (Q1–Q3, 51.1–78.5 yr), and mean body mass index (BMI) was 28.7 kg/m2 (SD, 8.4 kg/m2). Median ICU LOS was 1.9 days (Q1–Q3: 1.1–3.5 d), and hospital mortality was 10.8%. All demographic items had significant differences between survivors and nonsurvivors except gender and BMI.

Hospital mortality was higher for patients with SOFA greater than or equal to 2 (13.2%) than those with SOFA less than 2 (3.6%); the secondary outcome, a composite of in-hospital mortality and/or ICU LOS greater than or equal to 3 days, occurred in two fifths (41.2%) of patients. Patients suspected of infection had higher hospital mortality (12.5% vs 8.3%). Examining only these patients, SOFA had an AUROC of 0.74 (95% CI, 0.72–0.76; primary outcome) and 0.69 (95% CI, 0.68–0.70; secondary outcome).

The Sepsis-3 criteria identified the largest cohort of patients (49.1%, 5,784 cases), followed by CDC (31.9%, 3,761), Angus (28.6%, 3,368), Martin (14.7%, 1,734), CMS (11.0%, 1,302), and explicit (9.0%, 1,062). The in-hospital mortality rate was highest in explicit (31.4%), followed by CMS (27.2%), Martin (23.4%), CDC (18.6%), Angus (17.7%), and Sepsis-3 (14.7%). Figure 1 shows these trends graphically. The rankings for the composite outcome of mortality and long ICU LOS were similar, with the exception of the CDC and Martin criteria being transposed (Table 2).

Figure 1.

Percentage of patients detected by the criteria are shown in the wider light gray bar, and the mortality rate of these patients is shown in the thinner dark gray bar. Actual values are available in Supplemental Table 1 (Supplemental Digital Content 1, http://links.lww.com/CCM/D208). CDC = Centers for Disease Control and Prevention, CMS = Centers for Medicare & Medicaid Services, SOFA = Sequential Organ Failure Assessment.

Figure 2 shows three-set Venn diagrams for the criteria assessed. Figure 2A compares Sepsis-3, Angus, and Martin. All three criteria were satisfied by 1,420 patients (12%), whereas 5,488 patients (46.2%) did not satisfy any. Very few patients were identified by Martin alone (0.5%, 58), and similarly few for Angus (3.4%, 399). Figure 2B compares Sepsis-3, CDC, and CMS: 51.5% satisfied at least one criterion; 9.0% satisfied all three criteria. The majority of patients who satisfied CDC also satisfied Sepsis-3 (3,521 patients, 93.6%), and similarly those who satisfied CMS usually also satisfied Sepsis-3 (1,228 patients, 94.3). The explicit criteria, not shown in Figure 2, were entirely subsumed by Angus criteria and almost entirely subsumed by Martin criteria (Supplemental Figure 2, Supplemental Digital Content 1, http://links.lww.com/CCM/D208). Cronbach's alpha for Sepsis-3 varied from acceptable (0.40–0.49 vs explicit, CMS, Martin) to good (0.62 vs Angus and 0.76 vs CDC) (Supplemental Table 2, Supplemental Digital Content 1, http://links.lww.com/CCM/D208).

Figure 2.

Venn diagrams comparing overlap in populations identified by criteria presented. A, Martin et al (8), Angus et al (9), and Sepsis-3 criteria: 6,343 patients (53.8%) were captured by at least one criterion, whereas 1,420 patients (12.0%) satisfied all criteria. B, Centers for Medicare & Medicaid Services (CMS), Centers for Disease Control and Prevention (CDC), and Sepsis-3 criteria: 6,076 patients (51.5%) satisfied at least one of the criteria, whereas 1,062 patients (9.0%) satisfied all criteria.