The Global Open Source Severity of Illness Score (GOSSIS)

Jesse D. Raffa, PhD; Alistair E. W. Johnson, DPhil; Zach O'Brien, MBBS; Tom J. Pollard, PhD; Roger G. Mark, MD, PhD; Leo A. Celi, MD, MPH, MSc; David Pilcher, FRACP, FCICM; Omar Badawi, PharmD, MPH

Disclosures

Crit Care Med. 2022;50(7):1040-1050. 

In This Article

Abstract and Introduction

Abstract

Objectives: To develop and demonstrate the feasibility of a Global Open Source Severity of Illness Score (GOSSIS)-1 for critical care patients, which generalizes across healthcare systems and countries.

Design: A merger of several critical care multicenter cohorts derived from registry and electronic health record data. Data were split into training (70%) and test (30%) sets, using each set exclusively for development and evaluation, respectively. Missing data were imputed when not available.

Setting/Patients: Two large multicenter datasets from Australia and New Zealand (Australian and New Zealand Intensive Care Society Adult Patient Database [ANZICS-APD]) and the United States (eICU Collaborative Research Database [eICU-CRD]) representing 249,229 and 131,051 patients, respectively. ANZICS-APD and eICU-CRD contributed data from 162 and 204 hospitals, respectively. The cohort included all ICU admissions discharged in 2014–2015, excluding patients less than 16 years old, admissions less than 6 hours, and those with a previous ICU stay.

Interventions: Not applicable.

Measurements and Main Results: GOSSIS-1 uses data collected during the ICU stay's first 24 hours, including extrema values for vital signs and laboratory results, admission diagnosis, the Glasgow Coma Scale, chronic comorbidities, and admission/demographic variables. The datasets showed significant variation in admission-related variables, case-mix, and average physiologic state. Despite this heterogeneity, test set discrimination of GOSSIS-1 was high (area under the receiver operator characteristic curve [AUROC], 0.918; 95% CI, 0.915–0.921) and calibration was excellent (standardized mortality ratio [SMR], 0.986; 95% CI, 0.966–1.005; Brier score, 0.050). Performance was held within ANZICS-APD (AUROC, 0.925; SMR, 0.982; Brier score, 0.047) and eICU-CRD (AUROC, 0.904; SMR, 0.992; Brier score, 0.055). Compared with GOSSIS-1, Acute Physiology and Chronic Health Evaluation (APACHE)-IIIj (ANZICS-APD) and APACHE-IVa (eICU-CRD), had worse discrimination with AUROCs of 0.904 and 0.869, and poorer calibration with SMRs of 0.594 and 0.770, and Brier scores of 0.059 and 0.063, respectively.

Conclusions: GOSSIS-1 is a modern, free, open-source inhospital mortality prediction algorithm for critical care patients, achieving excellent discrimination and calibration across three countries.

Introduction

Intensive care medicine is characterized by the management of patients at the highest risk of deterioration and death,[1] yet includes a heterogeneous population with substantial variation in expected outcomes.[2] To account for this heterogeneity in risk, severity of illness (SOI) scores are integral to quality of care evaluations, resource management, and the stratification of patients in research.[4] Several scoring systems exist but may be limited by poor generalizability beyond their derivation cohorts and are known to decrease in accuracy over time.[5–7] The need remains for a high-quality SOI score that is internationally valid and thereby facilitates intensive care benchmarking on a global scale. For this reason, a consortium of investigators was formed to develop a collection of open-source ICU SOI scores; the Global Open Source Severity of Illness Score (GOSSIS). This work demonstrates the feasibility of building multinational SOI scores by leveraging existing critical care databases.

The Australian and New Zealand Intensive Care Society Adult Patient Database (ANZICS-APD) is one of the largest intensive care datasets in the world, containing high-quality patient-level data from more than 90% of ICUs in Australia and New Zealand (ANZ).[8,9] Similarly, the eICU Collaborative Research Database (eICU-CRD) contains granular data on more than 200,000 admissions across 335 ICUs in the United States.[10] We sought to harmonize these datasets so that robust international SOI scores could be developed as a proof-of-concept, with greater accuracy and external validity than scoring systems currently available. Furthermore, we aimed to outline a methodology for the future generation of global predictive models, to be used as national intensive care registries become increasingly widespread.[11]

processing....