Development and Validation of a Clinical Risk Model to Predict the Hospital Mortality in Ventilated Patients With Acute Respiratory Distress Syndrome

A Population-Based Study

Weiyan Ye; Rujian Li; Hanwen Liang; Yongbo Huang; Yonghao Xu; Yuchong Li; Limin Ou; Pu Mao; Xiaoqing Liu; Yimin Li

Disclosures

BMC Pulm Med. 2022;22(268) 

In This Article

Results

Participants and the Characteristics of the Final Cohorts

A total of 1596 patients (535 from MIMIC-III, 521 from MIMIC-IV and 540 from eICU, respectively) were included in the final cohort to be analyzed (Additional File 1: Figure S1). The subjects pooled from MIMIC-III and eICU were randomly divided into a training cohort (70%, n=752) and an internal validation cohort (30%, n=323). Data from MIMIC-IV was used for external validation. In the training cohort, the overall in hospital mortality was 32.7% and 358 (47.6%) patients developed severe ARDS within the first 48 h of ventilation. Age, comorbidity of liver diseases, comorbidity of malignancy, vasopressor usage at admission, and severity of ARDS are shown significantly different between the deceased patients and the survivors in the training cohort (Table 1). Comparisons upon vital signs, laboratory test results and urine output within both the first 24 h of ICU and the first 24 h of invasive mechanical ventilation between survivors and non-survivors in training cohort are shown in Additional File 1: Tables S1, S2. Differences in ventilator parameters within the first 24 h of ventilation in training cohort are included in Table S2. Characteristics of interval validation and external validation cohort are presented in Table S3 and Table S4, respectively.

Predictors Selection and Model Development

A total of 176 variables measured within the first 24 h of ICU admission and within the first 24 h of IMV were included in the LASSO regression (Additional File 1: Figure S2). Twelve variables were identified through LASSO regression selection as significant predictors of in-hospital death, including six acquired within the first 24 h after admission to ICU: age, mean of respiratory rate, maximum of international normalized ratio (INR), minimum of red blood cell count, minimum of red blood cell distribution width (RDW) and maximum of alveolo-arterial oxygen difference (AaDO2), and six acquired within the first 24 h of IMV: mean of temperature, maximum of lactate, platelet, mean red cell volume (MCV), and minimum of blood urea nitrogen (BUN) and white blood cell count.

Subsequently, these twelve variables were included in a Logistic regression model and eventually, nine of them outstood as independently statistically significant predictors of in-hospital mortality were included in the risk model. Five variables were ascertained within the first 24 h after ICU admission, including age (OR, 1.02; 95% CI, 1.01–1.03), mean of respiratory rate (OR, 1.04; 95% CI, 1.01–1.08), the maximum of INR (OR, 1.14; 95% CI, 1.03–1.31) and AaDO2 (OR, 1.002; 95% CI, 1.001–1.003), and the minimum of RDW (OR, 1.17; 95% CI, 1.09–1.27). And four factors were measured within the first 24 h after start of IMV, including the mean of temperature (OR, 0.70; 95% CI, 0.57–0.86), the maximum of lactate (OR, 1.15; 95% CI, 1.09–1.22), the minimum of blood urea nitrogen (BUN) (OR, 1.02; 95% CI, 1.01–1.03) and white blood counts (OR, 1.03; 95% CI, 1.01–1.06) (Table 2). Figure 1 presents the nomogram of our model. Our model had a good discrimination (AUC: 0.77; 95% CI: 0.73–0.80) in the training cohort, featuring significant superiority over SOFA, OASIS, SAPS II, APACHE IV and APPS (De Long method, model vs. SOFA: P < 0.001; model vs. OASIS: P < 0.001; model vs. SAPS II P < 0.001; model vs. APACHE IV: P < 0.001; model vs. APPS P < 0.001) (Figure 2a) and good calibration (Calibration slope: 1.000, P =0.741; Brier score = 0.175) (Figure 3a).

Figure 1.

Nomogram to estimate the risk of mortality in ARDS patients. INR international normalized ratio, RDW red blood cell distribution width, AaDO2 alveolo-arterial oxygen difference, Tempc Body temperature, BUN blood urea nitrogen, WBC white blood cell, vent ventilation, max maximum, min minimum. Note: Variable name with the prefix of vent means the data was collected within the first 24 h of invasive ventilation

Figure 2.

The ROC curves of our model and other severity scores. a Training cohort; b Internal validation cohort; c External validation cohort. SAPS II simplified acute physiology score II, SOFA sequential organ failure assessment, OASIS oxford acute severity of illness score, APACHE IV acute physiology and chronic health evaluation IV, APPS Age, PaO2/FiO2, and Plateau Pressure Score

Figure 3.

Calibration of our model. a Training cohort; b Internal validation cohort; c External validation cohort

Model Performance

Discrimination and calibration of the model were evaluated in both internal and external validation cohorts. Our model remained well-discriminated in the internal validation cohort (AUC: 0.75, 95% CI: 0.69–0.80), which was greater than APACHE IV, SOFA, OASIS and APPS (AUC: APACHE IV 0.65; SOFA 0.62; OASIS 0.63; APPS 0.62; Figure 2b). Although the discrimination was lower than that of SAPS II (AUC: 0.76), no statistical significance was observed (De Long method, model vs. SAPS II P =0.49). In addition, a considerable calibration was showed in our model (Calibration slope: 0.846; Brier score = 0.183) (Figure 3b). In terms of predicting in-hospital mortality, the DCA results of our model, SAPS II, OASIS, SOFA, APACHE IV and APPS were shown in Figure 4. DCA of our model indicates that if the threshold probability of a patient is set between 20% and 60%, then the use of our model is more beneficial to patients compared with the extreme situation of mortality of ARDS in all patients or none. These findings suggest that our model provides a higher net benefit across a reasonably wide range of threshold probabilities for predicting mortality of ARDS, and thus has good clinical utility. The net benefit of our model was also better than the SAPS II, OASIS, SOFA, APACHE IV and APPS in this range. We further externally validated our model in a cohort of MIMIC-IV and our model outperformed the SAPS II, OASIS, SOFA and APPS (Figure 2c). The AUC of our model in external validation was 0.70 (95% CI: 0.65–0.74) with a brier score of 0.208 (Figure 3c). The performance of our model in patients of direct ARDS and indirect ARDS is shown in Additional File 1: Figure S3. The performance of our model in patients of transferred and non-transferred from other hospitals is shown in Additional File 1: Figure S4.

Figure 4.

Decision curve analysis of our model and other severity scores. a Training cohort; b Internal validation cohort; c External validation cohort SAPS II simplified acute physiology score II, SOFA sequential organ failure assessment, OASIS oxford acute severity of illness score, APACHE IV acute physiology and chronic health evaluation IV, APPS Age, PaO2/FiO2, and Plateau Pressure Score

processing....