Development and Validation of a Clinical Risk Model to Predict the Hospital Mortality in Ventilated Patients With Acute Respiratory Distress Syndrome

A Population-Based Study

Weiyan Ye; Rujian Li; Hanwen Liang; Yongbo Huang; Yonghao Xu; Yuchong Li; Limin Ou; Pu Mao; Xiaoqing Liu; Yimin Li


BMC Pulm Med. 2022;22(268) 

In This Article


In this study, we developed and externally validated a clinical risk model and constructed a nomogram to predict the mortality of ventilated ARDS patients with LASSO method, which is suitable for the regression of high-dimensional data. Our model shows a moderate performance in predicting in-hospital mortality specifically for ventilated ARDS patients. Only nine simple variables routinely recorded in clinical practice are required for the prediction of in-hospital mortality in our model. Hence, our model can be easily implemented with the nomogram. In the validation cohort, the discrimination of our model was comparable to SAPS II and was significantly better APACHE IV, SOFA and OASIS.

Mortality prediction in ICU patients has been widely investigated in recent years, but the general ICU severity scores were not sufficient for predicting mortality in the population of invasively ventilated ARDS patients accurately and reliably. Several studies evaluated scoring systems (including APACHE IV, SOFA, APACHE II etc.) in ARDS patients, reporting poor to moderate discrimination for these scores.[24–26] In our study, the AUC of APACHE IV, SOFA and OASIS on predicting hospital mortality of ventilated ARDS were < 0.65 in internal validation cohort or external validation cohort, suggesting a low discriminatory power. Efforts on predicting mortality in patients with ARDS had been made by investigators. The APPS score, with a 9-point scale, incorporated the variables of age, plateau pressure and arterial oxygen partial pressure to fractional inspired oxygen ratio (PaO2/FiO2) reached an AUC of 0.80,[8] but its AUC significantly decreased to 0.62 in an independent cohort,[17] which is similar to the performance in our cohort. Zhao and colleagues constructed a model combining age, APACHE III, surfactant protein D (SP-D) and interleukin-8 (IL-8) for the prediction of ARDS mortality based on ALVEOLI cohort[7] and the performance in two external cohorts (FACTT and VALID)[27] were comparable to our model. However, neither SP-D nor IL-8 is prosaically tested in clinics, as well as the complicated calculation of APACHE III score consisting of a multitude of variables, turning the timely clinical decision making into a major challenge for intensivists confronting ARDS patients. Huang et al[28] constructed a model based on Random Forest algorithm showed better performance in external validation compared to our model (Random Forest vs. Logistic: 0.74 vs. 0.70) but included more variables (twelve) than ours. Generally, the performance of a scoring system improves as factors increase. In addition, Huang et al did not provide a visualized tool for evaluating the risk of mortality (nomogram or scoring system), which limits its clinical practicability. A systematic review[29] showed that regarding clinical prediction model with binary outcome, so far, no evidence supports that machine learning algorithm performs better than traditional Logistic regression in terms of prediction ability.

Therefore, we aimed at developing a model with a handful of routinely checked variables for mortality prediction for the ventilated ARDS patients that can be easily worked out by the bedside. Nine independent variables from 176 clinical features were finally identified using LASSO method and subsequent Logistic regression by examining the predictor-outcome association.

Interestingly, increased body temperature within the first 24 h of ventilation was negatively related to death in our model, which is consistent with results noted in two published studies.[30,31] Although a prospective clinical trial reported that aggressive fever suppression group showed a higher mortality compared to permissive group based on a cohort of critically ill trauma patients,[32] the underlying mechanism remained unknown.

In addition, RDW within the first 24 h of admission was included in our model as an important risk factor. RDW is a measurement of the amount of red blood cell variation in volume and size, which has been recently found to be abnormally increased in COVID-19[33] and an independent risk factor for the development and outcome of ARDS.[34–36] High lactate level is considered as a nonspecific marker for tissue hypoxemia, which has been reported as a predictive factor for a poor outcome among critical ill patients.[37–39] Another crucial predictive factor in our model is INR, which, however, was not included in existing risk scores. A previous study reported that INR was associated with hospital mortality of ARDS.[40] INR was also found to be significantly higher in ARDS patients with diffuse alveolar damage (DAD) compared with those without DAD.[41] Other variables such as advanced age, high respiratory rate, increased AaDO2, high BUN, and hyperleukocytosis were found to be associated with ARDS events or outcome of ARDS.[8,11,42–44]

Our model is simple for calculation and easy to use with the nomogram, and has robust discrimination and calibration. Besides, we carried out the decision curve analysis to explore the clinical use of our model, and there was a considerable range of alternative threshold probability. Also, our model was constructed based on multicenter data and the external validation was also performed, which improved its generalizability. Moreover, the predictors that we adopted are no extraordinary data regularly obtained from the patients, enabling ICU caregivers to predict the mortality risk of ventilated ARDS patients and improve clinical decision-making right at the bedside. A previous study which secondly analyzed the VALID trial, reported that direct and indirect ARDS have distinct features that may differentially affect risk prediction and clinical outcomes, while the discrimination of our model seems to be stable and did not affect by direct or indirect etiology.[22] The discriminations of our model seem to be not affected by direct or indirect etiology (direct: AUC: 0.69, 95% CI: 0.63–0.75) (indirect: AUC: 0.70, 95% CI: 0.62–0.78) (P=0.858). Whether the predictors of hospital mortality differ among different etiologies of ARDS is still unknown and studies focus on this are valuable.

Several limitations need to be acknowledged. First of all, as the study was retrospective and observationally designed, several inherent limitations like selection bias, loss to follow up and the presence of confounding factors cannot be avoided. Further prospective studies are needed to evaluate the effectiveness of our model. Secondly, some of the variables were excluded for the missing data although previous research has shown that they might be associated with mortality of ARDS patients, such as albumin,[45] hepatic function[46,47] and neutrophil-to-lymphocyte ratio (NLR).[48] Thirdly, some studies reported that echocardiographic findings were associated with the outcomes of ARDS and COVID-19,[49–51] but we did not include relevant variables because information of echocardiographic findings was only available in a small part of patients in MIMIC-III database but not recorded in eICU and MIMIC-IV databases. Future study including echocardiographic findings is appreciated. Last but not least, similar to previous risk sores, the results of external validation indicated that the discrimination and calibration were decreased compared with that of the training cohort and internal validation cohort, with an overestimate of hospital mortality, which would be owing to the changing strategy of managing ARDS over a long period. Further optimization with more updated data of ARDS patients (recent five years) would be appreciated.