(1300) Performance Evaluation of the Discharge Readiness Score Across eICU Data From 2007 to 2021

Introduction: Published in 2012, the Discharge Readiness Score (DRS) is currently in use in tele-ICUs across a network of over 400 hospitals across the US. It predicts mortality within 48 hours of ICU discharge, and it is used to continuously assess discharge readiness. The goals of this work were to make a minor revision to the existing model that reduced documentation burden of ventilation data, retrain on original data; and validate the updated model on a dataset from 2007-2021.

Methods: To develop the updated DRS model, the 7 original ventilation features were replaced by a binary variable (mechanically ventilated or not), and the logistic regression model was retrained using the initial published dataset (up to 2011). The validation was expanded to included data from 2007 – 2021 yielding 1,167,435 ICU patient-stays from the eICU database. The proportion of the patients who died within 48 hours (including DNR patients) of ICU discharge was approximately 1.4%.

Results: The updated DRS model achieved an AUC (area under the ROC curve) of 0.92 over the entire population between 2007-2021. Notably, this is consistent with the AUC reported in the original model. However, when analyzed per year, the AUC showed a drop over the COVID-19 period to 0.91 in 2020 and 0.90 in 2021. Over these two years, the 48-hour mortality rate increased to 1.6%, accompanied by a general rise in the length of stay from 2.6 days to 3.3 days. Additionally, there was a 3% increase in the proportion of male patients compared to the pre-COVID data. The updated model performed well across unit types. However, it showed a drop in performance for Cardiovascular ICUs (AUC 0.78) and Cardiac-surgery ICUs (AUC 0.88) compared to other units (AUC 0.92). The model also exhibited relatively consistent performance across admission types. This included step down unit patient admissions where the model showed good performance (AUC 0.90) despite not being trained on these units. The model also showed equivalent performance across race and gender (AUC 0.92 for males and females).

Conclusions: Although the model was trained on earlier data (up to 2011), it showed good performance for a large dataset including the COVID-19 period suggesting that it can be clinically used for patient management.