(709) Risk Factor Selection by Bootstrap to Determine Variables for a New Morbidity Risk Score

Introduction: One way to identify new morbidity in the PICU is by an increase in functional status score (FSS). We present a covariate selection method to model PICU new morbidity risk, defined as an FSS increase of 3 or more from ICU admission to discharge.

Methods: VPS, LLC registry data on 69,491 survivors from 27 units from April 2017 to March 2023 was used. The data were divided into 52,148 training cases and 17,343 cases for testing. Nonparametric bootstrap modeling was used to draw samples from the training set (N=100). Forward stepwise logistic regression by AIC was performed on each bootstrap sample, correlating the outcome with variables from PIM3, PRISM III, and Pollack’s trichotomous outcome study. Collinearity was controlled with a variance inflation factor less than 5, and significant p-values less than 0.05. Variables chosen by each bootstrap run were tabulated. Only variables appearing in at least 30% of the bootstrap runs were retained as new morbidity risk factors. A final model was built using these and expert selection, and validated on the test set. The test set was also used to investigate new morbidity in patients with common diagnoses (RSV,Asthma,etc). Dependence of new morbidity outcome on volume was investigated with a funnel plot of standardized morbidity ratio (SMorbR) versus unit volume.

Results: The morbidity rates for the training and testing sets were 6.16% (3,213 cases) and 6.28% (1,090 cases), respectively. In the test set, SMorbR was 1.00. The AUROC was 81.3%, and the GiViTI bisector fell within the calibration belt for all expected new morbidity risk ranges (p=0.26). The final model contained 25 variables.
Subgroup analyses on the testing (validation) set showed the group “Bronchiolitis Due to RSV” had the largest SMorbR (1.31, 95% confidence interval [CI] 0.86 to 1.75). The funnel plot with 95% CI did not show volume-outcome dependence. First admissions had larger SMorbR than readmissions, 1.10 (95% [CI] 1.03 to 1.17) versus 0.70 (95% CI 0.60 to 0.80).

Conclusions: A supervised data-driven covariate selection method for modeling risk of new morbidity was introduced. The test set showed excellent SMorbR and reasonable discrimination and calibration. This methodology may be useful for constructing other standardized score models.