(726) Evaluation of Clinician "Gut" Instinct to Improve Detection of Children at Risk for Deterioration

Introduction: Unrecognized deterioration poses a significant risk of harm to hospitalized children. Clinician ‘gut instinct’ may enhance detection of deterioration, but its predictive performance is unknown. After embedding a pilot clinician concern (CC) process into our rapid response system (RRS) on 4 medical units, we hypothesized that CC would have >50% sensitivity and > 10% positive predictive value (PPV) for deterioration.

Methods: We performed a retrospective study of CC from 7/2022-7/2023. The CC process asks bedside nurses to subjectively assess a patient’s deterioration risk along with vital signs on a 4-point scale - ‘no concern’ to ‘significant concern’ - with embedded decision support for moderate or significant concern to guide utilization of other RRS elements, which includes a ‘watcher’ program, medical emergency team, code blue team, and critical care outreach team.

We collected admissions, concern scores, and deterioration events requiring transfer to the PICU on pilot units. We excluded units with < 60% CC process utilization. During each admission, we evaluated the highest documented concern in every 12-hour interval starting at 7am and 7pm. Intervals with missing concern were analyzed as ‘no concern.’ For the primary exposure, we dichotomized the 4-point score into moderate or significant (‘high’) concern vs. no or slight (‘low’) concern in an interval. The primary outcome was PICU transfer in the index or following interval (i.e., 24 hours). We calculated point estimates and exact binomial 95% confidence intervals (CI) for sensitivity, specificity, PPV, and negative predictive value (NPV).

Results: Of 4 units, 3 (75%) met utilization criteria. Of 4468 admissions, 4095 (92%) had ≥ 1 documented score. Of 95,599 scores, 88,865 (92.96%) were ‘no concern;’ 6,020 (6.30%) ‘slight concern;’ 685 (0.72%) ‘moderate concern,’ and 29 (0.03%) ‘significant concern’. There were 195 PICU transfers. ‘High’ concern had a sensitivity of 23.3% (CI 19.9-27.9), specificity of 99.5% (CI 99.4-99.5), PPV of 22.6% (18.5-27.1), and NPV of 99.5% (99.5-99.6).

Conclusions: Clinician concern for deterioration had lower than hypothesized sensitivity, but high PPV – equivalent to a number needed to alarm of only 4.4. Future study will compare the marginal benefit of this system to existing RRS components.