Methods: We use a mathematical model of vancomycin-resistant enterococci (VRE) in an ICU to simulate two interventions with a known effect – a chlorhexidine gluconate (CHG) bathing protocol (1), and a decrease in hand hygiene adherence. In both cases, a stochastically simulated time series is obtained, and then two methods – a mathematical model and an interrupted time series – are used to attempt to estimate the value of the known, simulated effect.
Results: For both cases we show that as the number of events decreases, the effect becomes more difficult to detect. While unsurprising, this result provides a more intuitive illustration than power calculations. We conclude by showing a way to evaluate a hospital’s performance that addresses this problem, using a model to show an individual hospital’s infection rates in the context of stochastic uncertainty.
Conclusion: As infection control practice improves, it will become more difficult to detect new improvements, making the evaluation of new interventions harder. Conversely, increases in cases will become easier to detect. While this may be addressable in intervention studies by increasing the number of patients recruited or the number of sites, this may be cost prohibitive. Worryingly, in many circumstances such as in evaluations of healthcare quality, individual hospitals are judged in isolation, and as infection control improves, the results of these economically important evaluations will become dominated by the effect of random chance. New metrics, evaluation guidelines, etc. must be adjusted to take the role of randomness into account.
1. Dicks, K.V., et al. (2016). A Multicenter Pragmatic Interrupted Time Series Analysis of Chlorhexidine Gluconate Bathing in Community Hospital Intensive Care Units. ICHE, 1–7.
E. Lofgren, None