1823. Signal or Noise? A comparison of methods to identify outliers in antimicrobial use (AU)
Session: Poster Abstract Session: Antimicrobial Stewardship: New Methods and Metrics
Saturday, October 6, 2018
Room: S Poster Hall
  • Signal vs Noise Poster IDW2018 v3.pdf (329.5 kB)
  • Background: Antimicrobial Stewardship Programs (ASPs) use AU benchmarking data to help identify areas in need of investigation. The high frequency and wide variation in AU make statistical tests frequently significant.

    Methods: We compared four statistical methods of analyzing AU data to quantify how often statistically significant outliers occur. We analyzed days of therapy (DOT) per 1000 days present (dp) from 2017 in medical and surgical adult wards and three NHSN AU antibiotic groups: anti-MRSA agents (anti-MRSA), broad agents for community-onset infections (CO), and broad agents for hospital-onset multidrug resistant organisms (HO/MDRO). Outliers were defined as follows: 1. Units ≥90th or ≤10th percentiles, 2. Units with Standardized Antimicrobial Administration Ratios (SAARs) outside 95% confidence intervals (CI), 3. Units with observed rates outside 95% CI predicted by a generalized estimating equation (GEE) negative binomial regression model 4. Units with observed rate outside 95% CI predicted by mixed effects negative binomial regression model with hospital as a random effect. Adjustment in method 2 included hospital teaching status and location type. Methods 3 and 4 included adjustment for teaching status, location type, average age, average hospital length of stay, surgical volume, percent sepsis admissions, and average DRG weight.

    Results: Fifty-five units and 628,358 dp were included in the 1-year sample. Each method identified both positive and negative outliers. SAAR and GEE methods identified the largest number of outliers; percentiles identified the least (Table). The four methods identified different individual units as outliers (Figure).

    Conclusion: Overly sensitive statistical methods may produce more signals than are clinically meaningful. Investments of ASP resources to investigate such signals may vary widely depending on statistical method used.  Additional research is required to develop AU analysis methods with high positive predictive value.

    Table. Number (%) of outlier units identified using four statistical methods


    AU in DOT/1000 dp median (IQR)

    1. Percentile

    2. SAAR

    3. GEE model

    4. Mixed model


    84 (73-103)

    10 (18%)

    42 (76%)

    30 (55%)

    14 (26%)


    132 (106-184)

    10 (18%)

    50 (91%)

    22 (40%)

    14 (26%)


    132 (118-151)

    12 (22%)

    38 (69%)

    31 (56%)

    14 (26%)


    Rebekah W. Moehring, MD, MPH1, Eric Lofgren, MSPH, PhD2, Elizabeth Dodds Ashley, PharmD, MHS, FCCP, BCPS1, Deverick J. Anderson, MD, MPH, FIDSA, FSHEA1 and Yuliya Lokhnygina, MS, PhD3, (1)Duke Center for Antimicrobial Stewardship and Infection Prevention, Durham, NC, (2)Washington State University, Pullman, WA, (3)Biostatistics and Bioinformatics, Duke University, Durham, NC


    R. W. Moehring, None

    E. Lofgren, None

    E. Dodds Ashley, None

    D. J. Anderson, None

    Y. Lokhnygina, None

    Findings in the abstracts are embargoed until 12:01 a.m. PDT, Wednesday Oct. 3rd with the exception of research findings presented at the IDWeek press conferences.