Using natural language processing on electronic medical notes to detect the presence of an indwelling urinary catheter
Methods: A concept lexicon was developed based on domain knowledge, prior expertise and review of medical notes. Concepts were classified as evidence of either catheter presence or catheter absence. A reference standard set of 1595 randomly selected documents from inpatient admissions were annotated by human reviewers to identify all positively and negatively asserted concepts. An NLP algorithm was tuned using 100 documents from the set. Novel lexicon semantics including evidence of catheter absence and inherently negated terms were used. Electronic medical record note titles with the highest hit rate for concepts were identified. The NLP algorithm was then tested on a set of 1495 documents to determine agreement between NLP and human reference standard, sensitivity and positive predictive value (PPV).
Results: The overall cohort included 5,589 unique patients with 77,938 hospital days from two VA hospitals over a one-year period. The lexicon contained 590 concepts for catheter presence (eg. Foley catheter was placed) and 18 for evidence of absence (eg. Patient has bathroom privileges). Iterative review of NLP outputs on the training set included false positive analyses and fine-tuning of the algorithm. Overall, nurse’s notes were the most frequent inpatient note titles; these also yielded the highest number of concepts with respect to urinary catheters. The overall agreement between the NLP and reference standard was 71%. With 348 instances of ‘evidence of catheter presence’ the system found 246 for a sensitivity of 87%. With 84 false positive concepts associated with catheter presence, the PPV was 59%. For ‘evidence of catheter absence’, the agreement was 72% (450 instances), sensitivity was 77% and PPV was 68%.
Conclusion: We have shown that it is possible to identify the presence of an indwelling urinary catheter from the free text of electronic medical notes. Further refinement and scaling-up of NLP algorithms to large document sets is ongoing. This is the first key step in developing protocols to assist humans in large-scale review of patient charts for CAUTI.
T. Forbush, None
A. Redd, None
M. Carter, None
A. J. Gendrett, None
K. Gupta, None
Y. Suo, None
B. S. B. Durgahee, None
S. Krein, Blue Cross and Blue Shield of Michigan Foundation: Grant Investigator, Research grant and Research support
Department of Veterans Affairs: Employee and Investigator, Research support and Salary
M. Rubin, None
A. Sales, None
M. Samore, None
B. W. Trautner, None