433. Inter-rater Reliability of Lung Ultrasounds for Detection of Pneumonia in Children
Session: Poster Abstract Session: Pediatric Infections
Thursday, October 3, 2013
Room: The Moscone Center: Poster Hall C
Background: Lung ultrasonography (LUS) is increasingly being used to detect pneumonia in children. Using chest radiograph as a reference standard, studies have found LUS to have >85% sensitivity and specificity in detecting pneumonia. However, the interpreting sonologists have rarely been blinded to clinical information. The objective of this study was to determine the inter-rater reliability among pediatric radiologists blinded to the clinical information, with and without a standardized ultrasound protocol.

Methods: LUS performed between January 2011-March 2013 on children 3 months-18 years were included. Five pediatric radiologists were trained on LUS interpretation for signs indicative of pneumonia. Two sets of LUS were read. The first set consisted of 40 LUS that were performed for clinical reasons. These LUS were performed without a standardized protocol to ensure coverage of all portions of the lung. The second set consisted of 15 ultrasounds that were performed using a standardized, evidence-based protocol.  Inter-rater reliability was calculated using a kappa statistic. 

Results: The inter-rater reliability for lung consolidation was fair for standardized LUS (0.40) and for unstandardized LUS (0.36). Interstitial disease had slight to poor agreement regardless of protocol. Pleural effusion had fair inter-rater reliability in the unstandardized set (0.29) but almost perfect reliability in the standardized set (0.82) (Table 1).

Conclusion: These preliminary data suggest that standardization of sonogram readings leads to higher inter-rater reliability in detecting pleural effusions in children using LUS. However, reliability, while improved, remained low to moderate for other findings of pneumonia. This may indicate a need for clearer definitions in this relatively new application of LUS.

Table 1: Inter-rater reliability of lung ultrasounds using unstandardized and standardized ultrasound protocols*

Overall Conclusion

Unstandardized LUS

Standardized LUS

Lobar or Patchy Consolidation



Interstitial Disease



Pleural Effusion



* Classification of the strength of agreement: poor (<0.0); slight (0-0.20); fair (0.21-0.40); moderate (0.41-0.60); substantial (0.61-0.80); almost perfect (0.81-1.0)

Lilliam Ambroggio, PhD, MPH1, Matthew Test, MD1,2, Sara O'hara, MD3, Diane Babcock, MD3, Mantosh Rattan, MD3, Alan Brody, MD3, Maurizio Macaluso, MD, DrPH4, Samir Shah, MD, MSCE5 and Brian Coley, MD3, (1)Division of Hospital Medicine, Cincinnati Children's Hospital Medical Center, Cincinnati, OH, (2)Division of Hospital Medicine, Cincinnati Children's Hospital Medical Center, Cincinati, OH, (3)Department of Radiology, Cincinnati Children's Hospital Medical Center, Cincinnati, OH, (4) Division of Biostatistics and Epidemiology, Cincinnati Children's Hospital Medical Center, Cincinnati, OH, (5) Center for Pediatric Clinical Effectiveness, Children's Hospital of Philadelphia, Philadelphia, PA


L. Ambroggio, None

M. Test, None

S. O'hara, None

D. Babcock, None

M. Rattan, None

A. Brody, None

M. Macaluso, None

S. Shah, None

B. Coley, None

See more of: Pediatric Infections
See more of: Poster Abstract Session

Findings in the abstracts are embargoed until 12:01 a.m. PST, Oct. 2nd with the exception of research findings presented at the IDWeek press conferences.