Internal Medicine · 22 h ago
AI assistance improves radiologist performance in parotid ultrasound study
A prospective, two-phase observer-performance study evaluated a confidence-aware AI system for distinguishing pleomorphic adenoma from Warthin’s tumor on ultrasound. Across 321 reads of 107 external cases by three radiologists, AI assistance improved pooled accuracy and inter-reader agreement, although sensitivity shifted in opposite directions for the two tumor types.
- Accuracy increased from 76.3% to 81.6% with AI assistance.
- Inter-reader agreement improved in the observer-performance study.
- Sensitivity changes differed between the two tumor types.
- Real-world workflow evaluation remains necessary.
Researchers developed a confidence-aware AI pipeline using internal data from 949 ultrasound images and external data from 427 images. The system combined a YOLO detection model with a classification-model fallback. In a prospective two-phase observer-performance study, three board-certified radiologists assessed 107 external cases, producing 321 reads, with and without AI assistance.
Standalone detection and classification models had external AUROCs of 0.811 and 0.919, respectively. The combined pipeline outperformed both standalone models in 22 of 35 tested combinations. With AI assistance, pooled diagnostic accuracy rose from 76.3% to 81.6% (odds ratio 1.54, 95% CI 1.12–2.13; p=0.008), and Fleiss’ κ increased from 0.475 to 0.611. Readers followed correct AI predictions more often than incorrect ones (76.2% vs 51.1%; p=0.008).
Sensitivity increased by 18.1 percentage points for Warthin’s tumor and decreased by 9.3 points for pleomorphic adenoma, reflecting the AI’s prediction profile. The findings suggest potential for decision support, but the study involved three readers and a defined external case set; evaluation in real-world clinical workflows is still needed.
In practice
For radiologists interpreting ultrasound to distinguish pleomorphic adenoma from Warthin’s tumor, this study provides prospective observer-performance evidence that a confidence-aware AI system may improve pooled accuracy and agreement when used as decision support. The evaluation included three radiologists and 107 external cases, not routine clinical workflows. Sensitivity moved in opposite directions by tumor type, so the aggregate accuracy gain does not establish uniform benefit for each diagnosis. These results support further clinical evaluation, not a change in diagnostic practice on their own.
AI-generated clinical interpretation · Not editor-reviewed
Is this summary clinically accurate?
Help fellow clinicians: your rating sends inaccurate summaries straight to our editors.
Sign in to rate this summary →Source
BMC Medicine: Human–AI interaction in parotid ultrasound diagnosis: a prospective two-phase observer-performance study ↗This is an automated AI-condensed summary that has not yet been reviewed by an editor. Always consult the full item at the original source.