Automation Bias ID’d for Radiologists of All Levels of Experience

May 12, 2023

FRIDAY, May 12, 2023 (HealthDay News) — Radiologists with all levels of experience reading mammograms are prone to automation bias when supported with an artificial intelligence (AI) system, according to a study published online May 2 in Radiology.

Thomas Dratsch, M.D., Ph.D., from the University of Cologne in Germany, and colleagues examined whether automation bias, the propensity to favor suggestions from AI, affects radiologists reading mammograms according to their level of experience. Twenty-seven radiologists read 50 mammograms and provided their Breast Imaging Reporting and Data System (BI-RADS) assessment. Mammograms were presented in two randomized sets: a training set of 10 mammograms with the correct BI-RADS category suggested by the AI system and a second set of 40 mammograms with an incorrect BI-RADS category suggested for 12 mammograms.

The researchers found that the percentage of correctly rated mammograms was significantly impacted by the correctness of the AI prediction of BI-RADS category among inexperienced (mean, 79.7 percent correctly rated mammograms when AI predictions were correct versus 19.8 percent when AI predictions were incorrect), moderately experienced (mean, 81.3 versus 24.8 percent), and very experienced (mean, 82.3 versus 45.5 percent) radiologists. Compared with moderately and very experienced readers, inexperienced radiologists were significantly more likely to follow the suggestions of the AI when it incorrectly suggested a higher BI-RADS category than the actual ground truth.

“It was surprising to find that even highly experienced radiologists were adversely impacted by the AI system’s judgments, albeit to a lesser extent than their less seasoned counterparts,” Dratsch said in a statement. “Our findings emphasize the need for implementing appropriate safeguards when incorporating AI into the radiological process to mitigate the negative consequences of automation bias.”

Several authors disclosed financial ties to the pharmaceutical and medical device industries.

Abstract/Full Text (subscription or payment may be required)