An AI Got The Diagnosis Wrong. Doctors Trusted It Anyway

Doctors generally follow AI labels even when the clinical outcome in front of them suggests the AI is wrong, according to a study led by Aranzazu Vinas at the University of the Basque Country in Spain. Published in PLOS Digital Health, the research used two experiments with simulated cases. In each, 105 and 118 physicians, respectively, treated fictitious patients after an AI sorted them as “highly sensitive” or “lowly sensitive” to an experimental drug—yet recovery rates were identical across groups. Most clinicians didn’t catch the mismatch, and in one experiment the treatment appeared more effective for the AI’s preferred group despite equal improvement. The findings question how well human oversight works when context is limited.