Experts defer to AI in decision-making, Anthropic study finds
Professional analysts chose options that matched an AI assistant's preference 86 percent of the time in a new study by Anthropic, the creator of Claude.
In a research paper published on Tuesday, Anthropic reported that 50 professional analysts were asked to advise a fictional farming cooperative on whether an advisory program should be expanded. Participants made four choices between two defensible options before providing a final policy recommendation.
Unknown to the analysts, Anthropic had directed each participant's AI assistant to lean toward a particular decision to measure influence. The results showed that analysts chose options matching the AI's preference 86 percent of the time, significantly higher than the 50/50 outcome expected if they weren't influenced.
Study Limitations
The study had some limitations, noted Anthropic, including that it involved just one task and used AI assistants that had been deliberately set to have a preference. Researchers found that participants who had a preference before consulting their AI changed their decision 60 percent of the time when the AI disagreed.
Humans' tendency to defer to AI may be 'hard to identify, even for the experts themselves'
Policy Implications
Anthropic researchers warned that their findings raised the possibility that AI's 'priors' could shape consequential policy decisions, including in AI safety. They emphasized the need for oversight that examines how decisions are made, helping experts articulate their assumptions and values.
The study comes as Anthropic, which is preparing for an IPO valued at up to $2 trillion, and its rivals face intense public debate about managing the potential dangers of frontier AI. While proponents believe AI will revolutionize productivity, concerns persist about mass job losses and cyberattacks.
Related
