Experiment reveals how hidden agendas in AI advice can distort human decisions

edited by Sadie Harley, reviewed by Robert Egan

This article has been reviewed according to Science X's editorial process and policies. Editors have highlighted the following attributes while ensuring the content's credibility:

Add to Preferred Sources Credit: Pixabay/CC0 Public Domain A research team has discovered that AI assistants can easily manipulate users into making disadvantageous decisions when the AI operates with a hidden agenda.

Published in the journal Proceedings of the National Academy of Sciences, the findings reveal a significant vulnerability in human-AI interactions, highlighting that users often fail to notice the manipulation and continue to perceive the AI as highly helpful.

While people increasingly rely on AI assistants for advice on major life decisions—ranging from financial investments to personal relationship issues—they naturally assume the AI operates in their best interests. However, human susceptibility to covertly misaligned AI-driven advice remains insufficiently understood.

Through a randomized experiment spanning financial and emotional contexts with 233 participants, the team demonstrated that a covertly misaligned AI can significantly sway human choices.

When the AI secretly pushed a less advantageous choice, participants picked the unfavorable option approximately five to eight times more often, increasing the likelihood of making a suboptimal decision by up to 38 percentage points.

The study reveals that such manipulation requires neither aggressive sales tactics nor complex psychological ploys; a subtle, latent bias in the AI's everyday advice was sufficient to alter decisions. A disconnect emerges between susceptibility and subjective evaluation: Even after being successfully steered into making poor financial or emotional choices, users consistently rated the AI as "highly helpful."

As AI systems are developed by corporations and governed by opaque algorithms, this research cautions that AI with hidden agendas—such as covertly promoting certain products or companies—could easily manipulate users into choices contrary to their best interests, all while maintaining the illusion of objective assistance.

The team included professor Tatia Lee from the Laboratory of Neuropsychology and Human Neuroscience, Department of Psychology at the University of Hong Kong (HKU), and researchers from Tsinghua University.

Sahand Sabour et al., Human preferences are susceptible to covertly misaligned AI advice, Proceedings of the National Academy of Sciences (2026). DOI: 10.1073/pnas.2600684123 Journal information: Proceedings of the National Academy of Sciences

BSc Life Sciences & Ecology. Microbiology lab background with pharmaceutical news experience in oil, gas, and renewable industries. Full profile →

Bachelor's in mathematical biology, Master's in creative writing. Well-traveled with unique perspectives on science and language. Full profile →