Fluent language makes a hallucinatory chatbot just as credible as a curt, flawless one

edited by Swati Mestri, reviewed by Andrew Zinin

This article has been reviewed according to Science X's editorial process and policies. Editors have highlighted the following attributes while ensuring the content's credibility:

Add to Preferred Sources Credit: Pixabay/CC0 Public Domain Most people are by now quite familiar with ChatGPT, Gemini, Claude and other chatbots: Every day, people ask millions of questions and receive neatly formulated answers in return. Yet these language models still tend to run wild with so-called "hallucinations": unreliable, dubious answers that are presented with great certainty as the truth.

Communication scientist Marianna de Sá Siqueira and her colleagues investigated the effects of elegant language use on the credibility of chatbots.

The paper is published in the journal Computers in Human Behavior: Artificial Humans.

For the experiment, participants were shown one of four variants of a conversation with a chatbot about "The Lord of the Rings." Two variants featured fluent sentences and humanlike language, one of which gave correct answers and the other did not. In addition, there were two variants with short, businesslike answers, one of which provided entirely correct information while the other presented (subtly) fabricated facts.

Unsurprisingly, users were most positive about the chatbot with elegant language and correct answers. However, a chatbot that writes fluently but spouts factual nonsense was rated by participants as just as intelligent and pleasant as a chatbot that communicates in a stilted manner but does provide factually correct information. It also emerged that as many as 80% of participants did not notice the errors at all, even though they had been selected in advance for their basic knowledge of "The Lord of the Rings."

"Above all, it shows how easily people can be led astray and how quickly we come to trust something that is actually spouting nonsense," says Barbara Müller, associate professor and co-author of the study. "If a text sounds elegant and natural, people automatically attribute intelligence to it. You then subconsciously tend to think: This is fine, I don't need to check it any further."

The research highlights the danger of blind trust in artificial intelligence. Professor and co-author Tibor Bosse says, "So-called language models such as GPT and Claude are primarily trained to produce statistically plausible and easy-to-read sentences. These systems do not always take the truth very seriously. They are designed to sound human and convincing, but whether the information is factually correct is of secondary importance."

"People used to complain that chatbots were annoying and knew nothing. Now they sound so clever that we're too quick to believe them," says Bosse. According to him, this cannot be solved with a simple disclaimer at the outset.

"Most chatbots show a single line of text at the start stating that the output may not be entirely accurate, but those companies are certainly not going to indicate of their own accord with every answer that the system is uncertain. If we really want to protect users from chatbots spouting nonsense, we need to, for example, oblige tech companies to make that uncertainty visible. If a chatbot itself expresses doubt, you prevent people from taking nonsense as the truth."

Marianna A. de Sá Siqueira et al., Users' perception of chatbot language fluency and hallucination, Computers in Human Behavior: Artificial Humans (2026). DOI: 10.1016/j.chbah.2026.100412 Key concepts AI chatbot safety Provided by Radboud University Who's behind this story? Swati Mestri Swati Mestri holds a bachelor's degree in Electronics Engineering and has worked as a content editor since 2019. She has experience editing research documents across technology, health care, and materials science, and has a particular interest in technology and space. Full profile →

Master's in physics with research experience. Long-time science news enthusiast. Plays key role in Science X's editorial success. Full profile →