Why would your AI agent rather guess than look?

An eye under a magnifying glass, an illustration of AI agents checking sources

Try asking an AI assistant what a mid-range hotel in Lisbon costs for next weekend. You’ll get a number, usually stated with complete confidence. And if you then ask where it came from, the explanation gets vague fast.

Sometimes the agent really did check a booking site a few seconds earlier. And sometimes it checked nothing and handed you a figure from its training data, possibly from a summer two years back. It does so in the same even voice it uses when it’s right.

People tend to call this a hallucination. I’d push back on the word a little, since in many of these cases the model recalled a price that was real at some point. Nothing in the system asked it to confirm the figure still held.

Before an agent answers anything, it chooses between replying from memory and going out to find and read a page. In the developer and AI communities, it’s called the fetch decision.

The user never sees it. Almost none of the teams we work with at Decodo record it anywhere, even though their dashboards will show the cost of every query to a fraction of a cent.

In most agent frameworks, an engineer doesn’t even make that call. Web search is exposed to the model as a tool it may use. The model judges in the moment whether the question needs it.

That judgment is weaker than it looks. A model has no sense of how old any single fact in its memory is. So a hotel rate from 2024 sits in its weights with the same undated certainty as the boiling point of water. Nothing prompts it to treat one differently from the other.

Whatever does come back has to be stripped down before the model reads it. Every paragraph is paid for in tokens and seconds. Recall costs almost nothing by comparison.

Product teams then judge agents on speed and smoothness. In a review meeting, an agent that replies “I couldn’t load that page” looks broken while the one that quotes a stale price passes.

Over a few release cycles, every tweaked prompt and every default picked under deadline pressure makes the lookup a little easier to skip, until answering from memory becomes the agent’s normal behaviour.

You can’t see the fetch decision when you use an AI assistant, but you can usually spot what came out of it. An answer built on a live page tends to show where it came from, with a link to the page and often the date it was checked. An answer from memory reads just as smoothly, only without the link.

For anything that changes often, like a price or a flight time, I’d treat an answer with no source as a guess. Ask the assistant to look it up and show you the source it took information from. That one extra request often changes the answer.

Companies buying AI tools can ask the same question before they sign. Ask the vendor how often its agent checks the web before answering a question about price or availability. Also ask what the user sees when that check fails.

Web browsing is becoming a standard feature, and most AI agents can already do it. However, not all can. A few still can’t reach the web on their own.

For those, check whether you can connect a third-party web data service through a plugin or an API. That way, the agent searches and reads live pages instead.

Switching to a newer model is usually the first fix developers try, and it does less than they hope. The newest models and AI tools can browse the web, yet they purposely decide to skip it.

If the model judges a search unnecessary, it answers from what it learned in training. For something like a price, that can be well out of date.

Wording often decides whether a search happens. A question with “today” or “latest” in it will usually send the agent to the web.

“What does our main competitor charge for its entry-level plan?” can still get an answer from memory, even though that price may have changed twice since the model was trained.

The fix is in how the product is built around the model. If I were reviewing an agent product this week, I’d ask for four things:

That last point is the one we work on most at Decodo, and it’s why we built the Web Data API.

With one API key, an agent can search the web, see how a site is organised, collect pages across it, or pull the content of a single page.

It runs on the ethically-sourced IP pool, which is what lets it read sites that block automated visitors. The page comes back as clean text the agent can use straight away, rather than raw website code.

Go back to the same competitor question. With live web access in place, the agent opens the competitor’s live pricing page and reads today’s price there. The answer comes from the page, not from what the model remembers.

Most AI tools can already search the web, and many show their sources (when they want to). What they still lack is consistency.

An agent that checks the latest price on one question and answers from memory on the next, with no way to tell which is which, is hard to trust with anything that costs money.

Vaidotas Juknys is CEO at Decodo, a leading web data access layer for AI agents, LLMs, and automated tools. Vaidotas has over a decade of ex (show all) Vaidotas Juknys is CEO at Decodo, a leading web data access layer for AI agents, LLMs, and automated tools. Vaidotas has over a decade of experience spanning technology, telecommunications, and management consulting, with a clear focus on making web data practical for teams building the next generation of AI products. He regularly writes and speaks on cybersecurity, web data infrastructure, and how AI is reshaping the way businesses access the open web.

Get the most important tech news in your inbox each week.

Thanks! Check your inbox to confirm your subscription.

By subscribing you agree to our Privacy Statement.

Expertise from selected TNW Council members, admitted through an application and review process to a fee-based program. Opinions expressed by the authors are their own.