Why some websites are blocking AI agents from accessing the web

Amazon is blocking rival shopping agents, and even websites that welcome AI assistants can stop them with an ordinary CAPTCHA.

Erlin Diah/Shutterstock Add Engadget on Google: Preferred Source Google Discover An AI agent can take over your grocery shopping, right up until a website asks it to prove it's human. That's the problem Meta's Muse has encountered at Walmart (one of its retail partners), which says the verification checks weren't intended to stop the assistant. Amazon has been more direct, deliberately blocking the tool from shopping on its site, according to TechCrunch.

The appeal of personal AI agents like Meta's Muse is that these assistants can navigate websites, fill shopping carts and book travel instead of handing you recommendations and leaving the rest to you. However, giving an agent permission to act on your behalf doesn't mean a website has agreed to deal with it. Some restrictions are deliberate, while others come from security systems that can't reliably distinguish legitimate agents from unwanted automated traffic. Either way, you could end up back at the keyboard doing the very task you wanted to avoid.

davide bonaldo/Shutterstock Security is a bigger concern when money is involved. During JPMorgan Chase's October earnings call, CFO Jeremy Barnum said some agents use shared login credentials and screen scraping to transfer funds, methods he described as "not secure." The bank blocks those methods where possible, and Barnum warned that customers who hand over their credentials could be responsible if an agent takes an unwanted action. A botched grocery order is annoying enough, but an unauthorized bank transfer is a different kind of headache.

Amazon is also battling Perplexity over its Comet AI browser. The retailer argues that the agent violates its access rules and puts customer data at risk, while Perplexity maintains that users have authorized it to act. A federal appeals court vacated a preliminary injunction against Perplexity in August 2026, although the underlying dispute remains unresolved. There's also an interesting wrinkle: Amazon has its own Buy for Me feature that can purchase products from other brands' websites.

Businesses have commercial reasons to control automated access, too. eBay's user agreement prohibits unauthorized buy-for-me agents unless the company grants permission. Delta told TechCrunch it has no integration allowing third-party agents to book flights through its website, although the airline did not confirm whether it deliberately blocks specific agents.

Pocketlight/Getty Images Walmart's experience shows why a blocked agent isn't necessarily the result of a deliberate ban. Human-verification prompts can interrupt automated shopping sessions, while Zillow has acknowledged that its anti-scraping defenses sometimes catch agents it otherwise permits. From the user's perspective, both can look like an assistant that simply can't finish the job.

Anyone who's browsed through a VPN might recognize the routine. Google explains that shared VPN networks can trigger reCAPTCHA challenges when their traffic resembles automated activity. AI agents can encounter similar checks simply by navigating a website, even when following someone's instructions. Switching on a VPN won't magically override a site's restrictions on automated access, either.

Cloudflare, which previously introduced controls against AI web crawlers, now lets website owners distinguish between search crawlers, AI training bots and agents acting on behalf of users. Since September 15, new domains using Cloudflare have defaulted to allowing search crawlers while blocking AI training bots and agents on pages displaying ads. Website owners can adjust those rules, meaning a site blocking AI training bots isn't necessarily rejecting an assistant trying to buy something. Other platforms are taking a more controlled approach. Shopify's checkout tools let compatible browser agents update checkout details and submit orders after buyer confirmation, with control handed back to shoppers for required authentication steps.