Three bugs made Claude worse, and the checks never saw it

Posted on Oct 19 • Originally published at trustboundarystudio.com

Originally published at trustboundarystudio.com. The video version, with diagrams, is on YouTube.

In early August 2025, people using Claude started saying it had got worse. Answers that were off. Code with obvious syntax errors in it. In one example Anthropic gives itself, Thai characters in the middle of an English reply.

They were right. Between August and early September, three separate infrastructure bugs degraded Claude's responses. Nothing went down. Nothing was attacked. At the worst hour, on 31 August, 16% of requests to Sonnet 4 were affected, and the checks that are meant to catch exactly this had passed.

Anthropic's own post-mortem, published 17 September 2025, is almost entirely about routing, a server misconfiguration and a compiler. It is a clean example of the difference between a service being up and a service being right.

Anthropic serves Claude through its own API, Amazon Bedrock and Google Cloud's Vertex AI, on three kinds of hardware: AWS Trainium, NVIDIA GPUs and Google TPUs. Each needs its own optimisations. The rule across all of them, in the post-mortem's words: "Despite these variations, we have strict equivalence standards for model implementations."

So every change to the plumbing has to hold everywhere. In August 2025, three did not.

On 5 August, some Sonnet 4 requests began to be "misrouted to servers configured for the upcoming 1M token context window." That affected approximately 0.8% of requests to Sonnet 4, which is small enough to look like noise.

Then, on 29 August, "a routine load balancing change unintentionally increased the number of short-context requests routed to the 1M context servers." At the worst impacted hour, on 31 August, 16% of Sonnet 4 requests were affected.

It was not spread evenly. Routing was sticky, so "once a request was served by the incorrect server, subsequent follow-ups were likely to be served by the same incorrect server." Some people got degraded answers again and again.

The figure most likely to be misquoted: "Approximately 30% of Claude Code users who made requests during this period had at least one message routed to the wrong server type, resulting in degraded responses." That is a share of users, not 30% of requests, and it does not add to the 16%, which is a share of Sonnet 4 requests in one hour. The post-mortem gives no overall total, and neither does this article.

On Amazon Bedrock, misrouted traffic peaked at 0.18% of all Sonnet 4 requests; on Vertex AI it affected less than 0.0004% of requests between 27 August and 16 September. The fix was deployed on 4 September and, per the post-mortem, fully rolled out by 18 September.

The second bug was "a misconfiguration to the Claude API TPU servers that caused an error during token generation." A model writes its answer one token at a time, and a runtime performance optimisation "occasionally assigned a high probability to tokens that should rarely be produced given the context."

The result was Thai or Chinese characters in answers to English prompts, and obvious syntax errors in code. It affected Opus 4.1 and Opus 4 from 25 to 28 August, and Sonnet 4 until 2 September, when the change was rolled back. "Third-party platforms were not affected by this issue." Anthropic has since "added detection tests for unexpected character outputs to our deployment process."

To choose its next token, the model ranks candidates by probability, and on TPUs that used approximate top-k, a faster shortcut.

In December 2024, Anthropic found that its "TPU implementation would occasionally drop the most probable token when temperature was zero," and deployed a workaround. In August it deployed code to improve how Claude selects tokens. In the post-mortem's words: "Our fix removed the December workaround because we believed we'd solved the root cause."

The change "inadvertently triggered a latent bug in the XLA:TPU compiler." The approximation "sometimes returned completely wrong results, but only for certain batch sizes and model configurations." "The December workaround had been inadvertently masking this problem." And "The same prompt might work perfectly on one request and fail on the next."

It was confirmed on Haiku 3.5, rolled back on 4 September. Anthropic believes it "could have impacted a subset of Sonnet 4 and Opus 3 on the Claude API." Opus 3 was rolled back on 12 September, after user reports "compatible with this bug." On Sonnet 4, in Anthropic's words, "we were unable to reproduce this bug on Sonnet 4 but decided to also roll it back out of an abundance of caution." Third-party platforms were not affected.

The fix was to drop the shortcut: exact top-k, and some additional operations standardised on fp32. "Model quality is non-negotiable, so we accepted the minor efficiency impact."

The checks passed. "Our validation process ordinarily relies on benchmarks alongside safety evaluations and performance metrics. Engineering teams perform spot checks and deploy to small 'canary' groups first." And the evaluations "simply didn't capture the degradation users were reporting, in part because Claude often recovers well from isolated mistakes."

The symptoms overlapped. "Each bug produced different symptoms on different platforms at different rates." Taken together, "It looked like random, inconsistent degradation."

The timing hid the cause. "When negative reports spiked on August 29, we didn't immediately make the connection to an otherwise standard load balancing change."

The door was shut. "Our internal privacy and security controls limit how and when engineers can access user interactions with Claude, in particular when those interactions are not reported to us as feedback. This protects user privacy but prevents engineers from examining the problematic interactions needed to identify or reproduce bugs."

That control is correct to have. It is also why the people holding the evidence were the users, whose reports, Anthropic says, "all helped us isolate the issues."

"To state it plainly: We never reduce model quality due to demand, time of day, or server load. The problems our users reported were due to infrastructure bugs alone."

That is Anthropic's statement, and the document is the evidence it offers. It is linked below for anyone who wants to read it and decide.

Three commitments, under the post-mortem's own headings. More sensitive evaluations, able to tell a working implementation from a broken one. Quality evaluations in more places: "we will run them continuously on true production systems." Faster debugging tooling, to "better debug community-sourced feedback without sacrificing user privacy."

It also asked users to keep reporting, through the /bug command in Claude Code or the thumbs down in the apps.

Up is not right. Every one of these bugs left the service running. If your monitoring measures whether the system answers, and not whether the answer is correct, a quality failure is invisible to it.

Small changes stack. A routine load balancing change took a bug touching approximately 0.8% of Sonnet 4 requests to one touching 16% in the worst hour. Before you call a new change clean, find out what was already broken.

Workarounds hide things. Removing one can uncover whatever it was quietly covering. Treat its removal as a change in its own right.

Corrections are welcome, and any made are listed, dated, at the end of this article.

Why was Claude giving worse answers in August 2025?

Anthropic's post-mortem attributes it to three infrastructure bugs: some Sonnet 4 requests routed to servers configured for a 1M token context window, a misconfiguration on the Claude API's TPU servers that corrupted output, and a latent bug in the XLA:TPU compiler triggered by a change to how Claude selects tokens.

Did Anthropic reduce Claude's quality to save money?

Anthropic's post-mortem says: "To state it plainly: We never reduce model quality due to demand, time of day, or server load. The problems our users reported were due to infrastructure bugs alone." The document sets out the three bugs as its evidence.

What does 30% of Claude Code users mean in the Anthropic post-mortem?

Approximately 30% of Claude Code users who made requests during the period had at least one message routed to the wrong server type. It is a share of users, not of requests, and it does not combine with the 16% figure, which is of Sonnet 4 requests in the single worst hour.

Why did it take Anthropic weeks to find the Claude bugs?

The post-mortem says its evaluations did not capture the degradation users reported, each bug produced different symptoms on different platforms at different rates, the 29 August spike was not immediately linked to a standard load balancing change, and privacy controls limited engineers' access to the affected conversations.

Was Claude on Amazon Bedrock or Google Vertex AI affected?

Only by the routing bug, and far less: misrouted traffic peaked at 0.18% of all Sonnet 4 requests on Bedrock, and affected less than 0.0004% of requests on Vertex AI between 27 August and 16 September. The other two bugs did not affect third-party platforms.