ai newsDo., 29. Okt.
EilmeldungAn AI Breakthrough Has Made Decades of Computer Science Research ‘Irrelevant’

Anthropic Says Nobody Needs a Mental Model of Claude Code. Practitioners Are Testing That

Anthropic has published how the Claude Code team reviews code now that Claude writes roughly 90% of it, including a claim that nobody keeps a complete mental model of the codebase. The post documents a pull request that ran more than fifty review rounds overnight, a reviewer spending most of its ef

InfoQDonnerstag 10:58 · 29.10.4 Min. Lesezeit
Bild: InfoQ

The post, by Addy Osmani, Morgan Westlee Lunt and Qing Wang, is the companion to Anthropic’s account of replacing git with an in-house system after its monorepo went from roughly 2,500 merged changes a day in May to more than 30,000 in September. Review was the next thing to break. The target is that most pull requests "merge with at most one human in the loop, who can be the author, with no increase in reverts or incidents."

Nobody on the team keeps a complete mental model of Claude Code in their head, and we don’t think anyone needs to. When someone wants to understand a part of the codebase, they ask Claude to write an explainer for exactly that slice. It takes minutes and reflects the code as it is today.

That passage dominated the Hacker News discussion, where commenter jonahx set out what it trades away:

There is no longer anyone to turn to if this system breaks by, eg, having a critical bug that no one can figure out. No causal model in anyone’s head that can be appealed to, to slowly and methodically debug. The new trust model is: We trust Claude to always understand the system enough to debug anything or make any reasonable change we could want.

He also noted that many large legacy systems already work this way.

One documented case tests the explainer claim on Anthropic’s own repository. In August, Claude Code collaborator Boris Cherny answered a long-running complaint about issue auto-closure in a comment carrying the "Generated with Claude Code" footer. It said issues are closed when they have "had no activity for a while and are missing information needed to reproduce," and that "nothing is triaged solely by counting."

The repository’s issue-lifecycle.ts sets fourteen days to stale and fourteen more to closure, exempting only issues with ten or more upvotes. Developer marcindulak counted more than 6,000 issues labelled as carrying reproduction steps that were closed anyway as "inactive for too long." The remedy the comment offered, commenting to have an issue reopened, is blocked by a workflow that locks closed issues after seven days. That meta-issue is now locked.

The answer that started this issue, and misdiagnosed how issues in this repo are managed, was generated by a world-class model.

The failures the post documents are specific. The first appears only when both sides are machines:

Noise costs more when the author is an agent. A person can wave off a nit in seconds. An agent fixes it, which means another commit and another round of review.

"One CLI PR went through more than fifty rounds of review and fixes overnight before its author stopped it," the team writes. Running more rounds by default made things worse, so they capped them. On r/ClaudeCode, commenter ChemicalExcellent463 reported reaching the same control independently, stopping after two failed fixes because repeated failure means the agent’s picture of the code is wrong rather than the line it keeps editing.

The second failure is what those rounds were finding:

When we ran git blame on what the reviewer kept flagging, about 59% of it was code that had been there since the reviewer’s first pass. The reviewer was rarely finding new bugs, it was finding a few more old ones each round.

What gates a merge is a risk tier picked by a classifier, where "the change sets the tier, not the author." Low risk covers docs, copy, and additive changes behind a flag, and clears with no human approval. Medium risk, including caching, the auto-updater, and system prompts, needs one person. Two rules hold the line: "an agent’s approval never counts as a person’s," and the person who prompted a change "counts as one of the required reviewers."

That boundary drew a question the post does not answer. Reddit commenter Oximy-Naman asked whether low risk covers agent instruction changes shipped as documentation, since those "could change what an agent does without touching application code," and got no reply. The post assigns no tier to CLAUDE.md, the file its own reviewer loads on every review. On compliance, Hacker News commenter gaal_dornick reported that their own SOC 2 reviewer required every change to be seen by a second human before production; the post does not discuss audit requirements.

One control is missing, and the team says so:

We don’t yet have circuit breakers that tighten approval when those numbers move; engineers here want them before more code moves to zero-human merges.

Not everyone reads this as reckless. Commenter cheema33 argued that large organizations have never had anyone holding a complete model, making this an upgrade on the status quo, and commenter markbao reported spending a few thousand dollars a month on AI review against human review that consumes up to a third of an engineer’s time.

Parts of the system ship publicly. Code Review for Claude Code is generally available on Team and Enterprise plans, and the security review runs as an open-source GitHub Action whose README states that it "is not hardened against prompt injection attacks and should only be used to review trusted PRs."

Equal Experts helps organizations harness AI, modernize technology, and build better software through expert engineering, collaboration, and innovative delivery practices. Learn more.

Quelle: InfoQ (InfoQ). Original öffnen