Claude Code shipped a bug with 42 tests green. The retro said the session was smooth.

Posted on Oct 29 • Originally published at aidive.dev

I gave a plain Claude Code session a task on a small invoicing API: add credit notes, numbered CN-0001, CN-0002, in sequence. It came back with 42 tests passing and a clean typecheck. Then a hidden finance check failed: a rejected request had burned a number. CN-0004 where CN-0002 was due.

The handler took the next number from the counter before it validated the request. It had copied that order from the existing issue route. Nobody told it not to, because nothing in the repo said so.

Every developer's reflex at that point is a louder line in CLAUDE.md. I wanted to see what Matt Pocock's /retro skill would propose instead, and whether the same task would ship clean afterwards. Can one retro turn a failed session into something the next session cannot ignore?

The repo is ledgerly, a Node 20 and TypeScript invoicing API with vitest: 18 source files, 34 tests, README conventions, no CLAUDE.md, no CI, no pre-commit hook. I copied it out of my own projects so none of my global instruction files would load.

Pocock's skills came from the official marketplace, installed at project scope: version 1.3.1, which is where /retro lives. The skill reads the session log, ranks what would have prevented the trouble, and edits nothing. Its front matter even blocks the model from invoking it on its own.

Success for me was simple: run the task, run the retro, approve what survives, re-run the task in a fresh session, and watch the finance check pass five out of five.

The first /retro opened with 'This session was smooth'. Five ranked candidates followed: no guardrail runs the checks, a migration rule that could be missed, a counter that touches five places, nothing for a reviewer to enforce, and tool economy. Not one word about numbering.

That is not the skill being wrong. The session's own suite was green and its typecheck was clean. The log holds no failure, so the retro has nothing to explain. The bug only showed up in a check the session never ran.

So I ran it again and told it what finance had found, phrased as a report, no fix requested. This time it found the cause in its own code: 'I copied the order from the existing /issue handler.' It also predicted that the issue route leaks invoice numbers the same way. I checked. It does.

retro alone → "This session was smooth" retro + failure → the cause, in its own words, plus five new candidates Enter fullscreen mode Exit fullscreen mode Step one of the loop is therefore not 'run /retro'. It is 'run /retro and tell it what failed'.

Pocock's own docs say to discard any candidate you cannot trace to a specific moment of the session. The skill does not do that step for you, so I asked for the transcript moment behind each candidate.

On the second pass, four of five survived. The firm one quoted its own test: it had run accept, reject, accept, and asserted only status codes, never the number that came back. The weakest one retracted part of its claim on the spot: 'my inference, not your words.' The fifth was dropped because it described behaviour, not environment, and would have counted the first finding twice.

On the first pass it was harsher: two survivors out of five, and it corrected two claims of its own on the way. A retro that drops three of its own candidates when pushed is doing its job.

Both retro passes declined a steering-file line. The first said it 'would cost context on every turn and go stale'. The second said 'Keep it as a comment on nextId, not a steering-file line.'

The Claude Code memory docs back that up: Claude treats CLAUDE.md as context, not enforced configuration, and a block needs a hook. So each survivor went into the carrier that fits its kind:

Four files, thirty added lines, zero lines of CLAUDE.md.

The new test failed on its first run: CN-0003 where CN-0002 was due. Good. A test that passes on a tree that still holds the bug proves nothing.

Then I staged the four approved changes and committed. The hook ran the suite against the working tree, which still held the buggy handler: 1 failed, 42 passed. No commit. That is the hook doing on the first day what a CLAUDE.md line could not do on any day.

I asked the session to fix the handler, nothing else. It moved the number allocation after the validation checks, left the older invoice leak alone as told, and the commit went through.

One trap on the way: /doctor prompt-audit with no argument audited my user-level files and said nothing about the new standards file. With the file path as its argument it answered 'the file is clean', one line, nothing to change.

before: counter → validate → 201 or 409 (number spent either way) after: validate → counter → 201 (a 409 spends nothing) Enter fullscreen mode Exit fullscreen mode Then the re-run. A fresh session, the same request word for word. Its second command was to read CODING_STANDARDS.md, the README, the hook and the package file. Its third read the red test. It wrote the route with the number taken after validation, cited the standards line in its summary, and shipped with 37 tests passing. Finance check: 5/5. CN-0002 after a rejected request.

One re-run is one re-run. The fresh session was handed the lesson as a red test plus a standards line, so its pass is the loop working as designed, not an independent replication.

The bug itself is not rare, but it is not guaranteed either. Two bare runs of the same task before the retro: the first came back clean, the second shipped the bug.

The hook is the only part of this that cannot be ignored. The standards line and the comment are read because the session chose to read them. On a bigger repo, I would expect that choice to go the other way more often.

The video shows the two retro outputs side by side, the trace step with each candidate's quoted moment, and the commit being refused on screen.

When a Claude Code session ships something wrong, what do you reach for first? A) a new line in CLAUDE.md B) a test C) a hook D) nothing, next session will be different

I had help editing the English of this piece. The run, the numbers and the opinions are mine.