ai newsDi., 27. Okt.
EilmeldungiOS 27.2: Siri AI expandiert, aber nicht nach Deutschland

AI drafts, humans sign, tools can only say no

Most of the code on our team is written by Claude Code now. Several product lines, one shared database, a small team. Output stopped being the bottleneck a while ago. Accountability didn't get cheaper, though. It got harder. More changes landed than anyone could read, and the person signing off was

dev.toDienstag 08:38 · 27.10.4 Min. Lesezeit
Bild: dev.to

Most of the code on our team is written by Claude Code now. Several product lines, one shared database, a small team. Output stopped being the bottleneck a while ago.

Accountability didn't get cheaper, though. It got harder. More changes landed than anyone could read, and the person signing off was signing for things they had never really seen.

This post is about the rules we ended up with, and the small Claude Code plugin that came out of them. I'm mostly writing it because I want people to poke holes in the reasoning.

When AI-written changes started worrying us, we did the obvious thing: we added more checks. Every AI step got a human gate.

And when people can't keep up, something quiet happens. They start accepting verbal reports as verification. "It's handled" replaces "show me."

Then we had a payment-related incident. Looking back at the two days before it, the pattern was plain: the decision had run ahead of the evidence, and the gate was a person who was already out of attention.

So the chain broke at the human. But the only one who can answer "should this exist at all?" and "where does it stop?" is also the human. The problem wasn't too many humans or too few. It was human attention spent in the wrong places.

What to build. Approve the spec. Approve the code. Write down why. That's the human's whole job. Everything else can be drafted by the AI.

Before anything touches money, schema or a shared contract, the AI "grills" the plan. It lays the plan out as a decision tree and asks the human only the questions that belong to a human:

Implementation-level questions it answers itself, and for each answer it says where it's most likely to be wrong. The human overturns what they disagree with and signs a one-page sheet. Nothing gets built before the signature.

This flips the usual direction. Normally the human reviews the AI. Here the AI also reviews the human, at the one moment it's cheapest to be wrong: before any code exists.

Decisions used to live in chat threads, task cards and memory. Three months later nobody could say why something had been decided, only that it had.

Now every signed plan, every override and its reason goes into a separate Git repo we call the brain. Plain Markdown, dated, append-only. The AI reads it before asking anything, so the thicker it gets, the fewer questions climb up to a human.

Hooks, gates, CI and lint get veto power and nothing else. They can block an edit in a red zone. They can refuse a workflow that forgot to pin a cheap model. They can fail a build. They can never say yes.

That's what keeps the AI from quietly becoming the boss: the only way to get a "yes" is a human signature.

A rule that can't physically be pressed isn't a rule. We designed "the human approves the PR on GitHub," and halfway through realised the AI and the reviewer were using the same account. GitHub doesn't let you approve your own PR. The rule looked great on paper and was impossible in practice.

Rules in documents don't hold. Rules in tools do. We had a written guideline about not fanning out to too many subagents on the expensive model. One afternoon a single session spawned sixty-five of them anyway, most on the most expensive model. The guideline moved into a hook that same week.

I packaged this as a Claude Code plugin called Shipright. You mark "red zones" (defaults: **/migrations/**, **/payment/**, **/billing/** ). When Claude tries to edit one without a signed decision for the current branch, it gets this:

Blocked: migrations/0001_example.py is a red zone (**/migrations/**). No consensus doc covers branch feat/example. Exits: run /shipright:grill, or /shipright:grill --override "<reason>" (recorded in the brain log). Enter fullscreen mode Exit fullscreen mode Every block says what was stopped, why, and how to get out. An override is allowed, but it's written down with a reason and a date.

There's one subtlety. If the gate only checks "does a signed doc exist," the AI can just write the doc itself. So the gate has tiers:

In commit tier the AI also can't run git commit against the brain from Bash. A human has to.

I'd rather list these than have someone find them:

These are the parts I'm least sure about:

If you've built something similar, or tried this and watched it fail, I'd really like to hear how.

The plugin is open source (Apache-2.0): https://github.com/agoopintsai/shipright

Quelle: dev.to (dev.to). Original öffnen