My AI-written plan went through every check I could think of. The gaps were in what I never said.

The plan was 22,000 words long. Twelve agents had worked on it for close to two hours. Among them, three architects had drafted it, two judges had scored the drafts, a synthesis had merged them, and three reviewers running on a different model had tried to break the result. They raised 51 findings. The plan accepted all 51. Its claims about our code came with file and line references.

I read it the next morning, before work, and asked two questions. Neither had an answer in it. Both were about what the product has to become, and I had never spelled out either of them.

One line of context: I run engineering at GoodBarber, an app platform, and I needed an architecture plan for a large change to one of our production systems, delivered in steps. I'm not going to say which system, or describe the plan. This post is about how the plan was produced, where the method failed, and what I'd change.

I started in Claude Code, on its regular model, and asked which model should write the plan: itself, or the costly one, the most capable model we can use.

It picked the costly model for the plan, and kept itself for later: executing milestones once they are cut small, and reviewing. The reasons were worth keeping. A migration plan has to account for every path the current system takes, and a plan that forgets one recreates bugs you've already paid for; that's where the gap between the two models is widest. The costly model is 2.5 times the price per token, which is noise on a one-off plan that will steer weeks of work. And it's slow, which hurts every day and doesn't matter once.

It added one rule I would have forgotten: the plan's author should not review its own plan.

So the roles were set. The regular model interviews me, maps the code, writes the brief and runs the orchestration. The costly model plans. The regular model, as a different model, attacks the plan.

While it mapped the code, the regular model described to me what one of our neighbouring services does. I typed: "Don't trust the docs. Trust only the code."

Its reply was that this applied to what it had just told me. It had identified that service from folder names, URL routes and commit messages, and described another part of the system from a docstring. Leads, not proof. It re-checked those parts from executable code, and the sentence became a rule for every agent that followed. In the planning runs, it read like this (translated):

- Read only: you modify, create and delete nothing. You run no server, no migration, no script that touches a database or the network. - Only executable code is evidence (cite file:line). Markdown files, docstrings, comments, commit messages and folder names are leads to verify. - You can't ask the owner questions during this work: every uncertainty becomes an open question, with the assumption you made. Enter fullscreen mode Exit fullscreen mode The mapping ran as three parallel passes of read-only reader agents, each returning claims with their evidence. Two more things went into the brief, and I'd do both again.

My exact words, in a file. The regular model wrote down everything I had asked, verbatim and in order, including my answers to its clarifying questions. The brief is a reformulation, and reformulations drift. The file is what the brief gets checked against.

A read-back before launch. I told it I couldn't afford a misunderstanding. It answered with twelve numbered points, "here is what I understood", and four questions about ambiguities that would each change the plan. I answered the four. It then wrote the brief, about 2,500 words, launched the run, and sent me the brief to read while the costly model worked: if I spotted a misunderstanding, it would stop the run and relaunch with the fix.

const plans = await parallel ( ANGLES. map ( a => () => agent ( ` ${ RULES } \n ${ READ_BRIEF } \nYou are architect ${ a. key }. Your starting angle: ${ a. title }. ` + `The angle is a starting point, not a limit: your plan covers the whole deliverable.`, { label: `architect: ${ a. key } `, model: COSTLY_MODEL, effort: ' xhigh ' }))) Enter fullscreen mode Exit fullscreen mode The whole run:

The result came back truncated the first time, and the regular model rebuilt it from the agent's transcript: 13 milestones and 32 open questions, each with the costly model's working assumption, since nobody inside a workflow can stop and ask me.

Two details from the run stayed with me.

The judges didn't agree. One picked the security-first plan, the other the safest-migration plan. Adding up each judge's six marks, the top two plans were one point apart, and neither judge picked the plan on top of its own total. The lens seems to have weighed more than the marks. The synthesis took the safest-migration plan as its base and grafted ideas from the other two.

The judges spot-checked the architects' references. They ran 71 checks on file:line claims, a few of them on the same claim from both judges. 58 held exactly as written. The other 13 came back qualified, from "confirmed, but a few lines off" to plain wrong. The worst kind was a field cited at a precise file and line that doesn't exist: something the plan proposed to add, written up as if it were already there. That's nearly one check in five needing a correction, on plans written by the most capable model we can use, under explicit instructions to cite only code it had read. I'd keep the spot-check.

Back to the morning after. I won't say what my two questions were. What matters here is that the plan had answered both implicitly, with how the code works today.

I asked the regular model why it had missed them. Its answer, condensed:

The third point needs a nuance, because the requirements checker did better than that summary. It flagged one requirement nobody had stated: whether the new system must do everything the current one does before a customer is moved to it. That one sat right next to things I had said explicitly. The two that mattered had only been brushed in passing, in a few words the regular model pointed out afterwards, and today's code was built the other way.

That's the part I find most useful. The checker comparing the brief to the code found nothing wrong with those descriptions, because they were accurate. In this run, the fact-check couldn't flag the frame; if anything, it made it look more solid. "This is how it works today" passed review and was read as "this is how it must work".

When the costly model reworked the plan, it wrote its own account of the miss, and went one step further than the regular one. The first plan, it said, had turned the boundaries of the existing system into architectural guarantees. It had even recorded three weak signals in the code that pointed toward the two requirements, as facts, without ever turning them into questions. Its conclusion: a plan that states "here is what I'm taking for granted that you didn't say" would have surfaced them.

The regular model offered to have the costly one write an addendum. I asked for something else: give the costly model the two requirements and one question, does this call the plan into question? If not, fine. If it does, rework it.

The regular model ran that as its own workflow. Two independent assessments by the costly model, then a challenger on the regular model with one instruction: if in doubt, keep the more demanding verdict. All three said the same thing. Not an addition, a rework. The costly model rewrote the plan at maximum effort, to files this time to avoid another truncation. Three checkers attacked it again (44 findings: 3 blocking, 27 major, 14 minor), and the costly model revised.

The third version has a section the first one lacked: the findings it set aside or nuanced, each with a reason. One of them walks back an over-correction, where the intermediate version had taken one of the assessments' strongest claims at face value. I trust a reviser more when it pushes back on its reviewers. In the first run, it had accepted 51 out of 51.

The open questions went from 32 to 69. Fourteen of them were blind spots: things nobody had stated, posed as questions with an assumption each, not slipped in as requirements.

The third version had doubled, to 45,000 words. A list of 69 questions at the end of a document that long doesn't get answered. So I asked the regular model to put them to me instead: one at a time for the ones that would change the architecture, then in small rounds by theme.

For each one it gave me a few lines of context with what the code shows, the costly model's assumption, and its own recommendation if it had one. I could answer, or say "explain", "be more precise", "show me in the code". When I said I wasn't sure how something behaved in another part of our platform, it went and read that code before we moved on. Every answer went into a file next to the plan, in my words.

All 69 took about seventy minutes. Three principles came out of my answers that weren't in the brief, each one drawn out by a specific question: we deploy alongside the current system and destroy nothing; we're changing one part, not rewriting the platform; when the plan depends on another team's work, it states the need and leaves the solution to them.

The costly model folded the answers into a fourth version, under a rule the regular model set for it: a finding never overrides one of my answers. At most, the plan records the risk. The fourth version came back with fourteen new questions of its own, and I answered those too.

Then the plan went to two reviewers from other model families, two lenses each, with access to the code and not only the text. They raised 30 points. The regular model checked each one in the code: 26 held, 4 were discarded.

For the last round I opened a fresh session on the costly model, at maximum effort, with the plan, my answers, the verified review and my exact words. Before any rewrite, I asked for three things: one page on where it stood, including where the reviewers were wrong according to the code; a frank challenge of any of my own answers that created a risk, "I'll decide. Don't work around it in silence"; and its questions, one at a time or in small groups.

Its agents re-checked all 30 points. It agreed with the regular model's verdict every time: the 26 that held, and the 4 discarded. On the proposed fixes, it judged one right, one out of proportion and the other 28 incomplete. Of the 336 citations in the review, 312 held exactly and 20 were a few lines off; 2 weren't checked, and the 2 marked wrong were about a reviewer's wording, not the code. It still contested specific claims in 7 of the findings, with code to back each objection.

Then it contested about a dozen of my answers. The sharpest one: an answer of mine asked for something my own team had ruled out, deliberately, twice, with the reason in the code. I changed most of the answers it challenged. On one I kept my position and made it more precise.

It also worked the other way. Thirty questions in all, most of them in five groups over one evening. On testing, I asked for "as real as possible, as few mocks as possible", and it went back to the code and rewrote its proposal around that. When it leaned toward its own vendor's models, I wrote: no favoritism, we'll be multi-provider. And the one real misunderstanding of that evening was mine: I read one of its proposals as the opposite of what it said and called it a deal breaker. One clarification later, I accepted it.

The costly model also ran out. With 28 agents in parallel, the first verification pass hit my subscription's session limit in about half an hour; it resumed hours later, three agents at a time. Then, overnight, while writing the fifth version, the costly model reached its usage limit.

I could wait until its allowance reset, days later, or let the regular model finish. It kept the eleven parts the costly model had written and reviewed, reviewed the twelfth, wrote the rest with its own agents, two at a time, and assembled one file: about 115,000 words, with a section near the top that ties every change since the fourth version to one of my answers or one review finding. In its summary, it corrected a fact the costly model had given me during the confrontation.

The decision built on it held; its justification changed.

How do you get a model to tell you what it assumed before it hands you the plan, rather than when you find the hole?