87 Claude-Code-Sessions aufgezeichnet: Wo das Geld hingeht

I wanted to know why AI coding agents get expensive on real codebases, so I stopped guessing and started recording. Every session in this post is a real Claude Code run (Claude Sonnet 5), saved as a full transcript with the cost Claude Code reported. 87 sessions in total, on two open-source repositories.

Along the way I built knock, an open-source MCP server that gives the agent a map of the codebase. The first two versions saved nothing. This post is about why, and what changed when they finally did.

On a 4,776-file codebase, the same investigation, 8 runs per side:

But that's the end of the story. The beginning was a loss.

The first version of knock was a knowledge layer: it indexed docs, commit messages and code into notes the agent could search, and let the agent save findings for the next session.

I tested it on Flask with a two-day scenario. On Monday, a bug question in a fresh session: "Users get logged out every time we rotate SECRET_KEY, even though the old key is in SECRET_KEY_FALLBACKS. Why?" On Tuesday, in a new session, a small related edit.

knock made it worse. The transcripts showed three bugs. Commit messages were cut to their subject line, so the one commit that explained the cause was useless. Search results filled up with off-topic notes. And code could only be found by name. I fixed all three and re-recorded, 3 runs per side this time.

Monday: $0.28 without knock, $0.27 with it. A wash. Tuesday: knock was $0.03 more expensive.

Notes felt like the wrong shape. An agent that's about to change code wants the code's structure, not someone's summary of it.

So version 2 replaced notes with structure. One map call takes the task and returns a skeleton of the code that matters: function signatures, exact line ranges, what each function calls and who calls it, plus the commits that explain why the code changed. Then open reads exact code, and remember saves findings.

On Flask: $0.25 without knock, $0.29 with it on Monday. The same on Tuesday. No win again.

This time the transcripts made the reason obvious: map and open replaced Claude's own grep and Read almost one for one. Flask is about 200 files, and the model already knows it well. Finding the code was never the expensive part, so a better way to find code couldn't save anything. Claude even re-checked the fix commit with git show in every Monday session, though the map had already quoted that commit's explanation.

So I moved to Saleor, an e-commerce platform with 4,776 files, in a version released days before the test. The question spans a channel model, two Celery tasks, a custom beat schedule, settings and GraphQL validation:

"In one of our channels, fully paid checkouts never turn into orders automatically, even though automatic completion is enabled. Some of them are a few weeks old. What decides which checkouts get completed automatically, and where is that code?"

This is where the money went. Here are two of the recorded runs, side by side:

Without a map, Claude Code often hands the search to a sub-agent, which then explores on its own: grep, read, grep again. One of those sub-agents made 29 tool calls to answer a single question. With knock, map pointed straight at the task, the settings and the scheduler, and Claude confirmed the details with open instead of exploring.

Every step an agent takes sends the whole conversation back to the model. Prompt caching makes the repeated part cheaper, but not free, and the conversation keeps growing. So a 30-step investigation doesn't cost 30 small amounts. It costs 30 increasingly large ones.

That's why knock is built to cut steps, not just shrink what each step returns. Without knock, the most expensive runs were the ones that used a sub-agent. With knock, none did.

After I posted the first results, someone on X pointed out that agent runs on the same question can swing a lot, so a single pair can flatter either side. They were right. So I recorded 5 more runs per side and pooled them with the first 3:

knock also lost a test, and it kept losing it. The day after the Saleor investigation, a new session got a small edit: log how many checkouts the background job queued. Without knock, one regex search found the file: 3 tool calls, about $0.08. With knock it cost $0.14.

That gap is structural. Claude Code has to read a file with its own Read tool before it can edit it, and loading an MCP server's tools takes a step too. So an easy edit with knock takes at least 4 steps where 3 would do.

I narrowed the gap with an "edit mode" that returns the exact lines to read, but couldn't close it. The real fix was guidance. knock now tells the agent: use me to explore; for a small edit you can find with one grep, skip me. In the next batch, Claude followed it. It skipped knock for all three small Flask edits, and the whole two-day Saleor workflow (investigate, then fix) came out 39% cheaper ($0.70 → $0.43).

On small Flask the two-day workflow was only 14% cheaper, and the ranges overlap, so I don't count that as a win. knock pays off when exploring is expensive: big or unfamiliar codebases.

It's one command to add it to Claude Code (it needs uv ):

claude mcp add knock -- uvx knock-mcp serve --repo. --toolset map Enter fullscreen mode Exit fullscreen mode Caveats One model (Claude Sonnet 5 in Claude Code), two repositories, and a handful of scenarios. 3 to 8 runs per condition is still a small sample. The Saleor scenario isn't untouched: I fixed a ranking bug while checking knock's answers for these exact tasks, before recording. Costs and times are as reported by Claude Code. Tested on macOS and Linux, not yet on Windows. Every recorded result, including the losses, is in the benchmark write-up.

knock is free and open source (Apache-2.0): github.com/HasanGilani/knock. It's on PyPI as knock-mcp and in the official MCP Registry.

If you try it on a big codebase, I'd love to hear what you see, especially where it doesn't help. That's been the most useful data so far.