ai newsFr., 30. Okt.
EilmeldungXREAL AURA Launches With Android XR, Gemini and a Wired Compute Puck

Claude Code vs Aider: Terminal Agent Benchmarks on Real 500k-LOC Codebases

Claude Code vs Aider: Terminal Agent Benchmarks on Real 500k-LOC Codebases Executive Summary & Decision Matrix We ran both agents against three proprietary codebases (a 512k-LOC TypeScript monorepo, a 487k-LOC Go service fleet, and a 533k-LOC legacy Java Spring application) across 120 scripted task

dev.toFreitag 00:29 · 30.10.6 Min. Lesezeit
Bild: dev.to

We ran both agents against three proprietary codebases (a 512k-LOC TypeScript monorepo, a 487k-LOC Go service fleet, and a 533k-LOC legacy Java Spring application) across 120 scripted tasks: cross-file refactors, test generation, dependency upgrades, and bug localization from stack traces. Each task ran three times; we report medians.

Decision Matrix in One Sentence: If your agents must autonomously navigate a monorepo and you pay per outcome rather than per token, run Claude Code; if you pair-program in tight loops, commit via git conventions, or must route to local/cheap models, run Aider — and both can be provisioned in under five minutes via the links at the end of this review.

The performance gap traces to one design decision: how each tool builds its map of the repository before touching a file.

flowchart TB subgraph CC["Claude Code: agentic, tool-driven"] U1[User prompt] --> O[Orchestrator loop<br/>single event loop] O --> T1[Glob / Grep tools<br/>ripgrep subprocess] O --> T2[Read tool<br/>paged, 2k-line windows] O --> T3[Edit / Write<br/>string-replace w/ validation] O --> T4[Bash tool<br/>persistent shell session] T1 & T2 & T4 --> C1[Context window<br/>~200k tokens, auto-compact at 92%] C1 --> LLM1[Claude API] LLM1 --> O end subgraph AI["Aider: map-first, diff-driven"] U2[User prompt] --> RM[Repo map builder<br/>tree-sitter AST + PageRank<br/>graph ranking of symbols] RM --> P[Prompt pack:<br/>system + map + chat history] P --> LLM2[Any chat LLM:<br/>Anthropic/OpenAI/Ollama] LLM2 --> D[Unified diff / search-replace blocks] D --> V[Diff validator + linter hook] V --> G[Auto git commit<br/>attribution in trailer] end Claude Code is a closed agentic loop. It does not precompute repository knowledge; it searches on demand — globbing, grepping, reading files into a sliding context window, and compacting when the window fills. This is why it wins on cross-file tasks: it can chase a call chain six files deep mid-task, pulling exactly the definitions it needs.

The cost is token burn. Our instrumentation showed a median 148k input tokens per task on the monorepo, dominated by repeated file re-reads after each compaction cycle. Auto-compact triggers at roughly 92% window utilization and costs a summarization pass of its own — on the Java codebase, 9% of tasks hit two compactions, adding ~90 seconds and ~40k tokens each.

Aider inverts this. A tree-sitter pass extracts symbols (functions, classes, methods) from every tracked file, builds a graph, and ranks it with a PageRank variant to emit a repo-map capped at a configurable token budget (default 1k, tune to 4k for large repos). The map is built once at startup (0.4s cold, ~11ms incremental on file change via watchman-style caching) and shipped with every prompt.

The failure mode: on a 500k-LOC repo, a 1k-token map can only surface the top ~120 ranked symbols. Tasks referencing peripheral code — a migration script, a rarely-used factory — fail not because the model is weaker but because the relevant symbol never made the map. Aider's --map-tokens 4096 flag is mandatory, not optional, at this scale; with it, success rate on our suite rose from 71.4% to 79.2%.

Edit mechanics diverge sharply. Aider applies LLM-emitted search/replace blocks and verifies each against the current file content; a failed match triggers one re-ask cycle, then a hard stop. Claude Code validates string-uniqueness on replace and falls back to writing full files. Concurrency-wise, both are single-threaded orchestrators — neither parallelizes tool calls across files by default — so wall-clock is dominated by sequential API round-trips: Claude Code at ~2.8s per round-trip (thinking-enabled models), Aider ~1.1s (non-reasoning requests). That latency difference, multiplied by an average 14 vs 6 round-trips per task, explains the p50 gap almost exactly.

Rate-limit mechanics matter at org scale. Claude Code consumes Anthropic's ITPS (input tokens per minute) and OTPS budgets with bursty 40k+ token requests; a 12-developer team with parallel headless sessions in CI will hit org TPM caps and queue. Aider's requests are smaller (2–6k tokens) and survive aggressive shared-key throttling; it also tolerates any OpenAI-compatible endpoint, which lets you front it with a local LiteLLM proxy for key rotation and cost caps.

Harness: Dell Precision 7960 (64-core Threadripper PRO, 256GB ECC), tasks driven by a Python runner issuing prompts over each tool's headless/scripting interface, token counts from provider billing logs, diff validation via git diff --stat and human review of the 120-task output set.

The dependency-upgrade row is the tell: it requires discovering all call sites of a changed API across 400+ files. Claude Code's grep-and-chase loop finds them; Aider's ranked map omits low-PageRank call sites, and the resulting diffs failed compile 9 of 40 runs.

Egress is negligible for both (~2–6 GB/mo of prompt traffic per heavy team) — the real cost line is tokens, and Aider's map-first architecture cuts them 2.4x at the price of recall on peripheral code.

1. Claude Code context compaction amnesia. On tasks exceeding one window (our 240k-token dependency upgrades), auto-compact summarizes earlier file reads. The model then re-reads files — billing you twice — or worse, edits against a stale summarized understanding, producing diffs that reference renamed-but-not-yet-renamed symbols. Debug signature: git diff hunks that reference identifiers absent from HEAD. Mitigation: split tasks under ~100k tokens, or use the SDK headless mode with explicit --continue checkpointing.

2. Aider's auto-commit polluting git history. Default behavior commits every agent edit with aider attribution in the trailer. On a 15-dev team this generated 60+ junk commits/day, broke our git bisect workflows, and tripped branch-protection checks requiring linear history. Set --no-auto-commits and --no-checkpoint-use-git in aider.conf.yml — checkpoints otherwise create hidden commits on a shadow branch. Teams have also accidentally leaked.env contents by having Aider add files to the chat context ( /add ) that then get staged.

3. The silent bill multiplier (both tools). Claude Code hooks and MCP servers that run on every tool call can silently multiply API spend: our test org attached a linting MCP server that added 8k tokens per call — a 31% bill increase visible only as a flattened usage graph in the admin console. On Aider, pointing OPENAI_API_BASE at an OpenAI-compatible proxy that strips system prompts (some cheap routers do) causes search/replace blocks to fail validation in a retry loop: we measured a run burning $4.20 in 11 minutes retrying malformed diffs against a proxy that truncated the format spec. Cap with --max-chat-history-tokens and hard budget guards at the proxy.

4. Race condition on shared worktrees. Both agents editing the same checkout concurrently (a CI misconfiguration we reproduced) corrupt state: Claude Code's Edit tool validates against a cached read that another process has already mutated, emitting "file has been modified since read" aborts mid-hunk; Aider's diff-apply can apply a block into the wrong location if line numbers shifted between prompt and apply. Run one agent per worktree, enforced via git worktree isolation.

Buy Claude Code if your bottleneck is agent autonomy — cross-file refactors, large dependency migrations, headless CI agents that must succeed unsupervised. Its 86.7% vs 71.4% success delta on 500k-LOC codebases translates directly into engineer hours: at 2,000 tasks/month, the ~15-point gap is roughly 300 failed tasks you no longer manually finish, worth more than the seat cost at any tier above Startup. Rating: 4.8/5.

Buy Aider if you want token discipline, model portability, and git-native workflows — or if compliance forces on-prem inference. At $1.85 vs $3.90 per merged PR and the only credible path to local-model operation, it is the infrastructure engineer's tool. Rating: 4.5/5.

Run both. The dominant pattern in 2026 teams we surveyed: Aider for in-loop pair programming and cheap triage, Claude Code headless for the heavy cross-file jobs. Both offer free trials or open-source installs — provision Claude Code here (affiliate link) and grab Aider's setup guide here (affiliate link); a working agent on your real repo beats any benchmark table.

Quelle: dev.to (dev.to). Original öffnen