The self-annealing loop: make every AI agent failure leave a test behind

Posted on Oct 21 • Originally published at calyx-ai.net

When an AI coding agent hits an error, the easiest move is to get past it: retry, reset, reprocess. That makes today's run green and guarantees the same failure next week. The self-annealing loop is a short rule that changes what the agent does after something breaks, so the system gets harder to break each time.

Ask a coding agent to finish a job and you have told it, implicitly, that a finished job is the goal. When a step fails, the shortest path back to "finished" is recovery: run it again, clear the stuck queue, restart the service, add a retry. Each of those is reasonable in isolation. Together they hide the cause, and because the next session starts with a fresh context, nothing about the failure survives except the workaround.

A human engineer does the same thing under pressure. The difference is that the human usually remembers. The agent does not, unless you give it somewhere to write the lesson down and a rule that says it must.

Annealing is the metallurgy term for heating and cooling a metal so its internal stresses relax and it ends up tougher. In an agent-run system the "heat" is a failure. The loop is five steps, and the order is the whole point:

Recovery still happens. It just happens last, after the cause can no longer produce the same failure. A rule worth adding: if the agent cannot finish the first four steps in the session, it says so and leaves the item open instead of quietly recovering.

The loop only works if it is part of the agent's standing instructions, not a prompt you remember to type. In a D.O.E. repo (Directive, Orchestration, Execution, described in the first article ) it lives in CLAUDE.md. A condensed example:

## When something fails (self-annealing) 1. Diagnose the root cause. A symptom is not a cause. 2. Fix it in code/config/infra. 3. Add the test that would have caught it. 4. Add the standard to the directive's "Edge cases & learnings" (and a lint if it is mechanically detectable). 5. Only then recover. Never recover first and call it fixed. Record each pass in the anneal log: date, symptom, cause, fix, test. Enter fullscreen mode Exit fullscreen mode The anneal log matters more than it looks. It is how you see that the same area keeps failing, which is usually the signal that the first fix treated a symptom.

We run our store with a daily agent job on a small Linux box. Each day a script checks the storefront and the checkout endpoint, and if either returns nothing (HTTP status 000 ), it raises an ATTENTION note: the agent must diagnose and fix the cause before doing anything else.

Pass 1 was a real fix and still not the whole answer, which is normal. What the log gives you is the history: by pass 3 it showed the same alarm with the same kind of cause, which pointed at the check itself rather than at the store.

The last pass also shows the loop's honest exit: some root causes are not code. When the cause is a flaky network link, the right standard is "classify it correctly and tell a human", not another retry.

You do not need a framework to try this. Add the block above to your CLAUDE.md,.cursorrules or AGENTS.md, and give your most failure-prone job a directive with an Edge cases & learnings section. After a week, read that section. It is a list of failures that now have a test, a rule or a lint against them.

The free Agent Ops Kit Lite (MIT) has the CLAUDE.md with this loop, the directive template and a secrets linter. It also installs as a Claude Code plugin: /plugin marketplace add skibuff22/agent-ops-kit-lite.

Originally published at calyx-ai.net. Disclosure: I sell the full Agent Ops Kit ($19 launch price through November 2, then $29), which adds the self-annealing, bug-fix and agent-governance directives, three example directives, seven execution scripts and multi-agent templates. The free lite kit above stands on its own. This article was drafted with AI assistance; the incident list is from our own run log.