Fire Drills for Your AI Harness 🔥 - Chapter 8 - The only way to know a control works is to break things on purpose

Your AI coding agent has a harness. It has system instructions, tool permissions, validation scripts, automated tests, and guardrails designed to prevent it from doing something stupid. Great.

Now, one question: When was the last time you deliberately tried to break those guardrails? If the answer is "never," you don't know whether your harness works. You know only that you've configured it.

There's a difference. As AI agents take on more of the software development lifecycle, we're moving beyond the question of how to make agents generate code faster. We're facing a harder engineering problem: how do we build development systems that remain reliable when their primary worker can generate an enormous variety of code and behaviours?

The answer isn't to keep adding rules. It's to build a harness that can prove its controls work, detect when they stop working, and retire them when they're no longer useful. Let's look at how to do that.

These layers complement one another. A formatter cannot establish that business logic is correct. Unit tests cannot prevent an agent from accessing a secret it should never have been able to read.

Each control should address a clearly identified risk. Make drift detection automatic There's another important detail: some failures emerge gradually. A workflow that was reliable last month may become less reliable as dependencies change, repository conventions evolve, or new tools are introduced.

If drift detection depends on a developer remembering to investigate, it will inevitably be inconsistent. Instead, schedule periodic checks. Depending on your environment, these might include: • Checking whether required CI jobs are still present. • Detecting changes to protected configuration files. • Identifying outdated or duplicated validation rules. • Monitoring recurring agent failures. • Verifying that permission boundaries still match the intended architecture.

The important property is that the scan runs because it is scheduled, not because somebody has already noticed a problem. Automation turns drift detection from an occasional reaction into a repeatable engineering practice.

The last field is easy to overlook. Controls often become permanent because nobody defines when they should be reconsidered. You don't need to remove a control on a fixed schedule. Some safeguards should remain indefinitely. Others may need regular review. The objective is to make retirement a deliberate engineering decision rather than an operation that never happens. A simple workflow could be: