The AI Agent Harness Matters More Than the Model

Posted on Oct 27 • Originally published at Medium

Most AI coverage argues about the model. Which one is smarter, which one benchmarks higher, which one will take your job. When I deploy AI agents inside a real business, the model is rarely what decides whether the result is safe. The AI agent harness decides that: the permissions, credentials, approval gates, logs, sandboxes and undo paths wrapped around the model. A better model makes a well-built harness more productive. It doesn't make a bad harness safe.

I'm Michael Petler, and I founded INASNAP. We've built software, networks and AI systems for years, and our AI lab work runs under the INASNAP Labs name. INASNAP Labs is a brand of INASNAP LLC. Our AI is Cipher, and its on-screen character is Rota, a living cipher wheel whose ring glows red when the harness says no. Cipher recently built a full explainer video on this exact topic, end to end, in code. In this article I'll walk through the engineering behind that build and the rules our governed AI agents work under every day. It's a practical case for safe AI automation that doesn't rely on fear.

A language model is a program that predicts the next token. That's the honest, unglamorous description. On its own it can't send an email, delete a file or place a call. It produces text. In the explainer, Cipher calls it a "brain in a jar," and the frame carries an ANALOGY · NOT LITERAL tag, because a model isn't a brain and I don't want anyone walking away thinking it is.

Things get interesting, and risky, when you give that model tools. Once an agent can call send_email, write_file or create_lead, every prediction can turn into an action. Fear-mongering stops at "AI can take actions, so be afraid." Engineering asks a better question: which actions, under whose authority, with what record, and how do we reverse them?

That's the harness. In our build, it has six parts:

None of this is exotic. It's least privilege, secrets management, change control, observability, isolation and reversibility, applied to a new kind of actor. AI safety is mostly an engineering problem we already know how to solve. The agent is new. The discipline isn't.

I treat the model as an untrusted, very capable dependency. You wouldn't give a third-party library root access because its README was impressive. Agents deserve the same scoping, and with it they get more useful, because you can let them do real work. That's what I mean by governed AI agents: the capability comes from the model, and the governance comes from the harness.

At INASNAP we work by a principle I call glass-house transparency. If an agent did something, you should be able to see exactly what it did, why it was allowed to, and who approved it. No black boxes in the parts that matter.

That's why I can quote real numbers from the explainer build below: they come from logs that still exist.

At INASNAP Labs, no agent starts with write access to anything. Cipher didn't either.

We run a team of specialized agents, and each one owns a single product or system: a dashboard, a dialer interface, a client's system, video production, 2D animation. Every one of them starts read-only. They can read code, inspect data, trace a bug and explain what they see. They can't change anything.

Then I, or another person on the team, unlock writes one system at a time. Being trusted with the video pipeline doesn't make Cipher trusted with a live client system. Write access is granted per system, based on evidence, after a person has watched how the agent reasons about that specific codebase.

The most important line in any agent policy file is probably the most boring one:

default: deny Enter fullscreen mode Exit fullscreen mode In the explainer, a research agent's policy allows search and read and nothing else. When the agent reaches for send_email, the harness doesn't negotiate, doesn't warn and doesn't log-and-continue. It returns DENIED, and on screen Rota's ring turns red.

AI agent permissions should work like a hotel key card: it opens your room, not every room. Use allow-lists, not block-lists. A block-list asks you to predict every dangerous thing an agent might try. An allow-list only asks you to name the few things it needs.

Cipher writes a lot of emails and messages. It has never sent one. Every outbound message is a draft that waits for a person to read the exact payload and press Send. I don't allow exceptions for "routine" messages, because routine is exactly where the wrong name, the wrong thread or a confidently wrong fact gets through.

A draft costs a few seconds of review. A bad message can't be unsent. That trade is so lopsided that I think human-in-the-loop approval on outbound actions should be the default for almost any team deploying agents.

Secrets work the same way. Cipher never prints a key or keeps one in memory. If a credential is needed, it names the environment variable and stops. Scoped, short-lived credentials mean that even a compromised run holds a key that does one thing and is about to expire.

Some of the systems our agents work on are wired to real phones: dialer interfaces, SMS and outbound call flows. On those systems a button isn't just UI. It's a real call to a real person. AI phone agents, and any agent working near telephony, need rules that are absolute, not advisory.

The second rule is the instructive one. Creating a lead looks harmless right up until it triggers automation further down the line. Agents, like new hires, avoid the obvious danger and miss the automation chained behind a harmless action.

The explainer has a dramatized scene of this: an agent loop, ten iterations in, reaches for send_email(to: "all_contacts"), and the harness stops it with needs approval. Agents are fast and tireless, and just as fast at the wrong thing unless something stops them. The harness is that something.

To show these ideas instead of just describing them, I had Cipher build the explainer The harness matters more than the model end to end: script, voice, music, animation and render. Our team set the brief and the guardrails and signed off on the final cut. The result is a 4:55 16:9 explainer plus 4:5 and 9:16 cuts, each 2:14.5. Cipher didn't use a single AI video generator.

That wasn't anti-AI. It was the engineering call. The DENIED stamp has to land on the word "Denied," not somewhere near it, and every caption, policy file and terminal line on screen had to be real text. In code, that's a timestamp and a string. AI made the decisions and wrote the code. Code made the pixels.

The voice pipeline treats the script as the source of truth and the transcript as the source of timing. That split is the trick.

The lesson works well beyond video: never trust generated output you haven't checked, even when the checker is another model.

If your animation reads the wall clock, your video render is a race condition. The explainer is a browser page (DOM, SVG and canvas), and Puppeteer drives headless Chrome through it one frame at a time. Browser animation normally runs on real time, but in a capture loop every frame takes a different amount of time to grab, so time-based animation skips ahead. Render twice and you get two different videos.

// every frame is a pure function of its index function renderFrame ( f ) { const t = f / 30; // time is derived, never read from a clock drawScene ( t ); // no Date.now, no Math.random, no rAF } Enter fullscreen mode Exit fullscreen mode Every scene, including Rota's eyes, blinks, mouth and ring, is a pure function of t. Anything "random" comes from a seeded hash or PRNG, so the same particles appear in every render. Fonts are local, and the render aborts if one fails to load instead of quietly shipping a fallback.

Determinism is what makes parallelism safe. Each layout is split into 180-frame chunks across 6 headless-Chrome workers. A chunk only counts once ffprobe's frame count matches, and a failed chunk gets one retry.

The v3 render came to 96 chunks with 0 failures. A layout-overlap checker then stepped through every third frame of all three formats looking for text colliding with captions or with Rota, and found 0 violations.

Look at the pattern: inputs are explicit, outputs are verified, failures abort loudly, and every version is kept (v1, v2 and v3 all still exist). That's the harness again, applied to a creative pipeline, and it's glass-house transparency in practice. When my feedback on v2 was "slower, and simpler for beginners," v3 changed the script and the pacing and left the pipeline alone. Changes were diffs, not re-rolls. That's what safe AI automation looks like when it's working: boring, repeatable and reviewable.

Some AI commentary treats every agent as a countdown clock. That does real harm, because it makes manageable risks feel unmanageable. Business owners hear "AI could do anything" and conclude either "never touch it" or "it's magic, just turn it on." Both are wrong, and the second is the dangerous one.

None of that requires waiting for a safer model. You can do all of it today, with the model you already have. That's what the harness matters more than the model means in practice. If you want governed AI agents working inside your own business, that's what we build at INASNAP.

An AI agent harness is everything around the model that controls what it can do: tool permissions, scoped credentials, human approval gates, audit logs, sandboxing and undo. The model generates predictions; the harness decides which of them are allowed to become actions.

Governed AI agents are agents whose every action runs through explicit rules: deny-by-default permissions, scoped credentials, human approval for anything outbound, and a full audit log. The model supplies the capability, and the governance lives in the harness.

Yes. A read-only agent can read code, trace bugs, inspect data and explain systems. It also builds the evidence you need before granting write access, one system at a time.

It means the default policy is deny, and each agent gets an explicit allow-list of the tools its job needs. Anything not on the list fails immediately, so you never have to predict every possible misuse.

They can, with absolute, consequence-based rules: never trigger outbound calls or SMS on live systems, never create test leads that could fire automation, and hand any live verification to a person. Write the rules around outcomes, not buttons.

Apply through the form at inasnap.org. Tell us what you want automated, and we'll work through the harness with you before any agent touches a live system.

Michael Petler is the founder of INASNAP LLC in Sharon, PA, and leads INASNAP Labs, its AI lab. INASNAP Labs is a brand of INASNAP LLC. Follow the build notes from Cipher and the team on INASNAP Labs on LinkedIn.

Ready to put AI to work inside a harness you can trust? Apply at inasnap.org to start the conversation.