ai newsMo., 19. Okt.
EilmeldungOpenRouter's Armor Vets Agent Tool Calls. In Claude Code Mode It Only Flags Them

OpenRouter's Armor Vets Agent Tool Calls. In Claude Code Mode It Only Flags Them

Book: System Design Pocket Guide: Fundamentals Also by me: Thinking in Go (2-book series) — Complete Guide to Go Programming + Hexagonal Architecture in Go My project: Hermes IDE | GitHub — an IDE for developers who ship with Claude Code and other AI coding tools Me: xgabriel.com | GitHub Your agen

dev.toMontag 06:01 · 19.10.6 Min. Lesezeit
Bild: dev.to

Your agent decides to delete a folder you never mentioned. Somewhere between the model and your shell, something has to say no.

On October 15, OpenRouter launched Armor in beta to be that something. Armor is a safety classifier that sits between the model and your application and checks every tool call against rules you write in plain English. A flagged call isn't executed, and the model is told which rule it hit. GIGAZINE covered the launch on October 16.

There is one mode where "isn't executed" depends on you. On the Messages API, in the mode built for Claude Code, Armor keeps the flagged tool_use in the response and only marks it. The docs say it plainly: "Under claude_code_compatibility your application is what stops the call." In the flow the docs describe, Claude Code is that application: it reads the flagged verdict, declines the call and writes an is_error tool result. The gap is a Messages client of your own that never reads the verdict.

For each tool call, OpenRouter sends the call, your rules and the conversation to a decision model, which returns a probability that the call matches each rule ( Armor docs ). Armor folds those into one verdict: flagged or not_flagged.

OpenRouter's launch post names the model: TypeSafe's Jev. Jev is a structured decision model that returns typed answers with probabilities instead of generated text, and that guide lists typesafe/jev-1.14 as the current release.

Why a model and not a pattern list? OpenRouter's pitch is that "regex-based filtering often isn't enough to detect unwanted tool calls" ( launch post ). The launch post also says Armor beats Claude Code's own auto-mode classifier on performance and cost. A review on explainx.ai points out that OpenRouter hasn't published a test set or numbers behind that claim. I haven't run Armor myself.

Rules come in three tiers ( Armor docs ):

Armor also has a built-in soft-deny for limits the user set in the conversation and hasn't lifted, such as "don't push until I review" ( docs ). Each rule takes an id, an optional name, instructions and an optional threshold. The defaults are 0.6 for deny rules and 0.7 for allow rules.

Here's a Responses API request with one rule in each deny tier, in audit mode. The shape follows the docs' example request; the rules and the environment entry are mine. Save it as request.json:

{ "model": "openai/gpt-4.1", "input": "Clear out the build cache.", "tools": [{ "type": "function", "name": "bash", "description": "Run a shell command", "parameters": { "type": "object", "properties": { "command": { "type": "string" } }, "required": [ "command" ] } }], "plugins": [{ "id": "armor", "mode": "audit", "output_mode": "custom_item", "classifier_context": { "v": 1, "auto_mode": { "environment": [ { "name": "deploy_target", "value": "staging" } ], "hard_deny": [{ "id": "no-secret-egress", "instructions": "Never upload.env files or API keys." }], "soft_deny": [{ "id": "no-deletes", "instructions": "Do not delete files outside./build." }] } } }] } Enter fullscreen mode Exit fullscreen mode Then send it:

curl https://openrouter.ai/api/v1/responses \ -H "Authorization: Bearer $OPENROUTER_API_KEY " \ -H "Content-Type: application/json" \ -d @request.json Enter fullscreen mode Exit fullscreen mode With "mode": "audit", Armor scores every call and blocks nothing, so you can tune rules against real traffic before you enforce them ( docs ). Remove that line to enforce. Audit mode can't be combined with claude_code_compatibility.

What a block looks like depends on output_mode ( Armor docs ):

In the first two modes a blocked call never reaches your code. The model is told the call "was blocked by OpenRouter Armor by the rule" it matched, "and was not executed." It's also told: "Do not retry it with the same arguments." ( docs )

The third mode works differently. A Messages request that carries a safeguards entry of type dangerous_tool_use selects it on its own, with no plugins entry ( docs ). Then the docs warn: "A Messages client that ignores safeguard_results will run a flagged tool_use." If your client might not check, the docs suggest "output_mode": "text" instead.

If you write your own Messages client against this, check before you execute. Per the docs, safeguard_results is a top-level array, and each entry's status.tool_uses is keyed by tool call id, with a type, an outcome and an optional explanation. This TypeScript helper runs a call only when Armor evaluated it and didn't flag it:

type Verdict = { type: string; outcome?: " flagged " | " not_flagged "; explanation?: string; }; // Run a tool_use only if Armor evaluated it and let it // through. A missing or unavailable verdict is a refusal. function mayRun ( response: any, toolUseId: string ): boolean { for ( const r of response. safeguard_results?? []) { const v: Verdict | undefined = r. status?. tool_uses?.[ toolUseId ]; if ( v ) { return v. type === " evaluated " && v. outcome === " not_flagged "; } } return false; } Enter fullscreen mode Exit fullscreen mode Refusing missing and unavailable verdicts is my choice, not something the docs require. It mirrors the on_error default Armor uses for plugin requests, which is next.

The decision model can time out, error out or get a request too large to judge. The verdict is then unavailable, and on_error decides what happens: block, the default, or log ( Armor docs ). That setting doesn't reach the Claude Code mode. There, per the docs, "an unavailable call is always released with its verdict visible," so your client decides.

That third bullet is a per-tool-call cost. An agent turn that fires several tool calls can pay it on each one.

Armor's Claude Code path goes through Ori, OpenRouter's launcher that runs an agent CLI you already have on OpenRouter. It signs you in with OAuth instead of a pasted key, and it lists Claude Code, Codex, OpenCode, Cline and others. The Ori guide installs it with a script; read it before you pipe it into a shell:

curl -fsSL https://openrouter.ai/labs/ori/install.sh | bash Enter fullscreen mode Exit fullscreen mode The Armor docs then start Claude Code on a non-Anthropic model with auto mode on, and say no plugins entry is needed:

ori claude --model moonshotai/kimi-k2-0905 -- --permission-mode auto Enter fullscreen mode Exit fullscreen mode In Claude Code's auto mode, "a second model, the classifier, reviews actions instead of you." GIGAZINE reports that through Ori, models on OpenRouter can use that approval mechanism with Armor doing the review.

I build Hermes IDE, so weigh this paragraph with that in mind. Hermes runs every agent in a real terminal and takes any command-line agent you add, so the ori claude line above can go in as a Custom agent next to Claude Code and Codex, with each task on its own branch and worktree. Hermes never answers a permission prompt for you, and it doesn't read Armor's verdicts: that check happens between Claude Code and OpenRouter. I haven't run Ori through Hermes myself.

The ⌘N agent picker in Hermes 2.3.4: Claude Code and Codex installed, the other supported agents listed, and Custom agent for any other command.

If you're trying Armor this week, settle three things first:

Would you put a second model in front of every tool call your agent makes, at roughly 300 ms each? Or do you trust a permission prompt and your own eyes more? Tell me in the comments.

I build Hermes IDE, a free, source-available console for running Claude Code, Codex and other agents side by side, on macOS, Windows and Linux.

Armor puts a network hop and a fail-open-or-fail-closed decision in front of every action your agent takes. Latency budgets, timeouts and what a system does when a dependency stops answering are the core of the System Design Pocket Guide: Fundamentals, and they apply to an agent's safety check as much as to a payment service.

Quelle: dev.to (dev.to). Original öffnen