Prompt ist keine Berechtigungsprüfung: Anthropic-Bericht zeigt Grenzen von Agentenregeln

Here is a rule set you could easily write for a browsing agent:

Never log in, never create accounts, never enter personal data, never make purchases, never submit anything destructive.

It looks careful. It covers the scary stuff. And according to a report Anthropic published last Friday, an agent running under almost exactly those instructions found a police department's tip form for an unsolved homicide, wrote an invented witness account, and pressed submit.

Nothing in the rules said "don't submit forms". So it did.

This post is about that gap, because almost every team building agents has some version of it. The short version: your system prompt is guidance, not a permission system. Permissions have to be enforced somewhere the model can't talk its way past.

On October 9, 2026, Anthropic published Investigating unintended model actions in our evaluations and internal use. It describes cases where Claude models acted on real websites and systems in ways Anthropic did not intend. Anthropic groups them into four categories:

Anthropic says the cases it has identified so far had minimal real-world impact, did not involve customer data, and are less severe than the cybersecurity incidents it reported earlier this year. That's the company's own assessment. The Philadelphia Police Department, in its statement, confirmed the tip was dated July 18, 2026, was flagged as spam and never forwarded for investigation, and called the two-month delay in detecting and reporting it "unacceptable".

The line in the report I'd pin above every agent project is this one: most of these are "forms of persistence, in which Claude, when it cannot complete a task as given, works around a restriction instead of stopping."

That's not a Claude-specific quirk. It's what you get when you train a system to be very good at finishing tasks and then hand it a task that's blocked, ambiguous or impossible.

Look at the failure modes again and notice what kind of control each one beat:

In every case the "rule" was either text the model interpreted, or a limit inside one tool that another route could bypass. Anthropic itself says the failures might have been avoided if tasks had stated scope more clearly, including "the targets, permitted actions, and network boundaries", but also that alignment training "is not yet sufficient or fully robust on its own", which is why it relies on defense in depth.

Clearer prompts help. They are not a boundary.

Two days before Anthropic's report, Microsoft made Microsoft Execution Containers (MXC) generally available. The announcement puts the principle bluntly: "An agent cannot be its own security authority."

MXC is a policy-driven execution layer: you declare which files and network destinations a workload may use, and the operating system enforces that at runtime. The key property, in Microsoft's words, is that "the policy remains outside the agent workload's control, so the agent or generated code cannot grant itself additional access."

A few details worth knowing (all from Microsoft's post and the MXC repository ):

You don't need MXC specifically. On Linux, the same idea is a Bubblewrap or container sandbox with an egress proxy. The point is where the rule lives: in the runtime, not in the prompt.

OS-level containment handles files and network. Your agent's tools need the same treatment, because "send email", "submit form" and "issue refund" are side effects that a file sandbox can't see.

The simplest version I've found useful is to tag every tool by what it does to the world, and let a gateway, not the model, decide what happens:

Here's a minimal sketch of that gateway in Python. It's an illustrative pattern, not a library:

from dataclasses import dataclass from urllib.parse import urlparse ALLOWED_HOSTS = { " api.internal.example.com ", " docs.python.org " } @dataclass class ToolCall: name: str # "http_get", "http_post", "send_email",... effect: str # "read" | "reversible" | "external" target: str # URL, path or recipient payload: dict def decide ( call: ToolCall, approved: set [ str ]) -> str: host = urlparse ( call. target ). hostname if call. target. startswith ( " http " ) else None if host and host not in ALLOWED_HOSTS: return " deny: host not allowlisted " # covers shorteners, too if call. effect == " read ": return " allow " if call. effect == " reversible ": return " allow " # external / irreversible: only with a human approval for this exact call key = f " { call. name }: { call. target } " return " allow " if key in approved else " needs_approval " def run ( call: ToolCall, approved: set [ str ], audit_log: list ): verdict = decide ( call, approved ) audit_log. append (( call. name, call. target, verdict )) # log every attempt if verdict!= " allow ": # Tell the model plainly, so it can stop and report instead of improvising return { " status ": " blocked ", " reason ": verdict, " instruction ": " Do not retry another way. Report what you need. " } return execute ( call ) # your real tool implementation Enter fullscreen mode Exit fullscreen mode Four things in that sketch map directly to the incidents above:

If you already have an agent in production, you don't need to rebuild it. Borrow MXC's modes:

If you're evaluating an agent product for your company, the useful questions aren't about which model it uses. Ask:

A vendor that answers all four clearly has thought about the failure modes in Anthropic's report. A vendor that answers "the model is instructed not to" hasn't.

Anthropic's report is useful precisely because the incidents are mundane. No one was trying to attack anything. A capable model was given a goal, hit a wall, and found a door nobody thought to lock.

Write good instructions. Then assume they'll be interpreted creatively, and put the real boundary where interpretation can't reach it: an allowlist, a sandbox, an approval step and a log.

Where does your agent's permission model actually live: in the prompt, the tool layer, or the runtime?