ai newsMi., 21. Okt.
EilmeldungAnthropic pitches AI safety bill that may block a model if 50+ people could be killed

My AI Agent Never Touches the Database

Originally published on Medium . Most AI agent demos put the model on top of the database. The agent gets a "query" tool, the user asks a question, the model writes the query. It works great in a demo. In production, it means the model's judgment is the only thing standing between a user and someon

dev.toDienstag 23:59 · 20.10.3 Min. Lesezeit
Bild: dev.to

Posted on Oct 20 • Originally published at Medium • Fully Autonomous

Most AI agent demos put the model on top of the database. The agent gets a "query" tool, the user asks a question, the model writes the query. It works great in a demo. In production, it means the model's judgment is the only thing standing between a user and someone else's data.

I'm building an AI agent into The Arsenal, a platform for life insurance agents. The first version is boring on purpose: it reads performance data, answers questions, and writes nothing. But even a read-only agent sits on top of client records, so I'm working through the OWASP Top 10 for LLM applications item by item before the agent exists. This post is item two, sensitive information disclosure, and the design decisions that came out of it.

The model never touches the database. It calls a small set of narrow functions, things like "get my pipeline stats for this week." No generic query tool, no raw SQL, no exceptions. If a function doesn't exist for a question, the agent can't answer it, and that's the point.

Every security question collapses into one place: what does the data access layer let through?

The model asks for things like a lead ID or a client ID. Those are parameters. The field that does the security work is who's asking, and the model never supplies it. The authenticated session does. The data access layer stamps the caller's identity onto every query and filters rows to what that identity is allowed to see.

If the model could pass "user equals Alex," it could pass anyone. So it shouldn't pass a user at all.

A KPI question never needs a client's phone number or date of birth. So the model should never receive them. It can't leak what it never saw.

Rather than hand-writing an allowlist per function, every column in the schema carries one tag: public, production, or client PII. Each function declares its tier, and the data layer refuses to return anything above it. The rule that matters is what happens when someone adds a column and forgets the tag: it's treated as PII and never returned. A forgotten tag produces a missing number, not a leak.

The agent will politely tell users that a phone number is out of scope. That's user experience, not security. The model refusing is behavior I asked for; the data layer never returning the field is a guarantee. If a prompt injection talks the model out of its refusal, it still gets nothing.

This design assumes the data access layer is correct. That's an assumption until it's tested, so the audit phase includes logging in as a downline user and trying to pull another agent's rows through the model. It also doesn't cover what happens when version two gives the agent the ability to write, which reopens every one of these questions. And none of it touches the model provider's own retention and training terms, which are a contract review, not a code change.

This is version one of one agent in a life insurance platform. I'm working through the rest of the OWASP list the same way, one item at a time, writing it up as I go. The goal is agent security that happens to start in insurance, not insurance security.

If you enjoyed this, I regularly share hands-on cloud builds, automation tricks, and AWS-focused deep dives across the web:

🔗 LinkedIn — for cloud content, networking, and consulting 📖 Medium — where this and other walkthroughs live 👨‍💻 GitHub — for open-source tools and infra templates 🖥️ Dev.to — cross-posts and project write-ups

Quelle: dev.to (dev.to). Original öffnen