"AgentGuard", Way to secure your personal data from AI agents

A few weeks ago, I had one of those heart-stopping developer moments. I was tinkering with a local autonomous coding agent running on a small open-source model (Qwen 2.5 via Ollama). I gave it access to a bash tool so it could install dependencies, run test suites, and clean up build artifacts. Everything was going smoothly until I asked it to clean up some old logs. In the terminal output, I watched it calmly propose: bash rm -rf /var/log/mongodb

And right after that, it started poking around files outside its project folder.I hit Ctrl+C so fast I almost snapped my keyboard. The "Polite Prompt" Trap Like most developers, my first instinct was to fix it with prompt engineering: "You are a careful assistant. Please ONLY touch files in./workspace. NEVER use sudo. NEVER delete system folders. Pretty please." And you know what happened on the very next run?

Here’s the thing we don't talk about enough: Small, open-source models (3B, 7B, 14B) don't have the massive alignment layers that GPT-4 or Claude 3.5 have. When an open-source agent gets confused, hallucinates, or hits an edge case, your polite system prompt is the first thing out the window.

If an AI decides to read your private banking statement or a.env full of production tokens, you shouldn't have to convince the model to behave.

You wouldn't ask an untrusted web request to "please not do SQL injection"—you use parameterized queries.

So why are we begging AI agents not to nuke our hard drives?

I wanted a simple rule: The agent can think whatever crazy thought it wants, but before any tool touches my operating system, someone checks its badge at the door.

If the agent decides to read bank_statement.pdf, the system shouldn't plead with the AI. It should intercept the file call, say "Nope, not happening", and ensure 0 bytes of that file are ever read.

AgentGuard is essentially a bouncer sitting between your AI agent and your OS. It does three straightforward things:

[ AI Agent / Ollama / OpenDevin ] │ │ 1. "Hey, I want to run: cat bank_statement.pdf" ▼ ┌───────────────────────────┐ │ AgentGuard Gateway │ ◄── [ Backboard.io ] └───────────────────────────┘ (Remembers previous blocks) │ │ [ BLOCK ] [ APPROVE ] │ │ │ └──► Rings your dashboard bell: │ "Agent wants to pip install something. Allow?" ▼ Refusal sent back to Agent: "Access to banking file blocked. Stay in./workspace." │ ▼ Agent self-corrects: "My bad! Reading./workspace/logs/debug.log instead."

When an agent calls a tool (shell command, file read, file delete), AgentGuard inspects the request synchronously before execution.

Trying to touch.env, id_rsa, or a file matching bank or tax? → Blocked.

Escaping./workspace with../../? → Blocked. The file handle is never opened. The shell command is never spawned.

What about grey-area actions? Say the agent wants to run pip install pymongo.

You don't want to completely block package installs, but you don't want an AI silently pulling unknown wheels from PyPI either.

AgentGuard pauses the agent and puts the request into a Human Review Queue: You get a notification on a clean web dashboard. Each request has a 5-minute cryptographic timer so old, stale commands can't accidentally execute hours later.

You click [ AUTHORIZE & RUN ], and it runs inside a controlled sandbox right in front of your eyes.

One thing that drove me crazy about local models was amnesia. If you tell an agent "don't delete that folder", it apologizes... and then tries again two turns later.

To fix this, I plugged in Backboard.io for persistent cloud vector memory.

Whenever AgentGuard blocks a command, it writes a short lesson into Backboard:

"Incident: Agent tried to run 'sudo rm -rf'. Blocked by policy shell.destructive_commands."

Next time you start a task, AgentGuard pulls the relevant memories from Backboard and feeds them into the agent's context. The agent literally remembers what got it in trouble yesterday and stops proposing bad commands in the first place.

I didn't want a clunky terminal-only tool, so I built an industrial, dark-mode cockpit with React and Tailwind:

Top KPI Cards: Live count of blocked attacks, total actions governed, and learned Backboard memories.

Review Queue: Tabs for Pending, Approved, and Expired actions with inline terminal output.

Threat Lab: A built-in sandbox where you can test 1-click red-team scenarios (ransomware simulations, credential exfiltration, path traversal) to see if your local model can break out. Spoiler: 0 escapes.

Making It an NPM Package: Bring Your Own Keys (BYOK)

Right now, you can clone the repo and run it locally with FastAPI and MongoDB.

But I realized that's still too much friction for developers who just want to npm install something and start coding.

So I'm packaging this as a standalone Bring-Your-Own-Keys (BYOK) NPM SDK:

You drop in your own Backboard API key (for cloud memory) and your own LLM key (or Ollama endpoint):

typescript import { AgentGuard } from 'agentguard'; const guard = new AgentGuard({ backboardApiKey: process.env.BACKBOARD_API_KEY, workspace: './workspace', }); // Wrap any agent tool call: const check = await guard.inspect(proposedCommand); if (check.isBlocked) { // Feed safety feedback back to the LLM so it self-corrects! console.log( Blocked: ${check.explanation} ); } else { await guard.execute(proposedCommand); }

Zero servers to host. Zero background databases to manage. Just pure protection.

We are entering an era where AI agents aren't just autocomplete engines anymore—they are active operators on our computers. They write code, manage files, and execute bash scripts.

We have to stop treating AI safety as a philosophical prompt problem and start treating it as a systems architecture problem.

Don't convince the model to behave. Just put a guard on the wire.

If you're building autonomous agents or playing with local models, I'd love to hear how you're keeping them safe. Have you had an AI agent do something terrifying on your machine yet?