SkillSentry — The Laziest Way to Keep Your AI Agent's Skills Honest

Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady Submission 🦥

This is a submission for the Hacktoberfest Week 3 DEV Challenge

Every agent skill you install is a note written to your AI agent's future self: when this situation happens, behave this way. And almost nobody reads those notes before the agent does. I used to feel vaguely guilty about that. Then I realized the truly lazy move wasn't reading them faster — it was making the reading someone else's problem. Preferably a machine's. Preferably one that runs in 22 milliseconds.

That's why I built SkillSentry: a local-first security scanner for agent skills ( SKILL.md files) and MCP server configs, with an optional second opinion from a Gemma 3 model running on my own laptop.

skillsentry is a TypeScript CLI with zero runtime dependencies. You point it at a directory of skills or an MCP config, and it returns a 0–100 risk score with concrete findings mapped to the OWASP LLM Top 10 (2026), AST10, and OWASP MCP Top 10 — the same vocabulary your security team already speaks.

Everything is wired for the laziest possible workflow: SARIF 2.1.0 output that drops straight into GitHub Code Scanning, CI-friendly exit codes, and inline suppressions that require a written reason — so future-you never has to remember why a finding was ignored, the audit trail remembers for you.

There's also a dashboard with a drag-and-drop scan API, in case you're too lazy to even open a terminal.

The effort math: I currently run 40+ agent skills. Reading one carefully takes ~15 minutes; doing it again after every update is a task nobody on earth actually performs. SkillSentry does the whole folder in ~3.9 ms per file (benchmarked at ~2,500 files/s), 127 tests behind it, coverage gates enforced in CI.

What you're watching: a poisoned SKILL.md — the kind that politely asks an agent to exfiltrate its context — gets flagged DANGEROUS, risk 71/100, 5 findings in 22 ms. A clean skill passes clean. And notice what happens when no local model is running: the scan doesn't fail, it degrades gracefully to static-only. Laziness is a supported configuration.

git clone https://github.com/Ulk0r0/skillsentry.git cd skillsentry && npm install && npm run build node packages/skillsentry/dist/cli.js./your-skills --fail-on high Enter fullscreen mode Exit fullscreen mode Where Gemma comes in Static rules catch known shapes. Attackers write novel shapes. That's the gap where most security tooling quietly dies — so SkillSentry has an optional semantic layer, and it's where Gemma earns its sticker.

Point it at any OpenAI-compatible endpoint running an open-weight model — gemma3:4b on Ollama is the default, and llama.cpp server, LM Studio, or vLLM work identically:

ollama pull gemma3:4b node packages/skillsentry/dist/cli.js./your-skills Enter fullscreen mode Exit fullscreen mode That's the whole setup. If Ollama is listening on its default port, the semantic layer just switches on.

Gemma's job is to judge intent, not vocabulary. For every skill it returns a structured verdict: what the text is really trying to make the agent do, which manipulation techniques it uses (instruction override, covert exfiltration…), what data flows where, a severity with a confidence score, and a rationale that must cite the exact phrases that justify the verdict. The system prompt forces it to map everything onto OWASP LLM Top 10 / MCP Top 10 ids, so Gemma's opinion lands in the same report as the static rules — not in a separate "AI magic" panel.

The laziest part isn't any single feature — it's that the whole thing is designed so you can set it up once and then trust the forgetting. Exit codes for CI, audit trails for suppressions, graceful degradation for missing models. The system assumes you will forget, and it doesn't mind.

A security auditor for an open ecosystem shouldn't be a black box — that's just another stranger asking for the same trust it's supposed to be checking. So the auditor itself is auditable: zero runtime dependencies means the entire behavior is readable source; MIT-licensed; local-first by architecture, not by policy. You can verify every claim in this post from the repo.

And the semantic layer only works because Gemma 3 is an open-weight model. A closed API would mean shipping every skill file you audit to a third party — a data-leak dressed up as a security product. Open weights turn intent-level analysis into something you can run offline, for free, on hardware you own. That's not a convenience; it's what makes private security review of agent skills possible at all.

Finally, the standards are open too — OWASP LLM Top 10 (2026), OWASP MCP guidance, the agentskills.io spec. SkillSentry cites them instead of inventing a proprietary severity alphabet, so its output plugs into conversations your team is already having.