Shipping Software with AI Agents: How I Built a 48-Rule Scanner in 2 Days (and What 3 Audit Teams Found)

Posted on Nov 4 • Originally published at rehmanahmad.com

I am a head teacher by day and a solo developer by night. Last week I shipped a real tool — a 48-rule scanner that spots AI design slop in web projects — in about two days of evening work. I did not write most of it by hand. I directed a small team of AI agents, then handed the result to three independent audit teams and fixed everything they found. This is the honest story of shipping software with AI agents: what moved fast, what broke, and the checklist I now use before any solo release.

It started with a post someone shared with me: a hugely popular tool that scans for "AI slop" in design. It had tens of thousands of stars and a long feature list. Installing it would have taken five minutes.

I chose to build my own instead, for two reasons. First, I wanted a tool I fully understood — one file, zero dependencies, rules I could read and change myself. Second, I wanted to test a question I keep asking myself as a solo developer: can a one-person team with AI agents ship something genuinely useful, not just a demo that looks finished?

The answer turned out to be yes — with one condition. The agents are fast builders but careless reviewers. Everything that made the project real happened in the review stage, not the build stage. That is the lesson this whole article is about.

I have learned that agents do their worst work when the brief is vague. "Build me a slop detector" would have produced a generic tool with generic rules. So I wrote a short brief with three hard constraints before anyone wrote a line of code:

Those three constraints did more work than any clever prompt. They forced every later decision: the rule format, the output format, the exit codes. When an agent suggested a feature that broke a constraint, the constraint won.

By the end of the first evening, the core existed: a scanner with rules grouped into packs, including a pack for right-to-left layouts (I care about this because Urdu and Arabic content is badly served by most design tools) and a pack for the tired "2026 defaults" — the same cream background, serif heading, and sage-green accent combination appearing on every AI-generated landing page.

Day 2 was about making the tool usable by someone who is not me. That meant a live web checker anyone can try in a browser, an audit-report generator for teams that want a document rather than terminal output, and a small benchmark set so the rules could be measured instead of merely claimed. The benchmark matters more than it sounds: without it, "48 rules" is just a number. With it, I could say how often the scanner catches real slop and how often it cries wolf.

That is also the day the project got its name and its home: Deslop, open source on GitHub, with the live checker on the web. I publish free tools for teachers and students at filecobra.com, and I write about the shipping side of this work — the workflows, the mistakes, the checklists — on rehmanahmad.com/blog.

Two days, one solo developer, one team of agents. The tool worked. I was tempted to announce it right there. I am glad I did not.

Before any announcement, I handed Deslop to three independent audit teams and asked them to break it. Not to praise it — to break it. They came back with nine findings.

Some findings were small: messages that confused more than they explained, edge cases in a few rules. Some were serious: rules that fired on the wrong code, a threshold that let heavy slop pass as "mild," documentation that promised something the tool did not actually do. Every single one was real. None of them had been caught during the build, because the building agents were in "make it work" mode, and nobody was in "prove it works" mode.

Fixing all nine took another focused session, and it changed the tool in ways users can feel: clearer severity bands (clean, mild, heavy) so a score means something at a glance, tighter rules with fewer false alarms, and documentation that now describes exactly what the scanner checks — no more, no less. The version that shipped, v1.9.0, is a different tool from the one I nearly announced. Same 48 rules, much higher trust.

This is the part of shipping software with AI agents that nobody posts about. The build is the easy, photogenic half. The audit is where a demo becomes a tool. If you skip it, you are not shipping faster — you are shipping your bugs to your users and letting them do the audit for you, in public.

After this project, my mental model of AI agents is simple and I use it for every new build:

The practical rule I now follow: never let the builder audit its own work. A separate audit pass, with its own brief ("find bugs, rank them, show proof"), is a mandatory step in my workflow, not a nice-to-have. It costs one extra session. It saved this release.

This is the checklist Deslop earned. I run it before announcing anything now:

If you are a solo developer using AI agents, the temptation is always to announce at step two, when the demo works. Resist it. The gap between "works in my terminal" and "trustworthy for strangers" is exactly one audit wide.

Two things. First, I would order the audit earlier — after the core rules, not after the whole feature set — so fixes land before features pile on top of them. Second, I would write the honest limitation into the README from day one instead of adding it after the auditors asked for it. Limitations are not apologies; they are the reason people believe the rest of the document.

None of this makes me slower in any way that matters. Two days to build, one session to audit, and the result is a tool I can stand behind: open source, genuinely useful, and already published as a package other developers can install. The agents did not replace me as a developer. They replaced the empty hours — the scaffolding, the boilerplate, the first drafts — and left me the work that actually needs a human: deciding what to build, what "done" means, and whether it is good enough to put my name on.

If you are shipping solo with AI agents, try the audit step on your next project. Build it fast, then hand it to someone — or something — whose job is to break it. You will be annoyed by what they find. Your users will never know how close they came to finding it first.