vibeaudit: 11 static checks for the security holes AI-built apps ship with

The last few years of app building went like this: you describe what you want, a model writes the code, the code runs, you ship it. The tooling got dramatically faster; the verification step did not happen at all, and nobody - including the model that wrote the code - was checking.

The numbers are not theoretical. A June 2026 study on DEV.to scanned 62 Lovable apps: 63% had critical or high severity vulnerabilities, 10 findings per app on average. A separate study of 400 public Lovable, Bolt and v0 repositories, published by VibeSafe, found 59% had at least one critical issue and 18.5% had committed a.env file to git.

These are not random bugs. The same handful of mistakes keep showing up, in the same order of severity. That makes them automatable, and it is why I built vibeaudit.

vibeaudit is a command line tool. Point it at a project folder and it reads the code, the SQL migrations, the git history and the build output, then produces a ship-readiness report.

Three properties mattered more to me than the detection itself:

The score starts at 100 and every finding deducts points:

score = max(0, 100 - 25*critical - 12*high - 4*medium - 1*low)

The weights are chosen so that four criticals, or one critical plus a few highs, can sink a project on their own. A straight subtraction is trivially explainable to a client ("each critical finding costs 25 points") and deterministic, so two audits produce comparable numbers.

One rule sits on top of that: a single critical finding caps the verdict at "Do not ship yet", whatever the score. A score must never be able to hide a problem that puts customer data at risk.

Run against the bundled demo project - a small, deliberately broken app - here is the real output:

Ship readiness: 0/100 - Unsafe to ship | Critical | High | Medium | Low | |---|---|---|---| | 12 | 20 | 7 | 2 | Enter fullscreen mode Exit fullscreen mode And the plain-English paragraph that follows it, verbatim:

We checked 16 files in your project for the security mistakes that AI-built apps make most often - leaked passwords and keys, database tables that anyone on the internet can read, and admin pages that are not protected. We found 41 issues - 12 serious problems, 20 important problems, 7 medium problems and 2 small problems. That gives a readiness score of 0 out of 100, which means the app currently has serious security holes and should not be exposed to users or real data until they are fixed.

The most urgent one is: A powerful secret key is sitting in your project files. Anyone who gets hold of it can act as you, followed by stripe live secret key found in.env. All of these can be fixed without rebuilding the app.

We only looked at the files you gave us - we did not test your live site, so the online parts of your app still need a separate check.

The evidence keeps a short prefix so you can tell which key it is; the rest never appears:

SUPABASE_SERVICE_ROLE_KEY=eyJhbGciOiJI************************[REDACTED len=203] role=service_role Enter fullscreen mode Exit fullscreen mode That is the report's design: the reader is usually a founder who does not read code, so every finding carries a plain-English line, the file and line, the fix, and a prompt to paste back into your assistant.

The live checks touch the network, so they are fenced off.

1. The demo fixture's secrets are generated at runtime, not committed. GitHub push protection refuses any push containing a string shaped like a live credential - even a fabricated one - and it scans git history as well as the tip. A demo fixture full of realistic keys would make the repository impossible to publish, defeating the purpose of shipping the tool. So fixtures/build-fixture.mjs writes the vulnerable files at runtime, assembling each credential from parts so no contiguous secret pattern appears in the published source tree, and it creates the fixture's own git repository so the.env is genuinely committed rather than merely present. The demo still exercises every detection path it is supposed to.

2. Scanning a subdirectory of a repo still finds committed secrets. A client sends you one sub-project of a monorepo, and the file that leaks their service_role key is committed in the parent repository. A naive implementation checks for a.git directory in the target folder, finds none, and reports "no git history" - losing the finding that matters most. The fix was isGitRepo, which falls back to git rev-parse --show-toplevel and walks up to the repository root. git ls-files and git log still run from the scanned directory, so the paths they return stay relative to what you asked for and no unrelated project is pulled in.

There is nothing to install. Node 20+ is the only requirement.

# 1. See what it finds, against a bundled, deliberately broken demo app. node cli.js --demo # 2. Scan a real project. node cli.js --target./some-project # 3. Write the report to a file you can send to someone. node cli.js --target./some-project --out report.md # 4. Add the live checks. Only if you own the site or have written permission. node cli.js --target./app --url https://app.example.com --i-have-authorization Enter fullscreen mode Exit fullscreen mode Also useful: --json for machine-readable output, --no-history to skip the git history scan, --fail-on-critical for CI, --quiet for just the headline.

It is not a penetration test. It does not break in, exploit anything, attack your live app, or take a copy of your data, and it certifies nothing. It is not a compliance audit, and it is not a guarantee that your app is secure. It is a focused code and configuration review of the files you gave it - the subset of a real pentest that can be automated. If you need a signed penetration test report for a compliance file, that needs a certified tester, and this is not it. It does not cover your hosting, DNS, firewalls or third-party services beyond the keys it found.

If you would rather have a person read the results than run the tool yourself, that is the same review sold at a fixed price: $199 for the report, $599 with the fixes implemented, $899 for a pre-launch ship check that adds deployment hardening and a written checklist.

Try it: https://github.com/rafikulislammollaq-dev/vibeaudit

MIT licensed. Run node --test test/ and you should see 69 passing tests.

Command line (Node 20+, zero dependencies):

git clone https://github.com/rafikulislammollaq-dev/vibeaudit cd vibeaudit node cli.js --demo Enter fullscreen mode Exit fullscreen mode In your browser, nothing uploaded: rafikulislammollaq-dev.github.io/vibeaudit/scanner

Source: github.com/rafikulislammollaq-dev/vibeaudit - MIT licensed.

If you would rather have a person read the findings and explain them, that is the $199 review: rafikulislammollaq-dev.github.io/vibeaudit