How we're testing AI app builders: database isolation, auth, usage and source export

I work on Uvilno. This is our evaluation protocol, not an independent leaderboard, and we have not completed a matched cross-product study. We publish the unknowns because a polished generated page is easy to confuse with a finished application.

An AI builder can generate a plausible dashboard while its records disappear after reload. It can export a frontend while the identity service and database still live behind the vendor's API. It can say “fixed” while leaving the target button unchanged. Those are separate failure modes; a single aesthetic score obscures them.

Use two app visitor accounts, A and B. As A, create a uniquely labelled private task, edit it and reload. Confirm the server returns the edited record. As B, attempt to list, read, change and delete A's task. All four operations should fail without exposing the record. Then test a deliberately public catalog separately: public data and broken private isolation are not the same thing.

Record the schema and permission rules, not customer data. Test deletion after reload. Finally, restore an old website version and confirm it does not unexpectedly restore old customer records. Website versioning and transactional data history are different systems.

Leave the builder signed in as the creator. Sign into the generated app using another identity. Confirm the creator session is unchanged, then try another project. An embedded preview must not let app OAuth overwrite platform auth, and one project's visitor identity must not grant access to another project's private data.

We use project-scoped visitor identities in Uvilno's hosted app service. That describes the architecture; each supported provider still needs a real integration check. It is not a reason to skip the test.

Save baseline file hashes and screenshots. Request one exact change: “Remove the duplicate Sign in button next to Dashboard; preserve all other navigation.” Measure end-to-end time, record touched files and inspect the actual DOM. Check the other routes too.

Then send “thank you.” There should be no generated code or generation usage for a social acknowledgement. Introduce a controlled runtime fault and inspect the repair path, reservation release and saved checkpoint. A platform's “free fix” label is not evidence until the actual ledger is checked, including any rate or workload limits.

Count the original build, failed attempts, follow-up edits, database operations, hosting and source access separately. Record monthly versus annual billing. Do not convert one vendor's credits to another vendor's tokens without measured evidence. Also record what happens after subscription cancellation.

For a concrete published example, Uvilno offers a $0 Free tier, a $4.99 permanent source unlock for one project, Plus at $10/month, Pro at $20/month and Ultra at $59/month. Three individual source unlocks total $14.97. Plus plus one additional unlock totals $14.99 for that month and includes monthly usage, so those purchases are not identical. Pro includes unlimited project unlocks during the paid period, but generation is still usage-limited. Our deployed Stripe configuration remains in test mode until live payments are enabled; a test receipt is not revenue.

An export test starts with a clean machine, the exact lockfile and a different host. Check all routes, fonts, uploaded images, imports and environment bindings. Then test the data/auth/form dependencies. A ZIP does not transfer a hosted database or its private customer rows.

Uvilno's unlocked export includes generated source, referenced project images, bundled fonts and licenses. Optional hosted Uvilno.db and visitor authentication remain hosted dependencies; moving them requires a migration plan. Lovable, Base44, Emergent, Bolt.new, Replit Agent and v0 also need an explicit inventory of the services selected for a particular project. Cursor is a different comparison: the editor operates on a repository, while database and deployment architecture are choices the developer supplies. Check current vendor documentation rather than assuming one integration arrangement applies to every app.

Use the same brief, input assets, seeded data, hardware, review date and budget for every product. Include failures. Disclose affiliations. Keep source maps, secrets, identities and private customer content out of public evidence. Do not replace missing measurements with invented scores.

Our public benchmark methodology and 2026 buyer guide separate documentation from recorded checks. The SDK docs describe the hosted service boundaries. We would welcome corrections and comparable, reproducible results—including failures in our own product. That is more useful than calling every attractive hero section production-ready.