Black-Box Testing with AI Agents: Test the Rules, Not the Code

One of the most useful test reports I've seen an AI produce came from agents that never saw a single line of the application's code.

At a company I worked at, we tried a simple experiment. We gave the models the business rules of a system that was already built, and nothing else. No repository, no architecture docs, no database schema. The models wrote the tests from the rules, then ran those tests against the running application. At the end we got a list of every place where the application did not do what the business said it should.

That list was more useful than any green test suite, because every line in it was a rule the system broke. And the reason it worked is not that the models were especially clever. It's that they didn't know how the system was implemented, so they couldn't bend their tests around it.

This is black-box testing with AI agents: the agent tests the behavior against the specification, from the outside, the way a customer or an auditor would. This post covers:

Ask a coding agent to "write tests for RefundService " and it will read RefundService, see what it does, and write tests that check it does exactly that. If the code has an off-by-one in the refund window, the agent sees daysSinceDelivery < 30, and the generated test asserts that day 30 is rejected. Green suite. Wrong behavior. Now locked in by a test.

Testers have a name for the source of "expected": the test oracle, whatever you use to decide whether an observed result is correct. This isn't a model failure. It's what any tester does when the code is the only spec they have: the implementation becomes the oracle, so the tests can only ever confirm it.

It gets worse when the same agent wrote the code and the tests. Both come from the same understanding of the problem, so any misunderstanding shows up twice and cancels out. I covered a version of this in the context of dark factories: a check is only worth something when the thing being checked can't shape it.

That third row is the one people miss. If nobody implemented "only the order owner can request a refund", there's no code that suggests the test. A black-box agent reads the rule and tries it.

To be fair to white-box testing: an agent with the code and an independent list of requirements could catch this too. Code access isn't the defect. When an agent derives its expectations from the implementation, it reproduces the misunderstandings already in the code. In my view, keeping the code away from the testing agent is the simplest way to stop that from happening.

Black-box testing is old. The ISTQB calls it specification-based testing: you derive test cases from the requirements and the observable behavior, without looking at the internal structure. The classic techniques are still the right ones (chapter 4 of the ISTQB Foundation Level syllabus, free to download):

The techniques were never the hard part. The hard part was the labor: turning fifty business rules into a few hundred precise cases is slow, boring work, and people skip the boring edges. That's exactly where LLMs are useful. They're good at reading a rule in plain language and systematically expanding it into partitions, boundaries and combinations, and they don't get bored at case 140.

What the agent must not do is invent the expected result. The value of the whole approach depends on expectations coming from the rules, so the pipeline has to make that a hard boundary, not a polite request.

This is the setup I'd build today. After a one-time setup (step 0, isolation), it has four steps: write the rules, design the cases, generate the tests, then run and triage. Three agents do the work, and humans review at two checkpoints.

business rules ──▶ [1. test designer] ──▶ test cases + open questions │ human review │ [2. test generator] ◀──────┘ │ executable tests ──▶ run against staging (public interfaces only) │ [3. triage] ◀──────────────┘ │ findings report ──▶ human decides: bug, rule gap or bad test Enter fullscreen mode Exit fullscreen mode Step 0 (setup): isolate the agents from the code The simplest way to stop an agent from reading the code is to make sure the code isn't there. I wouldn't rely on instructions ("don't open src/ ") or on a deny list that may not cover every tool. Give the testing agents their own workspace:

qa-workspace/ # everything the testing agents can see ├── rules/refunds.md # business rules, each with an ID ├── contracts/openapi.yaml # the public API contract clients already get ├── cases/ # generated test cases (Markdown) ├── tests/ # generated Playwright tests └──.env.test # BASE_URL + test-only credentials (gitignored) # The application repository is not cloned here. Enter fullscreen mode Exit fullscreen mode The agents talk to the system only through what a real client would use: the public API and the UI of a staging environment. If the agent can't reach it from outside, it isn't part of the test.

Isolation is necessary, but it doesn't make the oracle independent on its own. That depends on where the expectations come from. If someone writes the rules by watching what the current system does, or "fixes" a failing expectation to match what the system returned, the oracle depends on the implementation again, just one step removed. Expectations come from rules that the business has reviewed, and a failing test changes only when the rule changes.

Garbage in, garbage out applies hard here. The rules don't need a formal language, but each one needs an ID and a single, checkable statement. Here's a hypothetical refund policy for an online store, which I'll use for the rest of the post:

# Refund rules - BR-01: A customer can request a refund up to 30 days after delivery. - BR-02: Refunds above 500.00 EUR require manager approval before payout. - BR-03: Digital items cannot be refunded after they have been downloaded. - BR-04: The total refunded for an order can never exceed the amount paid. - BR-05: Only the customer who placed the order can request its refund. Enter fullscreen mode Exit fullscreen mode Notice that these are written the way a product owner would write them, not the way a developer would. That's the point. If your rules only exist in people's heads or in Jira comments, collecting them into a file like this is the first deliverable, and often a valuable one on its own.

The first agent turns rules into cases. The prompt does two jobs: it tells the model which techniques to apply, and it forbids guessing.

You are a QA analyst. You have not seen the application's code and you must not ask for it. Input: the business rules below, each with an ID. For every rule: 1. Derive test cases using equivalence partitioning, boundary values, decision tables (for combined conditions) and state transitions (for lifecycles). Include invalid and unauthorized cases. 2. Write each case as: case ID, rule ID(s), technique, preconditions, Given/When/Then, expected result. 3. Expected results must come only from the rules. If a rule does not say what should happen, do not guess: add the case to an "Open questions" list instead. Output: a Markdown table of cases, then the open questions. Enter fullscreen mode Exit fullscreen mode For BR-01 and BR-02, a good output looks something like this (trimmed):

Q1 (BR-01): Is "30 days" counted in calendar days, and in which time zone? Q2 (BR-03): Does a partial download count as "downloaded"? Q3 (BR-02 + BR-04): Do two refunds of 300 EUR on the same order need approval? Q4 (BR-02): At 500.00 EUR or less, is the payout immediate or still queued? Enter fullscreen mode Exit fullscreen mode The open questions are a deliverable, not noise. Every one of them is a place where two developers could have implemented different behavior and both claimed they followed the spec. A product owner answers them in minutes, and the answers go back into the rules file.

This is the first human checkpoint: someone who knows the business skims the cases and answers the questions before anything runs. It's a much cheaper review than reading test code.

You could let an agent click through the app and decide on the fly whether things look right. I wouldn't. A run you can't repeat is a run you can't trust, and you want to re-run the same suite after every fix.

So the second agent turns approved cases into ordinary test code, here with Playwright, which covers both API and UI tests:

import { test, expect } from " @playwright/test "; import { createDeliveredOrder, loginAs } from "./fixtures "; // TC-02 / BR-01: day 30 after delivery is still inside the refund window test ( " TC-02 BR-01: refund accepted on day 30 after delivery ", async ({ request }) => { const customer = await loginAs ( request, " customer-a " ); const order = await createDeliveredOrder ( request, customer, { deliveredDaysAgo: 30 }); const response = await request. post ( `/api/orders/ ${ order. id } /refunds`, { headers: customer. headers, data: { amount: order. total }, }); // BR-01 says "accepted"; 201 is how the API contract says "accepted" expect ( response. status ()). toBe ( 201 ); }); Enter fullscreen mode Exit fullscreen mode This snippet is illustrative: loginAs and createDeliveredOrder are fixtures you have to provide, and baseURL comes from the Playwright config. The test name carries the case and rule IDs, which is what makes the final report traceable back to the business.

Notice where the 201 comes from. The business rule only says the refund is accepted; it says nothing about HTTP. The status code comes from the public API contract in the workspace, which is still a black-box source: it's what any client integrating with the API reads. So the generator gets two inputs, the approved cases (the behavior) and the contract (how that behavior shows up on the wire), and neither one is the implementation. If the contract doesn't say how "accepted" looks, that's another open question, not a guess.

If you test through the UI, Playwright's own test agents (planner, generator and healer, set up with npx playwright init-agents --loop=claude, or the loop for your tool) can do much of this step. The planner can take a product requirements document as context, which fits this approach well. Review what the healer changes, though: in a black-box setup, a test that fails because the app is wrong must stay failing.

Run the suite against staging. Then a third agent, or the same one in a fresh session, turns the raw results into the report people actually read. For each failure it records the rule, the case, what was expected, what happened, and the evidence (response body, screenshot or trace).

The agent also proposes a verdict, because not every red test is a bug:

The proposed verdict is a starting point; the second human checkpoint is someone confirming it before anything becomes a ticket.

Here's what a report for the refund rules could look like. This is a hypothetical run, built to show the format and the kinds of issues the approach catches:

## Findings: refund rules, staging | # | Rule | Case | Expected | Actual | Verdict | |---|-------|----------------------------------------------|-----------------|------------------------------|----------------| | 1 | BR-01 | Refund on day 30 after delivery | Accepted | 422 "refund window expired" | Bug | | 2 | BR-04 | Two partial refunds totalling 110% of order | Second rejected | Both accepted | Bug | | 3 | BR-05 | User B refunds user A's order | Rejected | 201 Created | Bug (security) | | 4 | BR-02 | Two refunds of 300 EUR on one order | Unclear (Q3) | Paid without approval | Rule gap | Enter fullscreen mode Exit fullscreen mode Look at what each line represents. Finding 1 is the off-by-one a white-box agent would have locked in. Finding 2 only shows up when you combine two rules, which is exactly what decision tables are for.

Finding 3 is a broken access control bug: an ownership rule written in business language turned into a security test without anyone asking for a "security test". Finding 4 isn't a bug at all, it's a business decision nobody made.

This is the kind of output I saw at that company, and it's why the approach stuck. Developers argue with test code. It's much harder to argue with "the rule says X, the system did Y, here's the response". The report speaks the language of the people who own the rules, so it gets acted on.

This approach complements your existing tests. It doesn't replace them.

Skip it, or don't expect much from it, when:

Two costs are easy to underestimate. First, false failures: the agent will sometimes misread a rule, and every bad test costs someone time to triage. The open-questions step and the human review of cases are what keep that rate down. Second, a stable environment: a flaky staging environment turns every run into noise, and noise is how teams stop reading the report.

Isn't this just BDD with extra steps? It shares the idea that behavior is the contract, and Given/When/Then is a fine format for the cases. The difference is who writes the scenarios and who can see what. In BDD the team writing the code usually writes the scenarios too. Here, the agent writing the tests never sees the code.

Can the same model write the code and the black-box tests? The same model, yes. The same session or context, no. What matters is that the test designer's input is the rules and nothing derived from the implementation. A fresh session in a separate workspace with no repository is enough.

Doesn't the agent need the code to find UI selectors? No. Playwright locators work best with roles and visible labels ( getByRole, getByLabel ), which come from what the user sees, not from the source. If a button can't be found by its role and name, that's often an accessibility finding in itself.

How is this related to holdout scenarios? It's the same principle. StrongDM's team, as described by Simon Willison, keeps end-to-end scenarios outside the codebase so coding agents can't see them. Black-box testing from rules is a way to build that holdout set without hand-writing every scenario.

What about secrets and test credentials? Use test-only accounts on a staging environment, keep them in a gitignored env file, and never point the agents at production. The agents only need what a regular customer would have.

Generating more tests was never the hard part. What AI agents make cheap is an independent check of whether the software keeps the promises the business made, written in the business's own words.

Have you tried letting agents test a system without showing them the code? I'd like to hear what the first findings report surprised you with, or why you decided against it.

I'm Diogo Soares, a Staff AI Engineer working on AI agents and legacy modernization. If this was useful, follow me here on DEV.to and on Instagram at @diogosoares.dev, where every article becomes a visual carousel you can save, like the 4-step version of this pipeline. This article also has a video walkthrough on my YouTube channel, Diogo Soares Dev, so subscribe there for the next ones. Let's also connect on LinkedIn, or check out diogosoares.dev.