Your AI Wrote a Test That Only It Can Pass
The model wrote the handler. The model wrote the test. CI went green. This pattern has become common enough that I now treat a green CI on an AI-authored PR as a starting point, not a verdict. The problem isn't that the model writes bad tests — it's that it writes tests optimized to pass the code i
Posted on Oct 9 • Originally published at powerduck.com
The model wrote the handler. The model wrote the test. CI went green.
This pattern has become common enough that I now treat a green CI on an AI-authored PR as a starting point, not a verdict. The problem isn't that the model writes bad tests — it's that it writes tests optimized to pass the code it just wrote.
A test suite written by the same model that produced the code answers one question: does this implementation do what this implementation does? That's a tautology.
It catches typos and obvious crashes. It does not catch:
In every case, CI is green and the feature is still wrong. The model isn't lying — it's optimizing for the metric it can see (passing tests) rather than the one you care about (correct behavior against an outside spec).
When the spec, the code, and the test all come from the same source at the same moment, you've removed the independent check that testing is supposed to provide. Classic testing wisdom already says this: you want a test that would fail if the implementation drifted from some external expectation. If the expectation itself was generated in the same pass, there's nothing to drift against.
This is why golden-master tests, contract tests, and externally defined fixtures have always mattered more than unit tests written in the same session. They don't share the model's blind spot.
You don't need a full rewrite. Three concrete moves have been effective:
The first two points are where a local-first spec tool earns its keep. We do this in Powerduck: the OpenAPI spec lives on disk, the model implements against it, and a contract check runs the spec against the running endpoint after the fact. If the model's implementation drifted from the spec, that check fails — even if every unit test the model wrote is green. It's a deliberately dumb check, and that's the point: it can't share the model's assumption.
AI-generated tests are not worthless. They're fast, they're comprehensive at the happy path, and they catch the stupid mistakes that used to eat review cycles. But they're one layer of defense, not the whole line.
The question to ask in review is no longer "does this test pass?" It's "what outside reference is this test comparing against?" If the answer is the code itself, you have a test suite that will stay green right up until the user files the bug.