Everything You Need to Know About Testing AI-Generated Code

Generated code and generated tests often come from the same source, so they can agree with each other and still be wrong. A test that asserts the output of a buggy function will pass. That is not a test of correctness. It is a record of what the code does.

Testing generated code means making sure the tests describe what the feature should do, not what the code happens to do.

Before the agent writes the implementation, write down what the feature must do. A short list of cases is enough: the normal case, the empty input, the invalid input, and the failure from a dependency. Give those cases to the agent as the specification. The tests then come from the requirements, not from the code.

A test that cannot fail proves nothing. Once the tests pass, break the feature on purpose: return the wrong value, skip the check, or swallow the error. If the tests still pass, they are not testing what you think they are.

Mutation testing tools automate this. They change the code in small ways and report which changes the tests miss.

Generated code tends to test the success path thoroughly and the failure path lightly. Make sure there are tests for timeouts, missing data, and errors from other services. These are the cases that matter in production.

Tests that depend on each other, on the order they run in, or on a shared database produce false results. Each test should set up its own data and clean up after itself.

Unit tests check functions. Integration and end-to-end tests check that the pieces work together. For a multi-service app, that means starting the services, calling them, and checking the result. A test that only imports a module can pass while the service never starts.

TDK CLI can start a local multi-service stack with tdk up, and tdk status --json reports readiness. A test suite can wait for readiness before it runs its end-to-end checks, so the tests do not race the startup.

When a test fails, an agent may change the test to match the code. Require the agent to explain why the expected value was wrong before it edits the test. Review every change to a test as carefully as a change to the feature.

Test generated code against behaviour you wrote down before the agent started. Make sure the tests can fail, cover the failure path, and run against the real system. Review test changes as seriously as code changes.