KI-Coding-Assistenten erfinden oft API-Endpunkte, die es gar nicht gibt
Coding-Tools generieren Code mit scheinbar korrekten Endpunkten, Headern und Antwortformaten, selbst wenn die eigene API diese gar nicht anbietet. Der Code laesst sich kompilieren und Tests laufen durch, scheitert aber in der Praxis.
Posted on Oct 13 • Originally published at powerduck.com
You ship a small feature on a Tuesday. The assistant writes the whole thing in one pass: a fetch wrapper, a couple of loading states, error handling. It even names the endpoint GET /v2/invoices/:id/line-items, adds the right headers, and parses the shape you'd expect.
It compiles. The unit tests pass. You merge it.
On Wednesday the page goes blank. The network tab shows a 404.
There is no /v2/invoices/:id/line-items. The endpoint that actually exists is /v1/invoices/:id/items. The assistant invented the path, guessed the version, and reconstructed a response shape from the three other SaaS docs it has seen in training that look vaguely like your billing system.
We say "hallucination" and picture the assistant confidently citing a paper that doesn't exist. This is quieter and more common: the assistant produces an API call that looks completely reasonable, because it is reasonable — that's how a generic invoices endpoint is usually shaped.
The problem isn't confidence. It's that the model has no live view of your system. It knows:
Two of those three sources are about somebody else's system. None of them is about what your service returns right now.
Because you let the assistant write the mock.
It generates the fetch layer, then writes a test that mocks fetch to return exactly the shape it assumed. The mock and the implementation agree with each other perfectly. They both disagree with production, and neither one knows.
This is the structural trap: the test isn't verifying the integration, it's verifying the assistant's guess. The mock encodes the very hallucination you were trying to catch.
You can fight this with prompts. "Double-check the endpoint paths." "Read the real API doc first." It helps a little. It does not help reliably, because the assistant still has to find, open, and trust the right document — and if that document is itself six months out of date, it's just a better-informed guess.
What actually works is giving the assistant a source of truth that is cheap to keep current and cheap to query:
We built Powerduck around this exact workflow: a local-first OpenAPI studio where the spec stays the source of truth, you can re-run it against a live endpoint, and the diff between documented and real is visible in seconds rather than discovered by a user.
Next time the assistant writes a call to an endpoint you can't immediately confirm, pause for twenty seconds. Open your real API source. Does that path exist? Does the field it's parsing actually come back null sometimes?
If the answer requires you to guess, your tooling is the problem. Prompts get you a cleaner guess. A live, local spec gets you an answer.
The Tuesday-after-Thursday 404 is expensive. The check that prevents it isn't.