LLM Prompts Need API Contracts, Not Git Diffs

TL;DR — Most teams version prompts the way they version code: line diffs and PR review. That hides the real risk, because prompt text is both the interface and the implementation at once. Borrow the API design split — define a contract for what a prompt must guarantee, separately from the wording that satisfies it — and prompt changes finally become testable.

Every team that ships LLM features eventually builds the same ritual: a prompt file lives in a repo, changes go through a PR, someone eyeballs the diff, and if it looks reasonable, it merges. This feels like engineering because it uses engineering tools. It isn't engineering, because the artifact being diffed is the wrong artifact.

A git diff on a prompt shows you that the wording changed. It tells you nothing about whether the thing the prompt was supposed to guarantee still holds. That's because in almost every production prompt, the interface and the implementation are the same string. There is no contract separate from the wording — the wording is the contract, informally, implicitly, and unenforceably. That conflation is the actual missing discipline, and it's a solved problem in every other part of software engineering.

In normal API design, nobody confuses the interface with the implementation. You define a contract: these inputs, this output schema, these invariants, these failure modes. Then you write an implementation that satisfies the contract. You can rewrite the implementation entirely — swap the database, change the algorithm, optimize the hot path — and as long as the contract tests pass, downstream consumers never notice.

Prompts don't get this treatment. A one-line wording change — swapping "summarize" for "briefly summarize," adding a clarifying example, reordering two instructions — is functionally equivalent to silently changing an API's return type, and it ships with the review rigor of a typo fix. The diff is small. The behavioral delta can be enormous, because autoregressive models are extremely sensitive to phrasing, ordering, and emphasis in ways that have nothing to do with how "big" the text edit looks. A two-word change can flip output format, change refusal behavior, or alter how much the model leans on context versus parametric knowledge. Line-diff review cannot see any of that, because line diffs measure textual distance, not behavioral distance.

The fix isn't a better diff tool. It's writing down the contract that the prompt is supposed to implement, independent of the prompt itself. A real prompt contract specifies things like: the output must be valid against a given schema; the response must not assert factual claims absent from supplied context; under missing or contradictory input, the system must degrade to a specific fallback state rather than guess; tone and length must stay within bounds regardless of input length; certain categories of request must be declined in a specific, detectable way.

None of that is wording. It's behavior. Once you have it written down as a contract, the prompt text becomes one possible implementation of that contract — and, critically, a replaceable one. You can rewrite the prompt for a new model family, trim it for cost, restructure it for a different context window, or swap in a totally different phrasing strategy, and the only question that matters is whether it still satisfies the contract. That question has a test. "Does this wording feel right" does not.

This split matters just as much on the context side as the prompt side, and context engineering tends to get this wrong in the opposite direction: teams obsess over retrieval quality and formatting and never write down what the context assembly step is actually promising the model. Is the injected context guaranteed to be deduplicated? Guaranteed to be under some token budget regardless of how many documents match? Guaranteed to preserve source attribution per chunk? Guaranteed to exclude anything above a permission boundary?

If those guarantees aren't written down as a contract, every change to chunking, reranking, or formatting is an untested interface change wearing the costume of a pipeline tweak. The context assembler is an API with callers — the prompt downstream of it is depending on properties of what it receives, whether anyone wrote those properties down or not. Undocumented assumptions don't disappear just because nobody documented them; they just fail silently, later, in production, usually attributed to "the model got worse" when the model didn't change at all.

Once contract and implementation are separated, versioning stops being cosmetic. A wording tweak that doesn't change the contract is a patch — low ceremony, fast to ship, safe to roll forward. A change that alters what the system guarantees — a new output field, a relaxed invariant, a different fallback behavior — is a breaking change to the contract itself, and it should be versioned, reviewed, and rolled out like one, with every downstream consumer of that prompt or context pipeline required to re-certify against the new contract before adopting it.

This is exactly the discipline teams already apply to internal APIs. Nobody lets a backend engineer silently change a response shape because the diff "looked small." The reason prompts get a pass is that the artifact looks like prose instead of code, so it gets reviewed like prose instead of an interface. That's a category error, and it's the single biggest reason prompt regressions make it to production undetected: the review process is calibrated to the wrong risk model.

"Prompt engineering" became a punchline partly because it had no artifact other than text and no way to verify that text against anything. Treating the contract as the first-class object fixes that. It gives you something to write regression tests against that survives a rewrite. It gives you a reason to version aggressively on behavior and loosely on wording. It gives reviewers a concrete question to ask — "does this still satisfy the contract" — instead of a vibe check on phrasing.

It also reframes what a prompt library should look like. Not a folder of strings, but a set of contracts, each with one or more implementations, each implementation certified against a test suite that exercises the contract's invariants and failure modes. The wording becomes disposable. The contract becomes the thing you actually maintain, review, and own — which is what should have been true from the start, because that's the part a downstream system actually depends on.