Your LLM Returned Valid JSON. Can Your App Trust It?

Getting an LLM to return valid JSON feels like a milestone. The response parses, the fields exist, and the application can finally do something with the output.

But there’s another question: are those fields actually correct?

Consider a support assistant that turns a customer’s message into a structured request:

{ "order_id": "ORD-1042", "action": "refund", "reason": "damaged_item" } Enter fullscreen mode Exit fullscreen mode Everything looks reasonable. Yet the customer might have written: “My item arrived damaged. Can I get a replacement? I don’t want a refund.”

This is the syntax check. Is the response valid JSON, or did it include extra text, broken quotes, or an incomplete object?

2. Does it match the expected structure?

This is the schema check. Are the required fields present? Are values the right types? Is action one of the allowed options?

A schema can reject an unknown action. It cannot, by itself, determine whether an allowed action matches the customer’s intent.

3. Is the result supported and permitted?

This is where application logic matters. Does the order belong to the authenticated customer? Did the customer request this action? Is the order eligible? Does the operation require confirmation?

These checks need information beyond the shape of the model’s response.

For a workflow like this, I’d treat the model’s output as a proposed interpretation. The backend would check the order, apply the relevant rules, and ask for clarification when the request is ambiguous.

It also helps to give the model a way to express missing information. A required field with no “unknown” option creates pressure to fill in something, even when the source doesn’t support a confident answer.

For example, an incomplete request could go into a needs_clarification state instead of being forced into refund or replacement.

That state should have a real application path: ask a follow-up question, preserve the existing information, and wait before taking action.

Check both the extracted fields and what the application does next. A response can pass every formatting check and still lead to the wrong operation.

Structured output makes model responses easier to process. Deciding when to trust those responses remains part of the application’s design.

Have you encountered an LLM response that passed validation but was still wrong in a way that mattered? How did you catch it?