LLM Text Classification: 4 JSON Schema Invariants for Reliable CRM Tags

Use JSON Schema when an LLM's classification result becomes a CRM action. Keep prompt parsing for disposable analysis, prototypes, and human-reviewed notes. The distinction is not about making the model “understand JSON.” It is about placing a checkable contract between uncertain text generation and a system that will create a task, change a field, or trigger an escalation.

For a logistics sales call, I would require four invariants before accepting a result: a closed set of labels, the right value types, required fields, and an explicit representation for uncertainty. Then I would validate the response, reject or quarantine failures, and only after that map it into CRM commands. A regular expression around a prompt-shaped answer cannot provide that boundary reliably.

The input might be a transcript in which a shipper asks about a lane, a delivery window, and a quote. The useful output is not a paragraph. It is a small record that another process can inspect:

classification = { " intent ": " quote_request ", " urgency ": " normal ", " follow_up_required ": True, " confidence ": 0.86, " evidence ": " The shipper asked for a quote on the Chicago-to-Dallas lane. " } Enter fullscreen mode Exit fullscreen mode The schema is the decision record. It says what values are legal and which fields are mandatory. A JSON parser answers one narrow question: “Can these characters be decoded as JSON?” That is useful, but it does not answer whether urgency is a permitted value, whether confidence is a number, or whether the record contains the fields needed by the CRM workflow.

Prompt parsing usually starts with an instruction such as “return intent | urgency | next step.” The parser then splits on a delimiter, trims whitespace, and hopes the model follows the layout. One extra sentence, a translated label, or a delimiter inside the evidence can move every value into the wrong column. The result may still look plausible to a reader. That is the dangerous case.

The four invariants create separate failure boundaries:

These rules do not make a classification correct. They make malformed and out-of-contract classifications visible. That difference matters when an automated tag becomes a sales task.

Consider a call where the shipper says, “The delivery is late, but I also need a quote for next month.” A single-label classifier has a domain decision to make; a parser has a string decision to make. If the response contains service_issue first and quote_request second, a delimiter-based consumer may silently keep the first value. A schema can reject an array when the contract expects one intent, but the team still has to decide whether to add multi-intent support, choose a primary intent, or send the call to review. The validation failure is useful because it puts that domain choice in front of the owner instead of burying it in a split function.

A prompt is an instruction, not a type system. It can improve the distribution of likely answers, but a downstream service still needs to establish what it received. Delimiter parsing adds another grammar whose edge cases are easy to underestimate: missing columns, repeated separators, an empty value, a quoted separator, or an answer wrapped in explanatory prose.

I started by thinking a short pipe-delimited format would be easier for a small classifier. Later I found the real cost was not splitting the string; it was deciding what to do when the split produced five fields, when a label had a different spelling, or when an apparently numeric score was high. The parser had quietly become a second, undocumented schema. It was harder to review than a declared contract.

There is also a retry problem. If parsing fails, should the application resend the same transcript, ask for a shorter answer, or treat the call as unclassified? Those choices affect duplicate CRM tasks and audit history. A schema-backed pipeline can record the exact validation error and preserve the original response. A prompt parser often turns all of these cases into a generic “parse failed” branch.

This is why structured output should be treated as a boundary in the application, not as a formatting preference in the prompt. The model remains probabilistic. The consumer becomes deterministic about what it will accept.

The critical path is deliberately boring: classify, validate, normalize only where the contract allows it, then perform a side effect. Validation must happen before a CRM write, and the original model response should remain available for diagnosis without being mistaken for trusted data.

from json import loads from jsonschema import ValidationError, validate CLASSIFICATION_SCHEMA = { " type ": " object ", " required ": [ " intent ", " urgency ", " follow_up_required ", " confidence ", " evidence " ], " additionalProperties ": False, " properties ": { " intent ": { " type ": " string ", " enum ": [ " quote_request ", " status_question ", " service_issue ", " other " ], }, " urgency ": { " type ": " string ", " enum ": [ " low ", " normal ", " high " ]}, " follow_up_required ": { " type ": " boolean " }, " confidence ": { " type ": " number ", " minimum ": 0, " maximum ": 1 }, " evidence ": { " type ": " string ", " minLength ": 1 }, }, } def classify_call ( transcript: str ) -> dict: raw_text = call_classifier ( transcript ) try: result = loads ( raw_text ) validate ( instance = result, schema = CLASSIFICATION_SCHEMA ) except ( ValueError, ValidationError ) as error: store_for_review ( transcript = transcript, raw_response = raw_text, reason = str ( error )) raise UnclassifiedCall from error if result [ " confidence " ] < 0.70: store_for_review ( transcript = transcript, raw_response = raw_text, reason = " low confidence " ) return result crm_action = to_crm_action ( result ) write_crm_action ( crm_action ) return result Enter fullscreen mode Exit fullscreen mode The endpoint names in this example are intentionally generic. The important properties are ordering and ownership. call_classifier produces an untrusted string. validate establishes the contract. to_crm_action is the only place that translates a classification into a side effect. A low-confidence result can remain visible without being allowed to create an automatic follow-up.

In production, I would also give each transcript an idempotency key. A retry after a timeout must not create two “call shipper” tasks merely because the first write was uncertain. That key is an application concern, separate from JSON Schema, but structured output makes the boundary explicit enough to test.

The schema itself deserves change control. Adding a new intent is a data migration for dashboards and routing rules, even if it looks like one line of JSON. Version the contract, keep fixtures for every accepted label, and include malformed responses in the test suite. Test the negative space: missing evidence, an unknown intent, a string confidence, an extra property, and a valid record with a low score.

The audit record should make a failed attempt reconstructable without making the transcript broadly visible: keep the contract version, the validation error, the raw response in an access-controlled store, the retry decision, and the eventual human disposition. If a timeout occurs after validation but before the CRM write, the idempotency key should let the writer check for an existing action before retrying. If the classifier is called again after a schema change, retain both schema versions so an analyst can explain why the same transcript produced different routing. This is more storage and operational discipline than a delimiter parser needs, but it is the discipline required when a tag controls work.

I would reject “parse whatever the prompt returns” for automated CRM actions. It optimizes for a quick first demo and pushes complexity into hidden coercions: converting yes to true, mapping urgent to high, or silently dropping an unrecognized field. Those coercions make dashboards look clean while weakening the audit trail.

The rejected option is still valid in two situations. First, a human is reading the output and no machine action depends on its shape. Second, the output is exploratory material that will be thrown away after analysis. In both cases, the application can tolerate more variation because a person or a later explicit step owns the interpretation.

A schema is not a substitute for domain review. other may be valid syntactically and still useless operationally. A confidence value may be well-formed without being calibrated. The contract should therefore capture evidence and route ambiguous records to a queue whose service level the team has agreed on. Do not promote every valid object to an automated decision.

The trade-off is real: a schema adds design work, validation code, fixtures, and a migration decision whenever the label set changes. It can also reject a response that contains useful information in an unexpected shape. That cost is justified for a CRM write, where a false task or incorrect tag is durable operational data. It is unnecessary overhead for a scratchpad summary that a person will read and discard.

This approach is also a poor fit when the output is inherently open-ended, such as a long narrative for an analyst. In that case, constrain only the metadata that downstream systems need and leave the prose free-form. Forcing every sentence into a closed schema can make the interface less useful without improving the decision.

Measure rejected records by reason, not only by overall success rate. A spike in unknown intents can mean the label set is stale. A spike in missing fields can mean a prompt or runtime change. A spike in low-confidence classifications can indicate transcript quality, a new call pattern, or a decision boundary that needs review.

Keep the raw transcript access-controlled because sales calls can contain personal and commercial information. Store the schema version, validation result, model response metadata that your runtime exposes, and the final CRM action together in an audit record. Redact or minimize transcript content in ordinary logs. Delivery and retention policies belong in the system design; they should not be improvised inside an exception handler.

The practical rule is short: use a schema when another system must trust the shape, and use prompt parsing only when a human or a disposable experiment can absorb the ambiguity. For logistics call tagging, that boundary is the difference between a useful classification and an automated CRM mistake that looks like clean data.