LLM Structured Outputs: JSON Schema and Validation

Last updated: July 26, 2026. By Ignacy Kwiecien, founder and editor-in-chief, DecodeTheFuture.org

This is general technical education. Provider features, supported schema keywords, data terms and SDK syntax change over time. Do not send model output directly to a database, payment flow, customer decision or external API without application validation, authorization and monitoring.

LLM structured outputs: the practical definition

An LLM structured output is a response designed to fit a machine readable contract such as an object or array with named fields, types and allowed values. The contract can be expressed in JSON Schema and enforced by a provider feature, a constrained decoder, an SDK, or a combination of those layers. The application can then parse the object into a typed model instead of guessing where an answer ends or stripping Markdown from a paragraph.

That makes a difference whenever model output is consumed by code. An invoice extractor may need vendor, invoice_number, issue_date, currency and total. A support classifier may need one of a finite set of queues. A document review workflow may need a decision, evidence spans and an escalation flag. In each case, the output contract is part of the application interface.

Structured output does not prove that the vendor name is correct, that the total came from the document, or that the classification is fair. A schema valid object can contain an invented amount, an empty evidence list, a stale conclusion or a semantically wrong date. The production pattern is a pipeline, not a single API flag.

The five layers of reliability

1. Provider response handling. Detect transport errors, refusals, incomplete output and token limits before treating the payload as a normal object.
2. Parsing. Convert the response into a JSON value and reject unexpected prose, code fences or malformed syntax.
3. Schema validation. Check object shape, required keys, types, allowed values and additional properties according to the dialect and provider subset.
4. Semantic validation. Apply application rules such as non-negative totals, cross-field arithmetic, source evidence and permitted state transitions.
5. Business decision. Accept, retry a transient failure, ask for review or reject. Do not let a model silently choose the recovery path for an irreversible action.

The layers answer different questions. Did the provider return something complete? Is it valid JSON? Does it match the contract? Does it make sense for this document or user? Is it safe and authorized to use? Combining all of those questions into one Boolean named success makes production debugging harder.

JSON Schema basics for model responses

JSON Schema describes constraints for JSON instances. Common building blocks include object, array, string, number, integer, boolean and null. An object can define properties, a required list and an additionalProperties policy. An enum can constrain a classification to a documented set of values.

A small schema is easier for a model, a provider and a human reviewer to understand. Start from the downstream decision. If a finance workflow only needs a currency, total, evidence quote and review flag, do not ask the model to recreate every formatting detail from an invoice. Every extra field creates another opportunity for ambiguity, missing data or an unsupported schema keyword.

This example is deliberately small. Whether the provider accepts a union, nullable field, extra keyword or a particular draft of JSON Schema is provider specific. A schema generated by a local library may contain features that the provider does not support. Treat provider compatibility as a build check, not as an assumption.

JSON mode versus structured outputs

Feature names are not universal standards. One provider may call a response format structured output, another may use a schema field, and another may expose an input_schema for tool use. Compare the actual endpoint, model, SDK and error surface. Do not assume that a schema accepted in one provider’s example is portable without a compatibility test.

Provider patterns: OpenAI, Anthropic and Gemini

OpenAI documents JSON Schema based structured outputs through response or text format interfaces. The documented subset has support boundaries, and the application must handle refusal or incomplete responses separately from a normal object. Clear key names, descriptions and evaluation examples help both the model and the integration.

Anthropic documents tools with a name, description and input_schema. That schema describes the structured input for a tool use block. It does not authorize the action, verify the arguments against an external system or make the tool safe. The application still needs permission checks, deterministic validation and an explicit result path.

Gemini documents structured response configuration with a schema and a supported subset of JSON Schema. Its documentation also warns that syntactically correct JSON can contain semantically incorrect values. That warning is the right mental model for every provider. Structured format is one layer of a reliable application, not the end of testing.

How to design a schema models can fill

Use names that explain the decision

needs_review is clearer than flag when the field routes a document to a reviewer. invoice_total is clearer than value when a downstream system expects a money amount. Descriptions should explain the meaning, not repeat the field name.

Make optionality explicit

A missing invoice number can mean that the source did not contain one, that the model failed to extract it, or that the document type does not use it. Use null only when the application has a defined meaning for null. Do not ask the model to invent a default merely to satisfy a required field.

Separate evidence from conclusion

If a result triggers a financial workflow, store an evidence quote, page, section or character span where possible. Evidence supports review and debugging. It is not automatically proof. A model can quote a nearby line or select a source that does not support the conclusion, so the application should validate the relationship where the risk justifies it.

Bound lists and categories

Use an enum for a controlled state set and explain what each value means. Bound arrays when the business process has a maximum. If the source can contain an arbitrary number of items, define a paging, truncation or review rule instead of silently discarding the tail.

Structural validation is not semantic validation

Consider an invented invoice from Northwind Services. The source excerpt says: Invoice 1042, issued 2026-06-18. Subtotal 1000.00 USD. Tax 230.00 USD. Total 1230.00 USD. A schema can verify that the response contains strings and numbers. It does not know whether the model copied the total from the source or guessed it.

A typed application model can add rules:

• currency must belong to the application’s supported set.
• total and subtotal must be zero or greater.
• subtotal + tax must equal total within a declared rounding tolerance.
• The issue date must be parseable and within the workflow’s allowed range.
• Evidence must not be empty for fields that trigger a financial decision.
• If the source omits a value, the result must be null or review, not a plausible replacement.

Pydantic, Zod or another validation library can implement those rules. The library checks the rules that developers wrote. It does not independently fact check a vendor name or prove that a quote came from the document. A source comparison, deterministic database lookup or human review may be needed for the final meaning.

A typed validation example

In Python, a typed model can turn structural parsing into a second, explicit application boundary. The example below accepts nullable monetary fields because an invoice may omit them, then checks the arithmetic only when all three values are present. The tolerance is an application decision for decimal handling, not a license to hide a material discrepancy.

The model checks a cross field rule that JSON Schema alone may not express in the form the business needs. It still does not establish that the invoice was authentic, that the currency was read correctly or that the numbers came from the right page. Those questions belong to evidence checks, source comparison and review policy. In a JavaScript or TypeScript application, a Zod schema can play a similar role, but the same boundary applies: types constrain values, while application logic establishes what the values mean.

Keep the provider schema and the application model related but not blindly identical. Pydantic or Zod can generate a JSON Schema, but the generated result may include a dialect, union, format or keyword that a provider endpoint does not support. Add a compatibility test that generates the schema, sends a harmless request or runs the provider schema checker, and fails the build when the supported subset changes. This is especially important when a model upgrade changes SDK parsing or refusal behavior.

Provider compatibility has its own failure modes

Provider documentation should be treated as an implementation contract for a particular endpoint, model family, SDK and API version. A feature name is not enough. Before deployment, record the exact request shape and test a normal response, a refusal, a truncation case, an invalid business value and a provider side schema rejection. Store the provider request identifier with the validation result so a later investigation can distinguish model behavior from an integration bug.

For OpenAI Structured Outputs, object schemas in the supported subset commonly require additionalProperties: false. That setting is visible in the small schema above and means that undeclared keys are not accepted by that contract. It can be a useful safety boundary, but it also means a schema copied from a general JSON Schema validator may need adjustment before it is accepted by the endpoint. Confirm the requirement for the exact API surface and model you use.

Gemini structured output supports documented JSON types such as string, number, integer, boolean, object, array and null, but support is still a subset rather than an unlimited implementation of every JSON Schema vocabulary. Very large or deeply nested schemas may be rejected or become difficult for the model and application to maintain. Prefer a compact contract with a review path for omitted detail, and test the maximum realistic nesting and response size before launch.

Anthropic tool use adds a different boundary. An input_schema can make the arguments to a tool predictable, but the application must still authorize the caller, verify the resource, enforce idempotency and decide whether an action is reversible. Never treat a valid tool call as permission to send money, change an account or expose private data. The structured object is an input to a policy decision, not the policy decision itself.

These provider differences belong in the production architecture rather than in prompt folklore. The AI architecture for production guide covers the surrounding boundaries, while this article focuses on the output contract inside that system. If the task has sensitive data or external actions, pair schema validation with the controls described in the AI agent security guide.

Failure handling without infinite retries

Retries create cost and latency. They can also repeat the same semantic error. Record a failure category, attempt count, model version, schema version and evaluator result. If a repair request is used, keep the original response and apply the same validation pipeline to the repaired object. A repair step should not bypass business rules.

Testing structured outputs before production

Build a fixture set before connecting the output to production systems. Include clean examples, missing fields, conflicting values, long documents, unusual dates and currencies, empty lists, ambiguous classifications and adversarial instructions inside source text. The goal is not to make the test set look easy. The goal is to expose the cases that cause an application to make a wrong decision.

Measure parse success, structural validation rate, semantic acceptance rate, field level accuracy, evidence support, abstention or review rate, median and tail latency, token cost, retry cost and failure category distribution. An average pass rate can hide a small category of critical failures. Separate critical fields from cosmetic fields.

The JSONSchemaBench paper is useful because it separates several dimensions that are often collapsed into one score: constraint compliance, schema coverage, efficiency and output quality. Use that framing to design your own task evaluation, not as a universal provider leaderboard. A system can comply with a schema quickly and still produce weak evidence, or produce high quality answers with a contract that covers too little of the downstream decision. OpenAI's evaluation material also supports keeping representative fixtures and held out cases rather than relying on a single successful demonstration.

Record a failure taxonomy, not only a pass percentage. A provider schema rejection is different from a parser error. A valid object with low confidence or missing evidence is different from a transient timeout. A review route is different from a retry. When the categories are explicit, the team can set separate budgets and owners. The AI agent ROI calculator is useful for thinking through the cost of retries, human review and latency, but a low cost is not a reason to accept an unsafe decision.

Our AI agent observability guide explains how traces can connect an output to its prompt, model, retrieved context, validation result and business outcome. For broader evaluation design, see the AI benchmark guide, but keep the fixtures here local to the task. A public benchmark cannot replace a representative private test set.

When structured outputs are the wrong tool

A strict object is not always the best product interface. If the user needs a human readable explanation, forcing every sentence into fields can make the answer worse. If the schema changes on every request, a rigid contract may add more failure modes than it removes. If the team has no semantic validator or review path, structural output can create false confidence.

Use a simpler response when the task is exploratory, creative or not consumed by code. Use a typed contract when downstream software needs bounded fields. Use retrieval or fine tuning for knowledge and behavior problems, not as substitutes for output validation. The RAG versus fine tuning guide covers that separate customization decision.

Production checklist

FAQ