Structured Output Across LLM API Providers: JSON Mode, JSON Schema, and What Each Provider Actually Supports (2026)

Getting an LLM to return valid JSON sounds simple until you try it across five providers. One gives you strict schema enforcement at the token level. Another supports json_object but not json_schema. A third uses a completely different parameter name. And when you route the same request through a gateway, the structured output parameter may or may not survive the translation.

We route structured-output requests across OpenAI, Anthropic, DashScope, DeepSeek, and SiliconFlow daily. This guide documents exactly what each provider supports, where the incompatibilities are, and how to build a production pipeline that gets reliable JSON regardless of which provider handles the request.

OpenAI-compatible means a provider exposes a chat-completions endpoint whose request and response shape matches the OpenAI API contract closely enough that an unmodified OpenAI SDK call works against it after swapping three values: API key, base URL, and model name. The minimum surface in practice is POST /v1/chat/completions with messages, model, and an OpenAI-shaped streaming response.

TL;DR Comparison Table

The critical takeaway: only OpenAI and Anthropic offer schema-guaranteed structured output via constrained decoding. DashScope, DeepSeek, and SiliconFlow support json_object mode — valid JSON syntax is guaranteed, but the schema (field names, types, nesting) is not enforced at the token level. You get valid JSON; you do not get guaranteed schema compliance.

How Structured Output Works Under the Hood

Before diving into provider specifics, it helps to understand why json_schema is fundamentally different from json_object.

JSON Object mode (type: "json_object") tells the model to output valid JSON. The provider constrains token generation so the output is parseable JSON — opening braces match closing braces, strings are properly quoted, etc. But the model can return any valid JSON structure. If you asked for {"name": string, "age": number}, you might get {"full_name": "Alice", "years_old": 25}. Valid JSON, wrong schema.

JSON Schema mode (type: "json_schema") uses constrained decoding with a finite state machine that tracks position within your provided schema. At each token, the model can only generate tokens that are valid at that position in the schema. The result is guaranteed to match your schema — not just valid JSON, but the exact structure you specified.

This distinction matters enormously for production systems. With json_object, you still need application-level validation. With json_schema, the provider handles it.

OpenAI: The Full Structured Output Stack

OpenAI offers the most complete structured output implementation. Two modes are available through the response_format parameter.

JSON Object Mode (Legacy)

from openai import OpenAI

client = OpenAI()

response = client.chat.completions.create(
    model="gpt-5.6",
    messages=[
        {"role": "system", "content": "Extract the event details. Return JSON."},
        {"role": "user", "content": "Alice and Bob meet at noon on Friday for lunch."}
    ],
    response_format={"type": "json_object"}
)

# Valid JSON guaranteed, but schema is not enforced
data = json.loads(response.choices[0].message.content)

You must include the word "json" somewhere in your messages — otherwise the API returns an error. The output is valid JSON but may not match any particular schema.

JSON Schema Mode (Recommended)

from openai import OpenAI
from pydantic import BaseModel

client = OpenAI()

class CalendarEvent(BaseModel):
    name: str
    date: str
    participants: list[str]

response = client.responses.parse(
    model="gpt-5.6",
    input=[
        {"role": "system", "content": "Extract the event information."},
        {"role": "user", "content": "Alice and Bob are going to a science fair on Friday."}
    ],
    text_format=CalendarEvent,
)

event = response.output_parsed  # CalendarEvent instance, schema-guaranteed

Key details:

• Constrained decoding — the model literally cannot generate tokens that violate the schema
• Supported models — GPT-4o and later, including GPT-5.6
• Schema features — nested objects, arrays, enums, optional fields, anyOf/allOf
• Refusal handling — if the model refuses, response.refusal is set; output_parsed is None
• Streaming — works with streaming; chunks assemble into schema-valid JSON

Schema restrictions: all fields must be required (use nullable types for optional fields), additionalProperties must be false, and recursive schemas have a depth limit.

Source: OpenAI Structured Outputs Guide (retrieved 2026-08-04)

Anthropic Claude: output_config with Grammar-Based Enforcement

Anthropic takes a different approach. Instead of extending response_format, Claude uses a dedicated output_config parameter with grammar-based constrained decoding.

import anthropic

client = anthropic.Anthropic()

response = client.messages.create(
    model="claude-opus-4-6",
    max_tokens=1024,
    messages=[
        {"role": "user", "content": "Extract entities from: The meeting is at Google HQ on Tuesday."}
    ],
    output_config={
        "format": "json",
        "schema": {
            "type": "object",
            "properties": {
                "entities": {
                    "type": "array",
                    "items": {
                        "type": "object",
                        "properties": {
                            "name": {"type": "string"},
                            "type": {"type": "string", "enum": ["person", "org", "location", "date"]},
                            "confidence": {"type": "number", "minimum": 0, "maximum": 1}
                        },
                        "required": ["name", "type", "confidence"]
                    }
                }
            },
            "required": ["entities"]
        }
    }
)

Key differences from OpenAI:

• Parameter name — output_config, not response_format. This is not OpenAI-compatible.
• Grammar resets between sections — when using extended thinking, the grammar applies only to the final response, not the thinking block. Claude can reason freely, then produce structured output.
• No json_object mode — Anthropic does not support the simpler "just give me valid JSON" mode. You either provide a full schema or use tool-use as a workaround.
• Incompatibilities — citations and prefix-filling (assistant message prefill) are incompatible with output_config.
• Supported models — Claude Opus 4 and Claude Sonnet 4.5 and later.

The tool-use-as-schema workaround remains available: define a "tool" whose input schema matches your desired output, force tool_choice to that tool, and parse the tool_use content block. This works on all Claude models that support tool use but adds latency and complexity.

Source: Anthropic Structured Outputs blog (retrieved 2026-08-04)

DashScope (Qwen): json_object Mode via OpenAI-Compatible API

DashScope supports structured output through its OpenAI-compatible endpoint, but only json_object mode — no json_schema strict enforcement.

from openai import OpenAI

client = OpenAI(
    api_key="your-dashscope-key",
    base_url="https://dashscope.aliyuncs.com/compatible-mode/v1"
)

response = client.chat.completions.create(
    model="qwen-plus",
    messages=[
        {"role": "system", "content": "Extract the user's name and age. Return JSON."},
        {"role": "user", "content": "Hi, I'm Alex Brown, 34 years old."}
    ],
    response_format={"type": "json_object"}
)

# Valid JSON, but schema is not enforced
data = json.loads(response.choices[0].message.content)

Key details:

• Supported models — Qwen-Max, Qwen-Plus, Qwen-Flash, Qwen-Turbo, Qwen-Coder, Qwen-Long (all in non-thinking mode). Multimodal models (Qwen-VL, Qwen-Omni) also supported.
• "json" in prompt required — the word "JSON" (case-insensitive) must appear in your system or user message, otherwise the API returns an error.
• Thinking mode caveat — models in thinking mode accept response_format: json_object without error, but some models may return content that is not strictly valid JSON when thinking mode is active.
• No json_schema — DashScope does not support type: "json_schema" with a strict schema. If you pass it, the API may ignore it or error.

The practical impact: if you route a request with response_format: {type: "json_schema", json_schema: {...}} to DashScope, you need to either downgrade to json_object or handle the incompatibility. The JSON will be valid; the schema compliance depends on the model following your prompt instructions.

Source: DashScope Structured Output docs (retrieved 2026-08-04)

DeepSeek: json_object Mode Only

DeepSeek's structured output support mirrors the simpler json_object approach.

from openai import OpenAI

client = OpenAI(
    api_key="your-deepseek-key",
    base_url="https://api.deepseek.com"
)

response = client.chat.completions.create(
    model="deepseek-v4-pro",
    messages=[
        {"role": "system", "content": "Parse the question and answer into JSON format."},
        {"role": "user", "content": "What is the highest mountain? Mount Everest."}
    ],
    response_format={"type": "json_object"}
)

data = json.loads(response.choices[0].message.content)

Key details:

• Supported models — DeepSeek V4 Pro, DeepSeek V4 Flash
• "json" in prompt required — same as OpenAI's json_object mode
• No json_schema — only json_object is supported
• Reasoning models — DeepSeek-R1 and reasoning modes have limited json_object support; the reasoning tokens may interfere with JSON output

Source: DeepSeek JSON Output docs (retrieved 2026-08-04)

SiliconFlow: OpenAI-Compatible json_object

SiliconFlow provides json_object support through its OpenAI-compatible API, though availability varies by model.

from openai import OpenAI

client = OpenAI(
    api_key="your-siliconflow-key",
    base_url="https://api.siliconflow.cn/v1"
)

response = client.chat.completions.create(
    model="Qwen/Qwen3-8B",
    messages=[
        {"role": "system", "content": "Return the answer as JSON."},
        {"role": "user", "content": "List the top 3 programming languages."}
    ],
    response_format={"type": "json_object"}
)

Key details:

• Model-dependent — not all models on SiliconFlow support response_format. Check the model card.
• OpenAI-compatible — uses the same response_format parameter as OpenAI
• No json_schema — only json_object mode
• Including "json" in prompt — recommended for consistent results

Source: SiliconFlow Chat Completions API (retrieved 2026-08-04)

The Gotchas: What Breaks in Multi-Provider Routing

When you route structured-output requests across providers — whether through TheRouter or any gateway — several incompatibilities surface.

1. json_schema Downgrade

If your request uses response_format: {type: "json_schema", json_schema: {...}} and the request routes to DashScope or DeepSeek, the strict schema enforcement is lost. The gateway can downgrade to json_object and inject the schema into the system prompt, but this is best-effort — the model may or may not follow the schema.

Mitigation: Add application-level JSON Schema validation after every response, regardless of provider. Libraries like jsonschema (Python) or ajv (JavaScript) add negligible latency.

2. Anthropic Parameter Translation

Anthropic uses output_config instead of response_format. A gateway routing to Claude must translate the parameter — and handle the fact that Claude does not support json_object mode at all. The gateway must either provide a full schema or fall back to tool-use.

3. The "json" Prompt Requirement

OpenAI (json_object mode), DashScope, and DeepSeek all require the word "json" in the messages. Anthropic does not. If your prompt does not contain "json" and the request routes to a provider that requires it, the API returns an error.

Mitigation: Always include "json" in your system prompt when using response_format. It does not hurt providers that do not require it.

4. Streaming + Structured Output Interaction

All providers support streaming with structured output, but the behavior differs:

• OpenAI — each chunk is a partial JSON fragment; the assembled result is schema-valid
• Anthropic — streaming with output_config works; grammar constraint applies across the stream
• DashScope / DeepSeek — streaming produces partial JSON chunks; only the final assembled result is guaranteed valid JSON

If you parse streaming chunks incrementally (e.g., for progressive UI updates), you need a partial JSON parser that tolerates incomplete objects.

5. Schema Depth and Complexity Limits

OpenAI's json_schema mode has limits:

• Maximum 5 levels of nesting for anyOf
• Maximum ~100 properties per object (soft limit)
• All properties must be required (use nullable for optional)
• additionalProperties: false is mandatory

DashScope and DeepSeek have no schema enforcement, so there are no schema limits — but there are also no schema guarantees.

Production Checklist

Before shipping structured output to production:

1. Always validate post-generation — even with json_schema mode, validate the response against your schema in application code. This catches edge cases like refusals, truncated responses, and gateway translation errors.

2. Use a retry strategy — if JSON parsing fails, retry with a simpler prompt or a different provider via fallback routing.

3. Set max_tokens high enough — a response that hits the token limit mid-JSON produces invalid JSON. Budget for the full response, especially with deeply nested schemas.

4. Log the raw response — when debugging schema mismatches, the raw response shows whether the issue is the model, the gateway translation, or your parsing code.

5. Test across all providers in your routing pool — a schema that works perfectly on OpenAI may produce different field orderings or naming conventions on DashScope. Your parsing code should handle field order variation.

6. Handle refusals gracefully — OpenAI returns a refusal field. Anthropic returns a stop_reason: "end_turn" with potentially empty content. DashScope and DeepSeek may return a text explanation instead of JSON. Your error handling should account for all patterns.

TheRouter Integration Note

TheRouter routes OpenAI-compatible requests through configured providers, including structured-output requests. When a request includes response_format, TheRouter preserves the parameter for providers that support it (OpenAI, DashScope, DeepSeek, SiliconFlow). For Anthropic, where the parameter name and semantics differ, the translation follows the provider-specific mapping.

If a request specifies json_schema mode and the routed provider only supports json_object, the schema enforcement is provider-dependent. We recommend adding application-level validation as a safety net, regardless of which provider handles the request.

For fallback routing scenarios — where a request fails on one provider and retries on another — structured output parameters are preserved across retries. The fallback provider may have different schema support levels, so the same validation advice applies.

Common Errors and Fixes

FAQ

Q: Can I use json_schema mode with reasoning/thinking models?

It depends on the provider. OpenAI supports json_schema with o-series reasoning models. Anthropic's output_config works alongside extended thinking — the grammar applies only to the final response, not the thinking block. DashScope notes that thinking-mode models may not return strictly valid JSON even with json_object set.

Q: Should I use tool-use or response_format for structured output?

Use response_format (or output_config on Anthropic) when you want the model's response itself to be structured JSON. Use tool-use when you are connecting the model to actual functions or APIs. The function calling guide covers the tool-use approach in detail.

Q: What about Instructor or Outlines for structured output?

Libraries like Instructor (Python/TypeScript) and Outlines (Python) provide framework-level structured output that works across providers. Instructor wraps provider clients and adds Pydantic/Zod schema enforcement with automatic retries. Outlines uses grammar-constrained decoding for local models. Both are production-viable, but they add a dependency layer and may not use the provider's native constrained decoding.

Q: How do structured outputs interact with prompt caching?

On OpenAI, requests with the same json_schema benefit from prompt caching — the schema is part of the cached prefix. On Anthropic, output_config does not interact with cache_control blocks. On DashScope, json_object mode requests cache normally via context caching.

---

Last verified: August 4, 2026. Provider APIs evolve; check official docs for the latest schema support.