Structured Outputs
Reliable JSON contracts for downstream services and automations
TheRouter.ai supports two output modes: `json_object` for simple object-only formatting and `json_schema` for strict field-level contracts. Prefer `json_schema` for production pipelines where deterministic parsing matters.
{
"model": "openai/gpt-4.1",
"messages": [{ "role": "user", "content": "Return order summary JSON." }],
"response_format": { "type": "json_object" }
}Model support
Structured outputs are supported by many OpenAI, Anthropic, Gemini, and select open-source models. Filter model capability metadata for `supported_parameters=structured_outputs` when choosing fallback chains.
Claude and `json_object`
Claude models have no native `json_object` mode. On the Claude models listed below, TheRouter.ai adds an instruction asking for a single JSON object and then validates the reply before returning it. This is instruction-based with validation, not constrained decoding.
Supported (as of 2026-10-19): `anthropic/claude-fable-5`, `anthropic/claude-fable-5-1`, `anthropic/claude-haiku-4.5`, `anthropic/claude-haiku-5-5`, `anthropic/claude-opus-4.5`, `anthropic/claude-opus-4.6`, `anthropic/claude-opus-4.7`, `anthropic/claude-opus-4.8`, `anthropic/claude-opus-5`, `anthropic/claude-opus-5-5`, `anthropic/claude-sonnet-4.5`, `anthropic/claude-sonnet-4.6`, `anthropic/claude-sonnet-5`, `anthropic/claude-sonnet-5-5`. On other Claude models (`anthropic/claude-opus-4`, `anthropic/claude-opus-4.1`, `anthropic/claude-opus-4.5-20251101`, `anthropic/claude-sonnet-3.7`, `anthropic/claude-sonnet-4`), a `json_object` request returns HTTP 400 `unsupported_feature`; use one of the models above or `json_schema` where that model supports it.
- Without streaming, you receive either a JSON object or HTTP 422 with error code `json_object_output_invalid`. The failure is not charged; retry the request or switch to `json_schema`.
- If the whole reply is wrapped in a single Markdown code fence, the fence is removed. No other change is made to the model output.
- With `stream: true`, the object arrives in a single content chunk at the end of generation (keepalive frames are sent until then), so there is no incremental output. The response has already started with HTTP 200, so a validation failure arrives as an SSE error event (
"object": "error", with `error.code` set to `json_object_output_invalid`) followed by `data: [DONE]`, not as an HTTP 422. It is not charged either. - Separately from validation: if the upstream stream ends before the reply is complete and the retries are exhausted, none of the buffered content is sent. Instead, after HTTP 200, an SSE error event arrives with `error.code` set to
"502", followed by `data: [DONE]`. That failed request is not charged. If a retry succeeds, you receive the validated object and the request is billed normally. - Turns that end in a tool call (
finish_reason: "tool_calls") or are refused (`content_filter`) are returned as-is and are not validated. - A reply truncated by the token limit (
finish_reason: "length") is returned unvalidated only if its visible text begins with{(optionally after an opening code fence). If the limit is reached during thinking, leaving no visible text, or after the model started with prose, the request fails with `json_object_output_invalid` instead. Leave enough `max_tokens` for both reasoning and the object. - Validation failures are not charged, so they are capped per account: after more than 20 `json_object_output_invalid` failures on Claude models within one UTC hour, further `json_object` requests to Claude models return HTTP 429 (`rate_limit_error`, code `json_object_invalid_rate_limited`) until the next UTC hour begins. `json_schema` requests and other models are not affected. The count includes streamed replies rejected for exceeding the 2 MiB stream limit below, and applies whenever a `json_object` request is served by a Claude model, including a fallback in a `models` list.
- With `stream: true` the reply is held until it is complete, up to 2 MiB of visible text, reasoning and tool-call arguments combined. A longer reply fails with `json_object_output_invalid` (not charged), even on a tool-call turn. Use `json_schema` for very large structured outputs.
- For guaranteed structure, use `json_schema`: on Claude it is enforced by constrained decoding and streams normally.