Send Chat Completion Request

Request Parameters

Reasoning across models

The reasoning object is translated to each model family's native reasoning surface, so the same request behaves consistently no matter which model serves it (including fallback):

The response field usage.reasoning_tokens reports reasoning/thinking token usage when the model provides it. An explicit 0 means the model reported zero reasoning tokens; an absent field means the model reported nothing.

Gemini OpenAI-compat base URL

POST /v1beta/openai/chat/completions is an alias of this endpoint with identical schema, authentication, and billing. It exists for clients of Gemini's OpenAI-compatibility layer, whose base-URL convention ends in /v1beta/openai/ — point such clients at https://api.therouter.ai/v1beta/openai and they work unchanged.

OpenRouter Routing Extensions

Examples (Non-Streaming)

curl -X POST https://api.therouter.ai/v1/chat/completions
  -H "Authorization: Bearer $THEROUTER_API_KEY"
  -H "Content-Type: application/json"
  -d '{
    "model": "anthropic/claude-sonnet-4.5",
    "messages": [{"role": "user", "content": "Write a haiku about routing."}],
    "temperature": 0.7,
    "max_tokens": 180,
    "top_p": 1,
    "presence_penalty": 0,
    "frequency_penalty": 0,
    "stop": ["\n\n"],
    "seed": 42,
    "models": ["anthropic/claude-sonnet-4.5", "openai/gpt-4o"],
    "provider": {"allow_fallbacks": true, "order": ["openai-api"]},
    "route": "fallback",
    "stream": false
  }'

Examples (Streaming)

curl --no-buffer -X POST https://api.therouter.ai/v1/chat/completions
  -H "Authorization: Bearer $THEROUTER_API_KEY"
  -H "Content-Type: application/json"
  -d '{
    "model": "openai/gpt-4o",
    "messages": [{"role": "user", "content": "Stream a short answer."}],
    "stream": true,
    "stream_options": {"include_usage": true}
  }'

Streaming Behavior

data: {"id":"chatcmpl_...","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"Hello"},"finish_reason":null}]}

data: {"id":"chatcmpl_...","object":"chat.completion.chunk","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: {"id":"chatcmpl_...","object":"chat.completion.chunk","choices":[],"usage":{"prompt_tokens":10,"completion_tokens":4,"total_tokens":14}}

data: [DONE]

Usage is emitted as a separate final chunk with choices: [] only when stream_options.include_usage is true.

Keepalive frames before the first token

A streaming response is opened as soon as the request has been routed, before the upstream model has produced anything. Until the first chunk arrives the gateway writes an SSE comment line every 15 seconds (the first one immediately):

: keepalive

: keepalive

data: {"id":"chatcmpl_...","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"Hello"},"finish_reason":null}]}

Comment lines are ignored by every SSE parser, including the OpenAI SDKs' stream iterators; no code change is needed. They keep proxies and load balancers from closing an idle connection while a reasoning model is still thinking, so long reasoning turns complete instead of ending in a 5xx from an intermediary. Errors that happen before routing (unknown model, insufficient balance) are still returned as plain HTTP errors; an error after the stream has been opened arrives as a single data: frame with "object": "error" followed by data: [DONE].

Non-streaming requests: 100-second first-byte limit

A non-streaming request receives no bytes until its JSON body is ready. The edge in front of the gateway closes a connection that has produced no byte for 100 seconds and returns 524 to you, whatever the gateway does afterwards. This is an edge limit, not a gateway guarantee: for reasoning models and long prompts use stream: true, where the keepalive removes it. A request cut this way is charged for its prompt only, never for output you did not receive.

What you are charged when a stream is not delivered in full

If your client disconnects, or the edge closes the connection, before the stream completes, the request is charged for the prompt plus the output chunks that were actually written to your connection — nothing the model generated after that. The request log and CSV export show why in delivery_outcome:

An empty delivery_outcome means the response was delivered in full and charged as usual.

Error Shape

{
  "error": {
    "message": "Missing required parameter: model",
    "type": "invalid_request_error",
    "code": 400,
    "param": "model"
  }
}

Error bodies do not include an error.metadata object. Request IDs are returned via x-request-id response header.

Response Headers