Model Fallbacks

Automatic failover across models and providers

Use the models array to define a fallback chain in priority order. TheRouter.ai will attempt the first model, then automatically retry the next model when the route fails because of downtime, rate limits, or moderation refusal.

How failover is applied

Fallbacks are evaluated at request time. The response model field always shows the model that ultimately served the request.

Basic fallback chain

Provide model IDs in priority order. TheRouter.ai stops at the first successful completion.

TypeScript

const response = await fetch('https://api.therouter.ai/v1/chat/completions', {
  method: 'POST',
  headers: {
    Authorization: 'Bearer <THEROUTER_API_KEY>',
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    models: [
      'anthropic/claude-sonnet-4.5',
      'openai/gpt-5-mini',
      'google/gemini-3-flash-preview',
    ],
    messages: [{ role: 'user', content: 'Summarize this document.' }],
  }),
});

Using with the OpenAI SDK

Keep your primary model and pass fallback candidates throughextra_body.models.

TypeScript

import OpenAI from 'openai';

const client = new OpenAI({
  baseURL: 'https://api.therouter.ai/v1',
  apiKey: process.env.THEROUTER_API_KEY,
});

const completion = await client.chat.completions.create({
  model: 'openai/gpt-5',
  extra_body: {
    models: ['anthropic/claude-sonnet-4.5', 'google/gemini-3-flash-preview'],
  },
  messages: [{ role: 'user', content: 'Draft a release note from these bullet points.' }],
});

Route-level control

Combine model fallbacks with provider routing to control failover behavior per request.

JSON

{
  "models": [
    "anthropic/claude-sonnet-4.5",
    "openai/gpt-5-mini"
  ],
  "provider": {
    "sort": "latency",
    "allow_fallbacks": true
  },
  "messages": [
    { "role": "user", "content": "Classify this support ticket." }
  ]
}