Multimodal Capabilities

Send images, PDFs, audio, and video to compatible TheRouter.ai models

TheRouter.ai supports multimodal input types on the same API surface you already use for text. Mix and match modalities per request to build richer user experiences without provider-specific integration code.

Supported modalities

Images for vision, OCR, and visual reasoning
Image generation for text-to-image and multimodal output
PDFs for document parsing and long-context analysis
Audio for transcription and speech workflows
Video for scene understanding and temporal analysis

Unified request shape

Use /chat/completions with structured content parts in each message.

multimodal-message.json

{
  "model": "google/gemini-2.5-pro",
  "messages": [
    {
      "role": "user",
      "content": [
        { "type": "text", "text": "Summarize this file and image" },
        { "type": "file", "file": { "filename": "report.pdf", "file_data": "data:application/pdf;base64,..." } },
        { "type": "image_url", "image_url": { "url": "https://example.com/chart.png" } }
      ]
    }
  ]
}

Content types by modality

Reference

text         -> { type: "text", text: "..." }
image        -> { type: "image_url", image_url: { url: "..." } }
pdf/file      -> { type: "file", file: { filename: "...", file_data: "..." } }
audio        -> { type: "input_audio", input_audio: { data: "...", format: "wav" } }
video        -> { type: "video_url", video_url: { url: "..." } }

Model compatibility

Not every model supports every modality. Filter candidates using model metadata fields such asinput_modalities and output_modalities.

URL vs base64

Public URLs are best for large assets and lower payload size. Base64 is required for local or private files.

Efficiency

Prefer URLs for large images and videos when the provider supports URL fetching; use base64 when content must stay private.

Models

Images