Multimodal API Routing Architecture 2026: How Gateways Handle Chat, Image, Video, and TTS Under One Endpoint

The 30-second answer: if you need every modality (chat, image gen, video, TTS, STT, embeddings) behind one API key and one billing relationship, OpenRouter is the most complete single-endpoint gateway today. If you are building in China and need Qwen models plus CosyVoice TTS plus Wanx image generation, DashScope gives you the broadest native multimodal coverage under one Alibaba Cloud account — but with separate task-specific API surfaces, not one unified endpoint. Gemini takes a third approach: instead of aggregating third-party models, it extends its own chat endpoint with built-in tools (code execution, computer use, Google Search) that blur the line between "modalities" and "capabilities." We think production systems should not force all modalities through a single gateway. Route text and image generation through the gateway that fits each workload, and use TheRouter to unify billing and fallback across providers regardless of modality.

TL;DR comparison table

OpenAI-compatible means a provider exposes a chat-completions endpoint whose request and response shape matches the OpenAI API contract closely enough that an unmodified OpenAI SDK call works against it after swapping three values: API key, base URL, and model name. The minimum surface in practice is POST /v1/chat/completions with messages, model, and an OpenAI-shaped streaming response.

How each gateway routes multimodal requests

OpenRouter: one base URL, model string determines modality

OpenRouter ships a single base URL (https://openrouter.ai/api/v1) that serves every modality. You switch between them by changing the model string and endpoint path. Their July 2026 blog post "Every Modality Through One API" lays out the architecture: text/chat, vision input, PDF, and audio input all ride /chat/completions with different content types in the message array; image generation, video generation, TTS, STT, and embeddings each get dedicated endpoints under the same base URL.

The key insight is that routing controls — failover, provider preferences, cost/latency sorting, data_collection: "deny" — work identically across modalities. An embeddings call gets the same retry semantics as a chat call. One API key, one bill.

from openai import OpenAI

client = OpenAI(
    base_url="https://openrouter.ai/api/v1",
    api_key="sk-or-...",
)

# Chat completion — same base URL
chat = client.chat.completions.create(
    model="anthropic/claude-sonnet-4",
    messages=[{"role": "user", "content": "Summarize this document."}],
)

# Image generation — same base URL, different endpoint
image = client.images.generate(
    model="openai/gpt-image-1",
    prompt="A router splitting light into different wavelengths",
    size="1024x1024",
)

# TTS — same base URL, different endpoint
speech = client.audio.speech.create(
    model="openai/tts-1",
    input="Multimodal routing is here.",
    voice="alloy",
)

# Embeddings — same base URL, different endpoint
embedding = client.embeddings.create(
    model="openai/text-embedding-3-small",
    input="Multimodal API routing architecture",
)

Trade-offs: OpenRouter aggregates many providers but does not host models itself. Latency depends on the upstream provider. Video generation is asynchronous (submit, poll for result). Not all providers support all modalities — you need to check model capabilities before routing.

DashScope: separate task APIs under one Alibaba Cloud account

DashScope (Alibaba Cloud Model Studio) takes a different approach. Chat completions use an OpenAI-compatible endpoint at https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions (or the newer workspace-specific domains). But image generation (Wanx, Flux), video generation (Wan), TTS (CosyVoice), and embeddings each use DashScope-native task APIs with different request schemas.

The OpenAI-compatible layer covers Qwen LLMs, Qwen-VL, Qwen-Coder, Qwen-Omni, DeepSeek, Kimi, GLM, and MiniMax for chat. But when you need image generation or TTS, you switch to DashScope-native endpoints with a different request format. One API key and one billing account still cover everything, but the code paths diverge.

from openai import OpenAI
import dashscope

# Chat — OpenAI-compatible endpoint
client = OpenAI(
    api_key="sk-...",
    base_url="https://dashscope.aliyuncs.com/compatible-mode/v1",
)
chat = client.chat.completions.create(
    model="qwen-plus",
    messages=[{"role": "user", "content": "Explain multimodal routing."}],
)

# Image generation — DashScope-native API (different SDK, different schema)
from dashscope import ImageSynthesis
result = ImageSynthesis.call(
    model="wanx-v1",
    input={"prompt": "A router splitting light into wavelengths"},
    parameters={"size": "1024*1024", "n": 1},
)

# TTS — DashScope-native API
from dashscope.audio.tts_v2 import SpeechSynthesizer
synthesizer = SpeechSynthesizer(model="cosyvoice-v2", voice="longxiaochun")
audio = synthesizer.call("Multimodal routing is here.")

Trade-offs: You get the broadest Chinese AI model coverage (Qwen, DeepSeek, GLM, Kimi, MiniMax) under one account with China-region low-latency access. But the OpenAI-compatible layer only covers chat — media endpoints require learning the DashScope-native API surface. If you are migrating from OpenAI, only the chat path drops in cleanly.

Gemini: built-in tools extend the chat endpoint

Google takes a fundamentally different approach. Instead of aggregating multiple providers behind one URL, Gemini extends its own generateContent endpoint with built-in tools: code execution, computer use, Google Search grounding, and managed agents. These are not separate modalities with separate endpoints — they are tool declarations in the chat request that the model can invoke during generation.

The Gemini API changelog notes that computer use launched in 2026 with "simplified actions with intents, built-in support for browser, mobile, and desktop environments." Code execution runs Python in a server-side sandbox and returns results inline. These extensions mean a single Gemini call can search the web, execute code, and interact with a browser — capabilities that other gateways would route to separate services.

from google import genai

client = genai.Client()

# Chat with built-in code execution tool
response = client.models.generate_content(
    model="gemini-2.5-flash",
    contents="Calculate the compound interest on $10,000 at 5% for 10 years",
    config={
        "tools": [{"code_execution": {}}],
    },
)

# Chat with built-in Google Search grounding
response = client.models.generate_content(
    model="gemini-2.5-flash",
    contents="What are the latest AI API pricing changes in July 2026?",
    config={
        "tools": [{"google_search": {}}],
    },
)

Trade-offs: Gemini's built-in tools are powerful but locked to Google models. You cannot route a Gemini computer-use request to an Anthropic or OpenAI fallback. There is an OpenAI compatibility layer, but it does not expose built-in tools. If you need multi-provider fallback, Gemini's tool extensions are not portable.

Modality coverage matrix

Pricing comparison

Multimodal pricing varies significantly by modality and provider. Chat token pricing is well-documented; media generation pricing is often per-request or per-second.

The key pricing difference is structural: OpenRouter passes through provider pricing with no markup across all modalities. DashScope and Gemini set their own first-party prices. For chat, this means OpenRouter's price equals the upstream provider's price; for DashScope, the price is whatever Alibaba sets for each Qwen model.

Rate limits

For multimodal workloads, rate limits are per-endpoint and per-model. A chat rate limit and an image generation rate limit are typically counted separately, but this is provider-specific. Check the specific endpoint documentation before designing batch pipelines.

Decision matrix: pick X if you need Y

What this means for routing architecture

The three approaches represent different design philosophies:

1. Aggregation gateway (OpenRouter): Route any modality to any provider through one API. Maximum flexibility, but you depend on the gateway for uptime and you add a hop.

2. Platform-native multimodal (DashScope): One cloud account covers chat, image, video, TTS, embeddings — but with separate API surfaces per modality and limited to Alibaba's ecosystem.

3. Model-native tool extension (Gemini): The model itself handles multiple "modalities" through built-in tools. Powerful for Google-stack teams, but not portable across providers.

For production systems, we think the right answer is usually a combination. Use an aggregation gateway or TheRouter for chat and vision where multi-provider fallback matters. Use platform-native APIs for media generation where latency and regional access matter more than provider diversity. Do not force video generation through a chat gateway just for the sake of having one API key.

The emerging pattern we see: teams start with one gateway for chat (where model switching is frequent), then add direct provider integrations for media generation (where they pick one model and stick with it). The gateway handles the messy part — failover, billing, model deprecation — while direct integrations handle the stable part.

Internal links

• Multimodal Model API Routing Comparison 2026 — our model-level comparison (which model to pick for vision tasks)
• Doubao Seedance Video API Guide — video generation through Volcengine Ark
• DashScope Qwen3.7 Series Complete Guide — DashScope chat API deep-dive
• Aliyun Bailian API Guide — DashScope platform overview
• Gemini 3.5 Flash Computer Use Guide — Gemini built-in tools deep-dive
• TheRouter Quickstart — unified routing and billing across providers

---

Sources retrieved 2026-07-22. Pricing, rate limits, and model availability change frequently. Verify against official provider documentation before making production decisions.