70% of Engineering Teams Now Run Multi-Model AI in Production — What Datadog's 2026 Data Reveals About Routing

Three years ago, most engineering teams picked one AI provider and stayed. Today, the default is a portfolio. Datadog's 2026 State of AI Engineering report — drawing on LLM telemetry from more than a thousand production customers — confirms what many operators have been feeling: multi-model AI routing production is no longer an advanced pattern. It is the baseline.

The question is whether your infrastructure is keeping pace.

What happened

Datadog analyzed LLM call telemetry across its customer base to produce one of the most data-grounded pictures of how AI is actually running in production. The headline numbers are striking:

• More than 70% of organizations now use three or more models. The share running more than six models nearly doubled in the past year.
• OpenAI's share fell from 75% to 63% — not because of absolute decline (Datadog customers using OpenAI more than doubled), but because Google Gemini and Anthropic Claude grew 20 and 23 percentage points, respectively.
• Claude Sonnet 4.6 reached 17% adoption within its first month of release, showing how fast teams absorb new models when the routing infrastructure allows it.
• GPT-4o, officially retired from the ChatGPT UI, still held a 22% share in March 2026 API traces — illustrating how production systems resist forced deprecation.
• Model churn is accelerating. Teams add new releases quickly but retire older models slowly, creating overlapping model portfolios that each carry their own quality, latency, and cost profile.

Why it matters for AI engineering teams

The Datadog data reveals a structural mismatch: teams are adopting multi-model strategies at a faster rate than they are building the governance infrastructure to manage them.

Each additional model in a production fleet introduces compounding overhead:

• Evaluation burden. The same prompt, tool call, or agent workflow can produce different results across models. Teams must maintain evaluation harnesses across every model they run — not just the one they trust most.
• Deprecation pressure. As providers accelerate release cycles, older model versions are retired faster. GPT-4o's UI removal while still sitting at 22% in API traces is a preview of the supply-side churn teams will face repeatedly.
• Cost variability. Different providers price tokens differently, cache differently, and bill async jobs differently. Without unified accounting, multi-model AI routing production creates reconciliation nightmares at billing time.
• Compliance fragmentation. Enterprise teams running models from OpenAI, Google, and Anthropic simultaneously face provider-specific terms, data residency requirements, and audit log formats. Consolidating those signals is a manual problem without the right gateway layer.

The report also confirms a troubling failure mode: framework adoption has nearly doubled year over year (from 9% to 18%), but frameworks accelerate building, not operating. Teams reach production faster, but the operational complexity multiplies with each provider added and each framework version pinned.

The router/operator angle

The Datadog report explicitly names gateway routing as the structural fix: "teams increasingly need to use a modular routing mechanism (such as a gateway service) to manage LLM requests rather than rely on direct model provider API calls throughout their environments."

For operators building or evaluating that gateway layer, the production data suggests three concrete decisions:

1. Route by workload stage, not by team preference. The teams Datadog identifies as pulling ahead treat inference as a pipeline — lightweight models for extraction and tagging, frontier models for synthesis. This tiered routing pattern requires a gateway that can inspect request intent and route accordingly, not a static provider alias.

2. Treat model deprecation as a first-class operational event. GPT-4o's continued 22% API share despite its UI retirement is a warning. Teams without deprecation-aware routing policies will absorb hard failures instead of graceful fallbacks when providers finally force cutover. Every model in your fleet needs a retirement plan encoded in your routing policy before the provider makes it urgent.

3. Make evaluation continuous, not point-in-time. The report identifies that organizations are "quick to add newer models but slower to retire older ones" — which means quality regressions accumulate invisibly. Routing without online evaluation is guesswork at scale. The gate for adding a new model to production should include continuous quality probes, not just a benchmark screenshot.

What TheRouter users should watch or try

If you are running multiple providers through TheRouter's routing layer, the Datadog data gives you a benchmark: 70%+ of production organizations are already in multi-model territory, but the ones closing the operational gap are the ones investing in:

• Explicit routing policies rather than ad hoc model selection per team.
• Unified provider accounting so that cost, latency, and failure rates are visible across the entire model fleet, not buried in per-provider dashboards.
• Fallback chains that handle provider throttling and model deprecation gracefully — before the provider forces the issue.

Check your routing configuration in TheRouter docs to ensure each provider in your fleet has a defined fallback and that your cost accounting captures cross-provider spend in a single ledger. Then review your oldest pinned model versions against provider deprecation timelines before the next round of model retirements hits.

The Datadog report is a useful baseline. Production multi-model AI routing is the norm. The differentiator is whether you are operating it deliberately or just surviving it.