OpenAI's Inline Moderation Scores in Every API Response: What the Zero-Latency Safety Signal Means for Multi-Provider Routing Teams

OpenAI's June 4 API changelog added something engineering teams have wanted since the standalone moderation endpoint launched: the ability to request omni-moderation-latest scores for both the user input and the model's generated output in a single API call, with no extra round-trip. The change applies to both the Responses API and the Chat Completions API.

For teams building or operating multi-provider routing layers, this is more than a convenience feature — it's a new signal your gateway can now embed directly in the generation path.

What happened

Starting June 4, 2026, you can pass a moderation object in any generation request:

{
  "model": "gpt-5.5",
  "input": [{"role": "user", "content": "..."}],
  "moderation": {"model": "omni-moderation-latest"}
}

The response includes response.moderation.input and response.moderation.output, each containing flagged, categories, and category_scores — the same structure as the standalone moderation endpoint. Key details:

• Free: The moderation model adds no cost to the request.
• Both directions: You get scores for the user prompt and the model's output, so you can catch problematic generations even when the input was clean.
• Tool-call aware: Moderation covers tool-call arguments and outputs that appear in conversation content. It does not cover tool names, schemas, or response-format schemas.
• Streaming caveat: Moderation results arrive after the full output — they are not included in partial deltas.
• Failure handling: If moderation can't complete, the field contains an error object instead of scores.

The moderation model runs in parallel with generation on OpenAI's side, so the latency overhead is near-zero in practice.

Why it matters for AI engineering teams

Until now, teams that wanted to moderate API output had two choices: (a) make a separate /moderations call after generation, adding a round-trip of ~200-400ms, or (b) run their own classifier pipeline. Option (a) was fast enough for batch workflows but painful for real-time chat and coding-agent use cases. Option (b) required maintaining a separate model deployment.

Inline moderation collapses both paths into one API call. For OpenAI-only setups, the migration is trivial — add one JSON field. But for multi-provider routing architectures, the implications are more nuanced.

The router/operator angle

This change introduces a provider-specific signal that doesn't exist on competing endpoints today, and that creates three concrete decisions for routing-layer operators:

1. Safety-signal fragmentation across providers. OpenAI now returns structured moderation scores inline. Anthropic returns a stop_reason: "refusal" with no category breakdown. DeepSeek, Qwen/DashScope, and most domestic Chinese providers return neither — they either block silently or return an error. If your routing layer needs to enforce a consistent content-safety policy, you cannot assume every provider gives you the same data. The question becomes: do you normalize a "safety score" from heterogeneous signals (refusal reason codes, inline moderation, your own classifier), or do you run a provider-agnostic safety pass on every output regardless?

2. Post-generation routing decisions. With inline scores, your gateway can now reroute after generation. If GPT-5.5 returns a flagged output (violence score > 0.8, for example), the gateway could automatically retry with a stricter system prompt, switch to a different model tier, or escalate to human review — all within the same request lifecycle. This is a new pattern that didn't have a clean API primitive before.

3. Audit trail and compliance alignment. Inline moderation gives you a per-request, per-response safety audit record without an extra API call. For enterprise teams operating under EU AI Act or equivalent governance frameworks, this is the cheapest path to a defensible audit log: every generation gets a safety score attached at the API layer, with zero additional latency or cost.

The catch: these scores only work if you route to OpenAI directly. If your gateway proxies through a provider that doesn't return inline moderation, you get nothing. That makes the feature a soft provider lock-in signal — not because it's proprietary, but because the alternative providers haven't shipped an equivalent yet.

What TheRouter users should watch or try

If your routing policy includes content-safety gates, the new inline moderation primitive is worth integrating when your traffic routes to OpenAI. The key question is whether your gateway should treat the moderation object as an optional enrichment field it passes through, or whether it should normalize safety signals across providers into a unified policy layer.

For teams using TheRouter's OpenAI-compatible endpoint: the moderation parameter passes through to the upstream provider, so you get the scores when the upstream is OpenAI. When the upstream is a provider that doesn't support inline moderation, the field is silently ignored — no error, but no scores either. This is worth monitoring: as more providers add equivalent primitives, your routing layer becomes the natural place to aggregate them.

What to watch:

• Whether Anthropic, Google, or DashScope add an equivalent inline safety signal in the next 6 months.
• Whether your compliance team requires per-response safety scores — if so, inline moderation only covers one provider.
• Whether OpenAI extends the moderation model to cover audio inputs (currently text and images only).