The Two API Settings That Tripled Agent Performance — And What They Mean for Your Routing Layer

OpenAI published an engineering post this week that starts with a benchmark puzzle and ends with an operator mandate: if your routing layer strips reasoning tokens or falls back to rolling truncation, you are degrading every GPT-5.6 agent run it handles.

The headline finding — retained reasoning and compaction tripled GPT-5.6 Sol's ARC-AGI-3 score from 13.3% to 38.3% while cutting output tokens by 6x — matters less as a benchmark result than as a signal about what the Responses API actually does that Chat Completions does not.

What the two settings do

Retained reasoning is preserved when you pass previous_response_id in the Responses API. Without it, GPT-5.6 Sol's private thinking tokens are discarded after every turn. The model sees the action history but not the reasoning that produced those actions, so it reconstructs its understanding of the task from scratch on each step. With reasoning retained, the model spends less compute per action and builds coherent multi-step strategies.

Compaction replaces rolling truncation. When a Chat Completions conversation hits a context limit, the standard approach drops the oldest messages. Compaction instead summarizes the conversation into a compressed representation, preserving what the model learned early in the session. OpenAI uses a 175,000-token limit and compacts rather than truncates when context grows.

The specific API change:

// Before: Chat Completions, reasoning dropped each turn
POST /v1/chat/completions
{ "model": "gpt-5.6-sol", "messages": [...history...] }

// After: Responses API, reasoning retained via previous_response_id
POST /v1/responses
{
  "model": "gpt-5.6-sol",
  "previous_response_id": "<id from prior response>",
  "input": "<next user turn>"
}

OpenAI's Responses API handles compaction automatically when you chain responses via previous_response_id. You do not need to implement a summarization step yourself.

Why this is a routing-layer decision, not just a model choice

If your gateway forwards GPT-5.6 requests to the Chat Completions endpoint, you are not running the model the way OpenAI deploys it in ChatGPT and Codex. You are running a configuration that OpenAI explicitly identified as responsible for 3x lower agent performance.

This matters for routing teams in three concrete ways:

Endpoint routing: Chat Completions (/v1/chat/completions) and the Responses API (/v1/responses) are different endpoints with different context management semantics. A routing rule that treats them as interchangeable will silently send Responses API–optimized workloads to the wrong endpoint if fallback is misconfigured.

State threading: The Responses API uses previous_response_id to chain turns. A proxy or gateway that strips or fails to forward this field breaks the reasoning chain. If your middleware normalizes requests to a stateless format, every multi-turn GPT-5.6 agent call is degraded.

Token accounting: Retained reasoning tokens appear in usage counts but contribute to future turn efficiency. A cost model that treats reasoning token output as waste — and a routing rule that optimizes by minimizing it — is directly counterproductive here.

The provider comparison operators need to make

This configuration gap exists specifically at OpenAI because GPT-5.6's architecture separates private reasoning tokens from output tokens, and that separation requires explicit API plumbing to preserve across turns.

Anthropic's extended thinking model works differently: reasoning tokens are emitted within the same message structure and can be passed back in subsequent assistant turns, but the mechanism is through the thinking content blocks in Chat-style messages rather than a separate response-chaining ID.

DeepSeek's V4 Pro and Flash models support thinking mode but do not have a Responses API equivalent with automatic compaction — operators must implement their own context management.

Gemini's Responses-equivalent (the Gemini Agent Platform session API) has a different session-token architecture where memory is handled through sessions rather than response chaining.

The implication: a routing layer that provides a unified Chat Completions interface to downstream callers needs explicit policy for which features are emulated, which are lost, and which require the caller to use the native endpoint directly.

What the benchmark actually measured

OpenAI's post is explicit: "Benchmarks rarely measure AI models in isolation. They also measure less visible choices about API settings, harness design, and prompting."

The ARC-AGI-3 benchmark used rolling truncation and discarded reasoning, which is how most third-party eval harnesses work. The result was a 13.3% score that underrepresents the model. The same model with production-matching settings scored 38.3% — approaching the 48% estimated human baseline.

This is a correction of record that affects routing tier decisions. If you are selecting between providers based on public benchmark results and those benchmarks use default Chat Completions settings, you may be comparing provider A running optimally against provider B running in a degraded configuration.

What operators running GPT-5.6 agent workloads must audit now

For teams routing any multi-step or long-horizon agent workload to GPT-5.6:

• Endpoint check: Is your routing config sending agent workloads to /v1/responses or /v1/chat/completions? The distinction matters for reasoning retention.
• State forwarding: Does your gateway forward previous_response_id between turns, or does it reconstruct stateless requests from a local context buffer?
• Fallback behavior: If the Responses API returns an error and your gateway falls back to Chat Completions, does the fallback preserve or discard the reasoning state?
• Cost model: Reasoning token counts in the Responses API reflect deferred compute savings across subsequent turns. Optimizing purely on per-call token cost misrepresents the efficiency of retained-reasoning workloads.

OpenAI's recommendation for API teams: use the Responses API (not Chat Completions), retain reasoning via previous_response_id, and enable compaction. These are now the production-equivalent settings for GPT-5.6 agent workloads.