OpenAI's 'Useful Intelligence per Dollar' Scorecard: The Routing Policy Checklist Every AI Gateway Operator Needs

The metric your routing policy is probably optimized for — cost per token — is the wrong metric. OpenAI made this argument explicitly on July 17, publishing a CFO-facing scorecard that redefines AI value measurement as "Useful Intelligence per Dollar." For teams running AI gateways, the implications are immediate and concrete: your routing logic needs to catch up.

What OpenAI published

The scorecard introduces four measurements for evaluating AI ROI:

1. Useful work completed — tasks finished to an acceptable quality bar, not tokens consumed.
2. Cost per successful task — the full cost of reaching a correct outcome (model price + retries + latency + human review time), not cost per token.
3. Dependability rate — what share of outputs are "ready to use" versus "needs correction" or "needs escalation."
4. Value at scale — whether each AI dollar buys more work over time as models, infrastructure, and routing improve.

OpenAI uses this framework to position GPT‑5.6's tiered family (Sol/Terra/Luna) as a routing decision: Sol for workflows where strong reasoning reduces retries, Terra for balanced throughput, Luna for high-volume tasks where token cost dominates. The argument is that a "cheaper" model is only cheaper if its failure rate and retry overhead are accounted for.

Why it matters for AI engineering teams

The framework validates what experienced routing operators have known informally but rarely measured formally: model routing must be outcome-aware, not just price-aware.

A routing policy built only on cost-per-token will under-route to capable models for high-stakes workflows (where one correct answer beats five cheap failures) and over-route expensive models for simple high-volume tasks.

This creates several concrete changes engineering teams should make:

• Track completion rate by route, not just latency and cost. If a cheaper route fails 20% of the time and requires a retry, the effective cost is 1.2× the nominal cost, plus latency and UX degradation.
• Segment workflows by retry sensitivity. Customer support resolution, code generation, and contract review have different retry cost profiles. Your routing policy should encode this, not treat all tasks identically.
• Measure "escalation rate" as a routing signal. If a route triggers frequent human escalation, it is under-performing on the dependability dimension — route to a stronger model tier, or flag for eval.

The router/operator angle

OpenAI's scorecard is directed at CFOs, but it is a gift for routing operators. It gives you a framework to justify model-tier routing decisions to finance teams using their own language.

Translated into gateway operations:

The practical implication: routing policies that do not feed back from outcome data are stale from day one. Every gateway needs a mechanism to correlate request-level cost with downstream outcome signals — even a simple "was this retried?" flag starts this loop.

OpenAI's framing of "smarter routing" as a driver of compute ROI is not accidental. It signals that the company expects routing-layer differentiation — across model tiers, providers, and inference configurations — to be a primary lever for enterprise AI economics.

What to watch

• GPT‑5.6 tier selection. If you are routing to OpenAI, the Sol/Terra/Luna tiering requires an explicit model selection policy. Luna is not a safe default for code generation despite lower cost — Sol's reduced retry rate often wins on the "cost per successful task" metric.
• Eval-driven routing. The scorecard's dependability framing pushes toward evals that track "ready to use" vs "needs correction" rather than just benchmark accuracy. Teams without task-specific evals should treat this as a gap.
• Provider diversity. The scorecard logic applies across providers, not just GPT‑5.6. DeepSeek V4 Pro, Claude Fable 5 variants, and Qwen3.7 models all have different retry-cost profiles by task type. A multi-provider routing policy built around outcome rates — not just listed token prices — is the next maturity step.

What TheRouter users should watch or try

TheRouter routes requests across configured providers and records per-request cost and latency in the billing ledger. Teams building toward "cost per successful outcome" measurement can start by tagging retry requests with a X-TheRouter-Retry: true header and correlating those events against route cost in the usage docs.

The next eval step is task-specific pass rates by provider/model. If your team is moving toward outcome-based routing policies, the provider routing guides and fallback configuration are the relevant starting points.

The "Useful Intelligence per Dollar" framing is useful beyond OpenAI's own product line. It is the language CFOs will use to evaluate AI spend — and the language routing operators should use when proposing model tier policies to finance and leadership.