OpenAI Assistants API to Responses API Migration: Routing Architecture Guide

OpenAI's Assistants API shuts down on August 26, 2026. The replacement path is the Responses API: Assistants become versioned Prompts, Threads become Conversations, Runs become Responses, and Run steps become output Items. If you run production agents through a routing gateway, treat this as an architecture migration rather than a simple endpoint rename: inventory openai.beta.* usage, move new sessions to Responses first, translate thread history only where needed, retest tool loops and streaming, and keep fallback routing on supported chat/response paths.

This guide is evergreen depth for teams that already saw the deadline. It focuses on code-level migration and routing decisions, while staying careful about product claims: TheRouter routes OpenAI-compatible requests through configured providers and supports provider/model routing and fallback where the live product path supports it. We do not claim universal model support, zero downtime, or guaranteed cheapest routing.

30-second answer

• OpenAI says the Assistants API is deprecated and will shut down on August 26, 2026 after Responses API feature parity (OpenAI Assistants migration guide, retrieved 2026-06-21).
• The object mapping is: Assistants → Prompts, Threads → Conversations, Runs → Responses, and Run steps → Items (OpenAI Assistants migration guide, retrieved 2026-06-21).
• There is no automatic Threads-to-Conversations migration; OpenAI recommends moving new user chats to Conversations and backfilling old Threads only as necessary (OpenAI Assistants migration guide, retrieved 2026-06-21).
• Responses API file search is a hosted tool that uses vector stores and returns file-search call items plus message items with citations (OpenAI File search guide, retrieved 2026-06-21).
• For TheRouter users, keep routing claims scoped: /v1/responses exists in the repo and translates Responses-shaped requests through chat-compatible provider paths, but provider-specific hosted tools must be tested per provider before production rollout.

What changes from Assistants to Responses

The old Assistants API bundled behavior, state, and execution into persistent OpenAI-side objects. The Responses API separates those layers so application code and routing gateways have clearer boundaries.

OpenAI describes Responses as a simpler model: send input items and receive output items, optionally using prompt and conversation objects for stored configuration and context (OpenAI Assistants migration guide, retrieved 2026-06-21). That simplicity helps routing, but it also moves responsibility into your application: retries, pruning, tool execution, and rollout strategy should be designed deliberately.

Find every affected callsite first

Start with a code inventory. The risky patterns are usually easy to grep:

grep -R "openai\.beta\.assistants\|openai\.beta\.threads\|threads\.runs\|assistant_id\|thread_id" \
  app/ services/ workers/ scripts/

Then classify each usage:

1. New-session chat flows. Migrate these first because they do not require history conversion.
2. Long-lived user threads. Decide whether to backfill history into Conversations or close old sessions with a UX handoff.
3. Tool-heavy agents. Rewrite the tool loop explicitly and add integration tests.
4. Retrieval/file-search flows. Rebuild vector-store assumptions and citation handling around Responses file search.
5. Background jobs. Replace polling Runs with response lifecycle handling, including idempotency and retry controls.

Do not wait for a perfect one-shot conversion. The safer path is a dual-run window: new sessions on Responses, legacy Threads read-only or selectively backfilled, and monitoring that compares answer quality, tool-call rates, latency, and cost.

Before/after: a minimal chat flow

The Assistants pattern typically creates or reuses a Thread, appends a message, creates a Run, polls it, then fetches messages. Responses collapses that into a response call, with optional Conversation state.

Before: Assistants + Thread + Run

from openai import OpenAI
import time

client = OpenAI(api_key="<OPENAI_API_KEY>")

thread = client.beta.threads.create()
client.beta.threads.messages.create(
    thread_id=thread.id,
    role="user",
    content="Summarize the deployment risks.",
)

run = client.beta.threads.runs.create(
    thread_id=thread.id,
    assistant_id="asst_...",
)

while run.status in ("queued", "in_progress"):
    time.sleep(1)
    run = client.beta.threads.runs.retrieve(thread_id=thread.id, run_id=run.id)

messages = client.beta.threads.messages.list(thread_id=thread.id)
print(messages.data[0].content[0].text.value)

After: Responses API

from openai import OpenAI

client = OpenAI(api_key="<OPENAI_API_KEY>")

response = client.responses.create(
    model="gpt-5.5",
    input=[{
        "role": "user",
        "content": [{"type": "input_text", "text": "Summarize the deployment risks."}],
    }],
)

print(response.output_text)

When you need server-side context, create a Conversation and pass its ID on response calls. When you need application-owned context, store your own message/item history and send the relevant window each time. The important migration decision is not syntax; it is who owns state after Assistants disappears.

Migrating Threads to Conversations without over-migrating

OpenAI states that it will not provide an automated tool for migrating Threads to Conversations and recommends moving new user threads onto Conversations while backfilling old ones as necessary (OpenAI Assistants migration guide, retrieved 2026-06-21).

A practical policy:

• Do not backfill abandoned threads. Keep an archive and migrate only when a user reopens the session.
• Backfill text first. Convert user text to input_text and assistant text to output_text; handle images, files, and tool outputs in a second pass.
• Preserve metadata separately. Thread metadata and Conversation metadata are not a complete audit log. Keep your own mapping table: old_thread_id, new_conversation_id, user_id, migrated_at, migration_version.
• Mark lossy conversions. If a Run step, tool output, or file citation cannot be represented exactly, record that gap and avoid silently changing behavior.
• Test context-window pruning. Conversations can store item streams, but you still need a policy for cost, latency, and old tool artifacts.

A lightweight backfill skeleton:

from openai import OpenAI

client = OpenAI(api_key="<OPENAI_API_KEY>")

def backfill_thread(thread_id: str):
    items = []
    for page in client.beta.threads.messages.list(thread_id=thread_id, order="asc").iter_pages():
        for message in page.data:
            content = []
            for part in message.content:
                if part.type == "text":
                    content.append({
                        "type": "input_text" if message.role == "user" else "output_text",
                        "text": part.text.value,
                    })
            if content:
                items.append({"role": message.role, "content": content})

    return client.conversations.create(items=items)

Treat this as a starting point, not a complete converter. Multimodal inputs, file citations, code-interpreter artifacts, and tool outputs need additional mapping and review.

Tool calls and file search need the most testing

Assistants hid much of the execution loop behind Runs. Responses returns output items, including tool-call items, and your application sends tool results back. That makes behavior more explicit, which is better for observability, but it also means your retry logic can accidentally replay a tool call if you do not add idempotency.

For file search, OpenAI documents file_search as a hosted Responses API tool backed by vector stores. When the model calls it, the response includes a file_search_call output item and a message item with citations (OpenAI File search guide, retrieved 2026-06-21).

Migration checklist for tools:

1. Move tool schemas out of Assistant objects and into versioned prompt/config definitions.
2. Add an idempotency key to every external side-effecting tool call.
3. Log every tool-call item and tool-result item with the response ID.
4. Decide which tools may be retried automatically and which require human or application confirmation.
5. For file search, verify vector store IDs, metadata filters, citation rendering, and max-result settings.
6. For hosted tools such as web search, computer use, MCP, and file search, test the exact provider path you will use in production.

Routing-gateway implications

The migration changes where routing decisions live. In Assistants, model choice could be embedded in the Assistant object. In Responses, the model is part of the response request. That is better aligned with gateway routing because each request can carry a model ID, fallback policy, and provider constraints.

For TheRouter-style OpenAI-compatible routing, separate three layers:

This distinction prevents a common migration mistake: assuming that because text generation routes correctly, every OpenAI-hosted Responses feature also routes correctly. Hosted tools and stateful conversations are product-layer behavior, not just model inference.

Before/after: routing through TheRouter where supported

If your Responses workload is text-first and compatible with the routed path you have tested, the base URL pattern mirrors other OpenAI-compatible migrations.

from openai import OpenAI

client = OpenAI(
    base_url="https://api.therouter.ai/v1",
    api_key="<THEROUTER_API_KEY>",
)

response = client.responses.create(
    model="openai/gpt-5",
    input="Classify this support ticket and return JSON.",
)

For fallback on compatible chat-completion flows, TheRouter documents a models array in priority order: TheRouter attempts the first model and retries the next when the route fails due to downtime, rate limits, or moderation refusal (TheRouter Model Fallbacks, retrieved 2026-06-21). Use fallback carefully for agents: all fallback candidates should support the same output format and tool contract.

import OpenAI from 'openai';

const client = new OpenAI({
  baseURL: 'https://api.therouter.ai/v1',
  apiKey: process.env.THEROUTER_API_KEY,
});

const completion = await client.chat.completions.create({
  model: 'openai/gpt-5',
  extra_body: {
    models: ['anthropic/claude-sonnet-4.5', 'google/gemini-3.5-flash'],
  },
  messages: [{ role: 'user', content: 'Draft the migration test plan.' }],
});

Fallback is resilience, not a zero-downtime guarantee. If every candidate fails, or if a fallback model cannot satisfy the same tool/schema contract, the request should fail loudly.

Rollout plan before August 26

Use the deadline to work backward:

A rollback plan does not mean going back to Assistants forever; the API is shutting down. It means you can temporarily route a subset of traffic to a previously validated Responses implementation, disable risky tools, or fall back to a simpler chat workflow while debugging.

Testing checklist for routing teams

Before you declare the migration complete, test these cases end to end:

• Conversation continuity: multi-turn context survives across requests and user sessions.
• Tool idempotency: retries do not duplicate external side effects.
• Streaming: event parsing works for text deltas, tool calls, completion events, and errors.
• File citations: citations render correctly and link to the intended uploaded files.
• Fallback semantics: fallback candidates share the same tool schema, output schema, and safety constraints.
• Usage accounting: token usage and billing metadata are captured for both normal and fallback responses.
• Provider errors: rate limits, provider outages, moderation refusals, and unsupported-tool errors are classified separately.
• Observability: logs include response ID, conversation ID, model served, fallback position, and tool-call IDs.

Internal links for the broader migration

If this is part of a larger provider-routing project, pair this guide with:

• Migrate from OpenAI SDK to TheRouter for base URL, API key, and model-ID migration.
• LLM API fallback routing guide for fallback chain design.
• OpenAI prompt caching cost optimization guide for cost controls around repeated context.
• OpenAI provider page and OpenAI GPT-5 model page for current routed model availability.

FAQ

Is the Responses API just Chat Completions with a new name?

No. Responses uses input and output items, supports Conversations, and exposes hosted tools such as file search. Chat Completions remains a simpler message-in/message-out interface.

Can we keep using old Thread IDs after August 26?

No. Plan as if Assistants API Thread and Run endpoints stop working on the shutdown date. Store your own mapping from old Thread IDs to new Conversation IDs if you backfill active sessions.

Should every Assistants workload use Conversations?

Not necessarily. Stateless or short-lived workflows may be simpler with application-owned history passed into each Responses call. Use Conversations when provider-managed item history reduces complexity more than it adds lock-in.

Can fallback routing switch providers mid-conversation?

Sometimes, but do not assume it is safe for tool-heavy agents. Fallback candidates must share the same tool contract, output schema, and context assumptions. Test multi-turn and tool-call behavior before enabling fallback for production agent sessions.

Sources

• OpenAI Assistants migration guide, retrieved 2026-06-21.
• OpenAI Responses API reference, retrieved 2026-06-21.
• OpenAI File search guide, retrieved 2026-06-21.
• TheRouter Model Fallbacks, retrieved 2026-06-21.
• TheRouter OpenAI SDK migration guide, internal reference.